JPH08201041A - Method for recognizing three-dimensional shape - Google Patents

Abstract

PURPOSE: To provide a method for recognizing the three-dimensional shape where the processing for obtaining distance information is speeded up by simpli fying the Process for obtaining a parallax. CONSTITUTION: The image of a target is picked up by TV cameras 11a and 11b from two different positions and first and second images are obtained from the TV cameras 11 a and 11b. Representative extraction parts 13a and 13b detect the position of the same representative point of a target among first and second images and obtains the parallax between the first and second images by associating the representative point in the first image with that of the second image at a representative associating part 14. When the parallax is obtained, the triangulation method is applied at a three-dimensional position calculation part 15, thus obtaining the distance between the TV cameras 11a and 11b and the representative point. By combining the above distance onto the two-dimensional shape of a target obtained by either the first image or the second image, the three-dimensional shape of the target can be recognized.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention picks up one object from different positions by an image pickup means such as a TV camera,
The present invention relates to a three-dimensional shape recognition method for recognizing a three-dimensional shape of an object based on parallax between images obtained at each position.

[0002]

2. Description of the Related Art Conventionally, a T
The surface of the object is imaged by an image pickup means such as a V camera, and the center of gravity of the object is obtained in a binary image obtained by binarizing the image for the density, and the position of this center of gravity is set as the position of the object. The method is known. However, this method cannot recognize the distance between the imaging device and the surface of the object. Therefore, when the dimensions of the object in the image obtained by the imaging means are the same, the distance (that is, the height of the object). ) Is different, it cannot be recognized. In other words, when using this kind of method for recognition of goods in the production process,
There is a restriction that two types of articles having different height dimensions cannot be mixed.

On the other hand, as a method for recognizing a three-dimensional shape which enables recognition of the height of an article, Japanese Patent Laid-Open No. Sho 60 has been proposed.
The "stereo image association processing method" disclosed in Japanese Patent Publication No. 173403 is known. That is, after two images are obtained by using two image pickup means arranged in a prescribed positional relationship, the parallax between the two images is obtained, and the known positional relationship between the two image pickup means and the obtained parallax are obtained. Then, the triangulation method is applied to obtain the distance information between the object and the imaging device. The parallax is obtained by associating the line segments with each other using the line segments extracted from the line components of the two images.

[0004]

However, in the above-mentioned method, it is necessary to associate all the line segments in order to associate the two images with each other, which causes a problem that the time required for the parallax obtaining process becomes long. is there. The present invention has been made in view of the above circumstances, and an object thereof is to provide a three-dimensional shape recognition method in which the process of obtaining parallax is simplified and the process of obtaining distance information is speeded up.

[0005]

According to a first aspect of the present invention, the first image and the second image are picked up by the image pickup means at two different positions with respect to the object from a direction substantially orthogonal to the surface of the object, and 1 image and 2nd image for each 2
The two binarized images are obtained, and the positions of the same representative points on the surface of the object in the first image and the second image are detected based on the two binarized images. By associating the representative points in the second image with each other, the first
The parallax between the image and the second image is obtained, the distance information between the image pickup means in the direction orthogonal to the surface of the object is obtained by applying the triangulation method, and the object obtained from the first image or the second image It is characterized in that the three-dimensional shape of the object is recognized by combining the two-dimensional shape of the object with the distance information.

According to a second aspect of the invention, the representative point is the center of gravity of the object in the binary image. Claim 3
According to the first aspect of the present invention, when the center of gravity in the binary image obtained from the first image and the center of gravity in the binary image obtained from the second image are matched, the first method is based on the similarity of the densities around the respective centers of gravity.
It is characterized in that the image and the second image are associated with each other.

According to a fourth aspect of the present invention, the principal axis of inertia of the object in the first image and the second image is obtained, and the center of gravity in the binary image obtained from the first image and the binary value obtained from the second image are obtained. It is characterized in that the first image and the second image are associated with each other based on the degree of similarity in the direction of the principal axis of inertia when matching the center of gravity in the image. According to a fifth aspect of the invention, the optical axes of the image pickup means are parallel when the first image and the second image are picked up,
Further, the direction connecting the centers of the first image and the second image is parallel to the horizontal axes of the first image and the second image, and the enlargement ratio of the image capturing means for capturing the first image and the second image is set to be equal. The positions of the representative points in the first image and the second image are corrected so as to match the positions of the representative points at the time, and the first image and the second image are associated with each other based on the corrected positions of the representative points. It is characterized by

According to a sixth aspect of the present invention, the first image and the second image, which are grayscale images, are picked up by the image pickup means in two different positions with respect to the object from a direction substantially orthogonal to the surface of the object, and the first image is obtained. And the position of one representative point on the surface of the object in the first image and the second image is detected by pattern matching between the second image and the reference image, respectively. By ascertaining the parallax between the first image and the second image by associating the representative points with each other, and by applying the triangulation method, the distance information between the object and the image pickup means in the direction orthogonal to the surface of the object is obtained.
It is characterized in that the three-dimensional shape of the object is recognized by combining the distance information with the two-dimensional shape of the object obtained from the first image or the second image.

According to a seventh aspect of the present invention, the representative point is extracted from the second image by limiting the area where the representative point exists in the second image based on the position of the representative point obtained from the first image. . The invention of claim 8 is characterized in that after the first image is captured by the image capturing means, the same image capturing means is moved by a predetermined distance to capture the second image.

The invention of claim 9 is characterized in that the first image and the second image are imaged by different imaging means.
The invention of claim 10 is characterized in that a plurality of representative points are obtained for one object, and the shape of the object is recognized based on distance information of the plurality of representative points. The invention of claim 11 is characterized in that the first image and the second image are images obtained by extracting a specific color component from a color image.

[0011]

According to the present invention, when the shape of the object is known in the production process, the distance information about all the line segments is not necessary for recognizing the object. In consideration of the fact that there are many cases where there is such a case, the calculation for obtaining the parallax is easily performed without using the line segment.

According to a first aspect of the present invention, in the stereo image method, the positions of the same representative points on the surface of the object in the first image and the second image are respectively detected, and the first image and the second image are detected. By associating the above representative points in
Since the parallax between the image and the second image is obtained, as compared with the conventional process of associating all the line segments in the image, the object in the first image and the second image is compared. This is a simple process of associating representative points, and the process of obtaining distance information can be speeded up.

According to the invention of claim 2, since the center of gravity of the object in the binary image is used as the representative point, the representative point with good reproducibility can be obtained by a simple method. According to the invention of claim 3, not only the center of gravity of the object obtained from the first image and the second image is associated with the position information, but also the first image and the second image are based on the degree of similarity in density around each center of gravity. Since other information is also added to the position of the center of gravity, the first image and the second image can be more accurately associated with each other.

According to the invention of claim 4, not only the center of gravity of the object obtained from the first image and the second image is associated with the position information, but also the principal axis of inertia of the object in the first image and the second image is associated. Since the first image and the second image are associated with each other based on the similarity in the direction of, the amount of information is larger than that when only the coordinates of the center of gravity are used, so that the association can be performed more accurately. become.

According to a fifth aspect of the present invention, the positions of the representative points in the first image and the second image are set so that the optical axes of the image pickup means for picking up the first image and the second image are parallel to each other, and And the second
The position of the representative point when the direction connecting the centers of the images is parallel to the horizontal axes of the first image and the second image and the enlargement ratios of the image capturing means for capturing the first image and the second image are set to be equal to each other. Since the first image and the second image are associated with each other based on the position of the corrected representative point, the representative points can be more accurately associated with each other, and the reliability of the obtained distance information can be improved. The higher the quality.

According to a sixth aspect of the present invention, by pattern matching the first image and the second image with the reference image, the position of one representative point on the surface of the object in the first image and the second image is determined. Since the detection is performed, the process of obtaining the position of the representative point is simplified. According to the invention of claim 7, the representative point is extracted from the second image by limiting the area where the representative point exists in the second image based on the position of the representative point obtained from the first image. Since the representative points of the second image are extracted in a limited range based on the obtained information, the process of extracting the representative points in the second image is simplified, and the three-dimensional shape can be recognized with higher speed and accuracy. You can

The invention according to claim 8 is the first invention using the image pickup means.
After capturing the image, the same image capturing unit is moved by a predetermined distance to capture the second image, so that the three-dimensional shape can be recognized at low cost by using one image capturing unit. According to the invention of claim 9, since the two image pickup means are used, the first image and the second image can be obtained in a short time, and moreover, the positional relationship between both the image pickup means can be fixed. The parallax can be obtained accurately.

According to the tenth aspect of the invention, a plurality of representative points are obtained for one object, and the shape of the object is recognized based on the distance information of the plurality of representative points. Therefore, the object is recognized more accurately. be able to. According to an eleventh aspect of the present invention, the first image and the second image are images obtained by extracting the specific color component from the color image, and by using the color information, the representative points can be easily and accurately associated with each other, and the calculated distance can be obtained. Increased reliability of information.

[0019]

【Example】

(Embodiment 1) The apparatus used in the present invention comprises two TV cameras 11a and 11b, as shown in FIG. Both TV
As shown in FIG. 2, the cameras 11a and 11b have an optical axis A.
a and Ab are parallel and both TV cameras 11a and 11b
The line segments connecting the lens centers of are parallel to the horizontal direction of the image, and the TV cameras 11a and 11b are set to have the same enlargement ratio. Hereinafter, the arrangement of the TV cameras 11a and 11b set in this way will be referred to as a parallel stereo arrangement. The fields of view of both TV cameras 11a and 11b are set so as to include at least the representative point 2 of the object 1.

The images captured by the TV cameras 11a and 11b are stored in the image storage unit 12 such as a frame memory.
image storage units 12a, 1b, which are respectively stored in a and 12b.
Based on the images accumulated in 2b, the representative point extraction unit 13a,
The position of the representative point of the object 1 is extracted at 13b. In the present embodiment, the images captured by the TV cameras 11a and 11b are grayscale images, and the representative point extracting units 13a and 13b binarize the grayscales of the pixels and then extract the representative points. Here, for example, the center of gravity of the object 1 in the binarized image is used as the representative point.

When the representative point is extracted from the two images picked up by the TV cameras 11a and 11b, the representative point associating unit 14 associates the representative point 2 with each other, and the representative point 2 of the object 1 in each image is correlated. Match the same points. Here, it is known that the point corresponding to the representative point 2 of the object 1 in the image exists on the epipolar line determined by the arrangement of the two TV cameras 11a and 11b, and as described above, the TV camera 11a. , 11b in a parallel stereo arrangement,
The epipolar line is parallel to the horizontal axis of the image. Now, when the horizontal direction of the image is the X direction and the vertical direction is the Y direction,
In the images obtained by both TV cameras 11a and 11b, the Y coordinate of the point corresponding to the representative point 2 becomes equal. That is, assuming that each image is obtained as shown in FIG. 3, the Y coordinates of the representative point 2 in both images are the same as shown in FIG.

If the representative point associating unit 14 associates the representative points 2 as described above, the three-dimensional position calculating unit 15
Then, depending on the difference in the X coordinate of the representative point in both images, both TVs
The parallax of the representative point 2 viewed from the cameras 11a and 11b can be obtained. That is, assuming that the coordinates of the representative point 2 in each image are (x ₁ , y ₁ ), (x ₂ , y ₂ ), y ₁ = y ₂ as described above, and x ₁ and x ₂ The difference between the two corresponds to the parallax. Since the distance between the centers of the lenses of both TV cameras 11a and 11b and the focal length of the lenses are known, if the parallax is obtained, the triangulation method is applied to calculate the TV cameras 11a and 11b from the following equations. The distance between the object 1 and the representative point 2 can be obtained. Z = B × F / D (where, Z: the distance between the TV cameras 11a and 11b and the representative point 2 of the object 1, B: the distance between the centers of the lenses of the TV cameras 11a and 11b, F: the TV camera 11a, 11b lens focal length, D: parallax) Generally, in the production process, the surface on which the object 1 is arranged and T
Since the distance between the V cameras 11a and 11b is known, if the distance between the TV cameras 11a and 11b and the representative point 2 of the object 1 is obtained as described above, the distance from the reference plane on which the object 1 is arranged is determined. The height of the representative point 2 of the object 1 can be obtained.

In the production process, both TV cameras 11
Since the optical axes of a and 11b are orthogonal to the surface of the object 1 on which the representative point 2 is set, the three-dimensional position calculation unit 15
The shape and position of the object 1 can be recognized by sending the three-dimensional position of the representative point obtained in step 1 to the object recognition unit 16. As described above, the shape and position of the target object 1 can be recognized only by associating the representative point 2 with the target object 1 obtained by the stereo image method. Therefore, the processing is simple and the processing time is short.

(Embodiment 2) In the present embodiment, as shown in FIG. 5, the shape and position of the object 1 are recognized as in the case of Embodiment 1 (from the image accumulating units 12a and 12b in Embodiment 1 to the object). The recognition unit 16 is collectively shown as the three-dimensional recognition apparatus 10), and the robot 3 is controlled by using the result, so that the plurality of types of objects 1 ₁ and 1 ₂ are different trays 4 ₁ and 4 respectively. An example of classifying into ₂ and arranging them is shown. Each object ₁ 1, 1 _2, the height (i.e., TV camera 11a, the distance between 11b) differ only, other dimensions are equal.

As shown in FIG. 6, the TV cameras 11a, 1
Object 1 by 1b₁, 1₂Image of (S
1), both images are binarized (S2). Next, binarized
Object 1 from both images₁, 1₂Find the center of gravity of
(S3), by correlating the centroids obtained in both images
The parallax is obtained (S4). In this way parallax is calculated
Then, the three-dimensional position of the representative point 2 can be obtained (S
5). Here, object 1₁, 1₂The height of the
One object 1₁The height of Z₁If so, of the representative point
Height Z is Z₁Which object to match
1₁, 1₂You can recognize what. Therefore, Z = Z
₁It is determined whether or not (S6), Z = Z₁Is established
If the robot 3 is moved (S7), the object 1₁Grab
(S8), tray 4 ₁(S9). Also, Z =
Z₁If is not established, the robot 3 is moved (S1
0), object 1₂Grasp (S11), Tray 4₂Aligned to
Yes (S12).

As described above, a plurality of different heights are used.
Object of type 1₁, 1₂Sort and Tray 4₁, 4₂
Can be aligned. In this embodiment, two types
Object of class 1₁, 1₂Is explained, but further
If there are many types of objects, do the same.
Can be. Other processes are the same as those in the first embodiment. (Embodiment 3) This embodiment also has a plurality of types of pairs as in the second embodiment.
Elephant 1₁, 1₂The robots and sort them by the robot 3.
Tray 4₁, 4₂The following shows an example of aligning. However, the target
Thing 1 ₁, 1₂Is not only the height but also the plane containing the representative point 2.
The brightness of is different. Object 1₁, 1₂Surface light
In order to make it different₁, 1₂Surface of
You can set different reflectances.

Now, as shown in FIG. 8, one TV camera 1
It is assumed that there is one object in the field of view of 1a and two objects are imaged in the field of view of the other TV camera 11b. Here, when the Y coordinates of the representative points in each image are all the same, which of the two objects in the field of view of one TV camera 11b is the target object is identified only by the Y coordinate. I can't. Therefore, the difference in brightness of each object is used.

That is, binarization is performed as a representative point of each object.
Center of gravity G in the image₁, G₂, G ₃And each center of gravity G
₁, G₂, G₃Brightness of pixels in the vicinity of 8
Average value V₁, V₂, V₃Ask for. To the same object
Average brightness V₁, V₂, V₃Is almost equal to
Therefore, it is within the field of view of the other TV camera 11b.
Average brightness V for two existing objects₂,
V₃And an object within the field of view of the one TV camera 11a
Brightness average value V₁If you find the difference between
The smaller the center of gravity G₁, G₂, G₃Similarity of density around
Since it is considered that the degree of similarity is high, the pair with higher similarity is the target pair.
Elephant G₁, G₂, G₃It can be determined that
That is, | V₁-V₂|, | V₁-V₃The value of | is small
Center of gravity G₁, G₂, G₃By associating pairs of
Therefore, the three-dimensional position of the representative point of the object can be obtained.
It

After that, depending on the height, the object 1₁, 1₂The cut
Separate and Tray 4₁, 4₂To line up. that's all
The processing procedure of is summarized in FIG. That is,
Object 1 by TV camera 11a, 11b₁, 1₂of
Capture the image (S1) and binarize both images (S1)
2). Next, the object 1 is extracted from both binarized images.₁, 1₂of
Center of gravity G₁, G₂, G₃As a representative point
(S3), center of gravity G₁, G₂, G₃Brightness near 8
Average value V₁, V₂, V₃Is calculated (S4). Next, | V ₁
-V₂| And | V₁-V₃Find the magnitude relationship with | (S
5), | V₁-V₂| Is | V₁-V₃When less than
Has a center of gravity G₁And the center of gravity G₂And are representative points of the same object
Corresponding (S6), | V₁-V₂| Is | V
₁-V₃｜ When it is larger, the center of gravity G₁And the center of gravity G₃When
Are regarded as representative points of the same object and are associated (S
7). The subsequent processing is the same as that in the second embodiment, and the parallax is obtained.
The three-dimensional position of the representative point (S8), Z = Z₁Or not
Is determined (S9), Z = Z₁If
Object 3 is moved (S10).₁Grab (S
11), tray 4₁(S12). Also, Z =
Z₁If is not established, the robot 3 is moved (S1
3), object 1₂Grasp (S14), Tray 4₂Aligned to
Yes (S15).

As described above, since the representative points are associated with each other using the information on the brightness, the TV camera 1
Even if a plurality of objects 1 ₁ and 1 ₂ exist within the visual fields of 1a and 11b, there is a high probability that they can be associated with each other, and the reliability of the obtained three-dimensional position is high. In this embodiment, two types of object 1 _1, 1 ₂ has been described, can be treated in the same manner even if there are still many types of objects.

(Embodiment 4) This embodiment is also an example in the case where a plurality of objects are present in the visual field of one TV camera 11a, 11b as in the case of Embodiment 3, and is representative in Embodiment 3. Although the average value of the brightness in the vicinity of 8 points is used, in the present embodiment, the representative points are associated by comparing the directions of the principal axes of inertia of the objects.

That is, as shown in FIG.
Regarding the density of the images taken by the cameras 11a and 11b
In the binarized image, the principal axis of inertia B of the object₁, B₂, B
₃Direction (angle to the horizontal) θ₁, Θ₂, Θ₃To
Ask. For the same object, the principal axis of inertia B₁, B₂,
B₃Direction θ₁, Θ₂, Θ₃Should match. Shi
Therefore, one pair is now in the field of view of the TV camera 11a.
There is an elephant, and there are two pairs in the field of view of the TV camera 11b.
If there is an elephant, obtain it with the TV camera 11b.
Inertial principal axis B of each object in the image₂, B₃Direction θ₂,
θ₃And the object in the image obtained by the TV camera 11a
Principal axis of inertia B₁Direction θ₁The difference between
It can be considered that the ginger corresponds. That is, |
θ₁−θ ₂｜ and │θ₁−θ₃｜ seeking a size relationship with
Center of gravity G, which is the representative point for the Saihou group₁, G₂, G
₃Should be associated with each other.

The above processing procedure is summarized as shown in FIG.
Become. That is, the pair of TV cameras 11a and 11b
Elephant 1₁, 1₂Image (S1), both images 2
The value is converted (S2). Next, the object from both binarized images
1₁, 1₂Center of gravity G₁, G₂, G₃As a representative point
Along with the principal axis of inertia B₁, B₂, B₃Direction θ₁,
θ ₂, Θ₃Is calculated (S3). Next, | θ₁−θ₂｜ and
│θ₁−θ₃The magnitude relationship with | is obtained (S4), | θ₁−
θ₂| Is | θ₁−θ₃｜ When it is smaller than the center of gravity G₁
And the center of gravity G₂And are regarded as the representative points of the same object.
Response (S5), | θ₁−θ₂| Is | θ₁−θ₃From
When large, the center of gravity G₁And the center of gravity G₃Of the same object as
It is regarded as a representative point and associated (S6). That is,
Principal axis of inertia B₁, B₂, B₃Direction θ₁, Θ₂, Θ₃Kind of
High similarity means direction θ₁, Θ₂, Θ₃Is the difference
Because it is small, | θ₁−θ₂｜ and │θ₁
−θ₃The main axis of inertia B₁, B
₂, B₃Direction θ₁, Θ₂, Θ ₃Can determine the similarity of
Of. The subsequent processing is the same as that in the second embodiment, and the parallax
To obtain the three-dimensional position of the representative point (S7), Z = Z₁
It is determined whether or not (S8), Z = Z₁If is established
The robot 1 is moved (S9), and the object 1₁Grab
(S10), tray 4₁(S11). Also,
Z = Z₁If is not established, move the robot 3 (S
12), object 1₂Grasp (S13), Tray 4₂In order
Line up (S14).

As described above, since the representative points are associated with each other by using the directions of the principal axes of inertia of the objects, a plurality of objects 1 ₁ , 1 ₂ are included in the visual fields of the TV cameras 11a, 11b.
Even if there is, there is a high probability that matching will be possible,
The reliability of the obtained three-dimensional position becomes high. In this embodiment,
2 types of objects 1 _1, 1 ₂ has been described, can be treated in the same manner even if there are still many types of objects.

(Embodiment 5) In this embodiment as well, a plurality of types of objects are sorted, and the robot 3 is used to align the objects in a tray. In each of the above-described embodiments, the epipolar is used. This is an example of the case where the representative point exists on the line, but in the present embodiment, as shown in FIG. 12, the case where the representative point (centers of gravity G ₁ and G ₂ ) deviates from the epipolar line will be described. (Y ₁ ≠ y ₂ ). In short, both TV cameras 11a and 11b are slightly displaced from the parallel stereo arrangement. In reality, the two TV cameras 11a, 1
Since it is difficult to adjust the positional relationship of 1b so as to have a perfect parallel stereo arrangement, it is useful in actual use to assume a positional relationship that is slightly deviated from the parallel stereo arrangement.

Therefore, in this embodiment, as shown in FIG.
Sea urchin, obtained by imaging with each TV camera 11a, 11b
Center of gravity as a representative point of the object obtained from each image obtained
G₁, G ₂Coordinates of (x₁, Y₁), (X₂, Y₂) Based on
Then, both TV cameras 11a and 11b are arranged in parallel stereo.
Center of gravity G₁, G₂The coordinate position of (S
1 to S4). Such coordinate transformations are well known and are parallel scans.
It is called the Tereo transformation. Parallel stereo conversion
G for the center of gravity after₁′, G₂′, Each coordinate is (x₁′,
y₁′), (X₂′, Y₂′), Parallel stereo
Epipolar lines after conversion are parallel to the horizontal direction of the image
From y₁′ = Y₂'become. That is, x₁′ And x₂′
The parallax can be calculated based on the difference between
G₁′, G₂'Can be associated (S5). After that
Processing is the same as the processing procedure after step S5 of the second embodiment.
It is like.

As described above, when the TV cameras 11a and 11b are not arranged in parallel stereo, correction is performed by adding a procedure for performing parallel stereo conversion, so that the subsequent processing is performed in the same manner as in the case of parallel stereo arrangement. Can be done. The other processing procedure is the same as that of the second embodiment. (Embodiment 6) In each of the above-described embodiments, the center of gravity of the object in the binarized image is used as the representative point of the object, but such a representative point is displaced depending on the direction of illumination and the like. In some cases. Therefore, in this embodiment, the reference image set in advance for the reference object is used, and T
The position of the representative point is obtained by pattern matching between each image captured by the V cameras 11a and 11b and the reference image. Here, if the position of the representative point is defined in the reference image, the representative point may be any point on the object.

In the pattern matching process, as shown in FIG. 13, the reference image is pattern-matched with the images picked up by the TV cameras 11a and 11b, and the position where the degree of similarity is maximum is obtained (S1, S2). S3).
Since the representative point is set in the reference image, the position of the representative point is obtained from the position obtained by pattern matching, and the parallax is obtained by associating the representative points obtained from both images with each other (S4). Subsequent processing is the same as the processing procedure from step S5 onward in the second embodiment, and after determining the three-dimensional position of the representative point, the types of objects are sorted and aligned on the tray.

(Embodiment 7) This embodiment is a method for obtaining the position of the representative point of the object in each image by pattern matching with the reference image, as in the case of the embodiment 6. However,
Based on the finding that if the position of the representative point is obtained for one image, the Y coordinate of the representative point of the other image and the Y-coordinate of the representative point obtained from the one image substantially match in the case of parallel stereo arrangement. By limiting the range in which the representative point is searched when obtaining the representative point from the other image, the processing speed is increased.

That is, as shown in FIG. 15, after obtaining the coordinates of the representative point from one image (the image on the left side) and then obtaining the representative point from the other image (the image on the right side), within the range T shown in the figure. By performing pattern matching while limiting the Y coordinate, the position where the degree of similarity with the reference image is maximized can be extracted in a short time as compared with the case where pattern matching is performed in the entire area of the image. Of.

The processing procedure described above can be summarized as shown in FIG. 14, and the position of the representative point is obtained by pattern matching with one of the images captured by the TV cameras 11a and 11b with the reference image (S1, S2). ) After that, the area of the Y coordinate for pattern matching with the reference image in the other image is limited (S3), and pattern matching with the reference image is performed within the limited range to extract the representative point ( S4). As described above,
The process after extracting the representative points from both images is the same as that of step S5 and subsequent steps of the second embodiment, and the parallax is obtained from the difference between the X coordinates of the representative points to obtain the three-dimensional position of the representative points, and the objects are sorted. Then align them on the tray.

(Embodiment 8) In each of the above embodiments, two TVs are used.
Although the cameras 11a and 11b were used, in this embodiment, FIG.
As shown in FIG. 7, one TV camera 11 is used and the object 1
By relatively moving the TV camera 11 and the TV camera 11, the same processing as in the case of using the two TV cameras 11a and 11b is enabled. Actually, the TV camera 11
Is fixed and the object 1 is moved, and when the workpiece is the object in the production process or the like, since the workpiece is transported, it is practical to adopt such a processing method. There is a nature.

That is, as shown in FIG. 16, the object 1 is imaged by the TV camera 11 (S1), and the object 1 is moved in the known direction by the known movement amount (S2).
The image of the object 1 is captured again by the TV camera 11 (S
3). In this way, the images at each position of the object 1 can be stored in the image storage units 12a and 12b.
Subsequent processing is the same as step S2 and subsequent steps in the second embodiment.

(Embodiment 9) In each of the embodiments described above, one
Although one representative point was used for each target object of
In the example, multiple representative points are used for one object
Here is an example: For example, the object 1 is shown in FIG.
Sea urchin, multiple (here three) pins C₁, C₂, C₃
Are two types of articles with
C₁, C ₂, C₃Assume that the heights of the different. Both
About goods Pin C₁, C₂, C₃Dimensions excluding the height of
Shall be equal. Therefore, each pin C₁, C₂, C₃of
If a representative point is set on the top surface,
There are three representative points for (object)
Become. Images taken by TV cameras 11a and 11b
(S1 in FIG. 18), the density is binarized with an appropriate threshold.
By doing (S2 in FIG. 18), as shown in FIG.
On pin C₁, C₂, C₃Extract only the top surface of
Can be. Therefore, each pin C₁, C₂, C₃On the top of
Center of gravity G₁~ G₆Is extracted and used as a representative point (see FIG. 18).
S3). Center of gravity G₁~ G₆The correspondence is the same as in the second embodiment.
By assuming a planar stereo arrangement, each
Center of gravity G₁~ G ₆Be matched on the epipolar line.
You can That is, the corresponding center of gravity G ₁~ G₆Is the same Y
Since it has coordinates, it can be easily associated
(S4 in FIG. 18).

Center of gravity G₁~ G₆After associating
, The corresponding center of gravity (G₁, G_Four), (G₂, G_Five),
(G₃, G₆) Of the three representative points from the parallax
A three-dimensional position is obtained (S5 in FIG. 18). In addition,
As in step S6, the pattern of the height data of the three representative points is arranged.
Memory Pa, and in step S7, the arrangement pattern is stored.
Object 1 that is registered in advance with Pane Pa₁Height of
Data arrangement pattern P ₁By comparing and
You can identify the type of elephant. In other words,
The processing is the same as the processing after step S7 in the second embodiment.
It is sorted and arranged in trays according to the type of object.
Can be made.

(Embodiment 10) In the present embodiment, TV cameras 11a and 11b capable of capturing a color image are used, and the object 1 has a representative point colored in a color different from other parts. It is provided. Now, assuming that the color of the representative point is red, as shown in FIG.
Imaging is performed with a and 11b (S1), and only the red component is extracted from the image of the object 1 (S2). Since the representative point is the center of gravity of the red component, the center of gravity of the red component is extracted (S
3) The extracted centroids are associated (S4). The subsequent processing is the same as the processing after step S5 in the second embodiment.

[0047]

According to the invention of claim 1, in the stereo image method, the positions of the same representative point on the surface of the object in the first image and the second image are respectively detected, and the first image and the second image are detected.
Since the parallax between the first image and the second image is obtained by associating the representative points in the image with each other, the first image can be compared with the conventional process of associating all the line segments in the image. This is a simple process of associating the representative points of the target object in the second image with those of the second image, and has an advantage that the process of obtaining the distance information can be speeded up.

According to the second aspect of the present invention, since the center of gravity of the object in the binary image is used as the representative point, the representative point having good reproducibility can be obtained by a simple method.
The invention of claim 3 not only correlates the center of gravity of the object obtained from the first image and the second image with position information, but also
The first image and the second image based on the similarity in density around each centroid.
Since the images are associated with each other, there is an advantage that the first image and the second image can be more accurately associated with each other by adding other information to the position of the center of gravity. According to the invention of claim 4, not only the center of gravity of the object obtained from the first image and the second image is associated with the position information, but also the direction of the principal axis of inertia of the object in the first image and the second image is determined. Since the first image and the second image are associated with each other based on the degree of similarity, there is an advantage that the amount of information is larger than that when only the coordinates of the center of gravity are used so that the association can be performed more accurately. .

According to a fifth aspect of the present invention, the positions of the representative points in the first image and the second image are set such that the optical axes of the image pickup means for picking up the first image and the second image are parallel, and And the second
The position of the representative point when the direction connecting the centers of the images is parallel to the horizontal axes of the first image and the second image and the enlargement ratios of the image capturing means for capturing the first image and the second image are set to be equal to each other. Since the first image and the second image are associated with each other based on the position of the corrected representative point, the representative points can be more accurately associated with each other, and the reliability of the obtained distance information can be improved. It has the effect of increasing the property.

According to the sixth aspect of the invention, the position of one representative point on the surface of the object in each of the first image and the second image is determined by pattern matching the first image and the second image with the reference image. Since the detection is performed, there is an advantage that the process of obtaining the position of the representative point becomes simple. According to the invention of claim 7, the representative point is extracted from the second image by limiting the area where the representative point exists in the second image based on the position of the representative point obtained from the first image. Since the representative points of the second image are extracted in a limited range based on the obtained information, the process of extracting the representative points in the second image is simplified, and the three-dimensional shape can be recognized with higher speed and accuracy. It has the advantage that

According to the invention of claim 8, the image pickup means is used.
After capturing the image, the same image capturing unit is moved by a predetermined distance to capture the second image, which is advantageous in that the three-dimensional shape can be recognized at low cost using one image capturing unit. According to the invention of claim 9, since the two image pickup means are used, the first image and the second image can be obtained in a short time, and moreover, the positional relationship between both the image pickup means can be fixed. There is an advantage that the parallax can be obtained with high accuracy.

According to the tenth aspect of the present invention, a plurality of representative points are obtained for one object, and the shape of the object is recognized based on the distance information of the plurality of representative points. Therefore, the object can be recognized more accurately. The advantage is that According to an eleventh aspect of the present invention, the first image and the second image are images obtained by extracting the specific color component from the color image, and by using the color information, the representative points can be easily and accurately associated with each other, and the calculated distance can be obtained. There is an advantage that the reliability of information becomes high.

[Brief description of drawings]

FIG. 1 is a block diagram showing a first embodiment.

FIG. 2 is an operation explanatory diagram of the first embodiment.

FIG. 3 is a diagram showing an image captured by a TV camera according to the first embodiment.

FIG. 4 is a diagram showing an image after binarization in the first embodiment.

FIG. 5 is a schematic configuration diagram showing an example of an apparatus according to a second embodiment.

FIG. 6 is an operation explanatory view showing the processing procedure of the second embodiment.

FIG. 7 is an operation explanatory view showing the processing procedure of the third embodiment.

FIG. 8 is a diagram showing an image after binarization according to a third embodiment.

FIG. 9 is an operation explanatory diagram illustrating a processing procedure of the fourth embodiment.

FIG. 10 is a diagram showing an image after binarization in the fourth embodiment.

FIG. 11 is an operation explanatory view showing the processing procedure of the fifth embodiment.

FIG. 12 is a diagram showing an image after binarization in a fifth example.

FIG. 13 is an operation explanatory diagram illustrating a processing procedure of the sixth embodiment.

FIG. 14 is an operation explanatory view showing the processing procedure of the seventh embodiment.

FIG. 15 is a diagram showing an image after binarization in Example 7.

FIG. 16 is an operation explanatory view showing the processing procedure of the eighth embodiment.

FIG. 17 is a conceptual explanatory diagram of the eighth embodiment.

FIG. 18 is an operation explanatory view showing the processing procedure of the ninth embodiment.

FIG. 19 is a perspective view showing the shape of an object recognized in Example 9.

FIG. 20 is a diagram showing an image after binarization in Example 9.

FIG. 21 is an operation explanatory view showing the processing procedure of the tenth embodiment.

[Explanation of symbols]

1 Target 1 ₁ Target 1 ₂ Target 2 Representative Point 3 Robot 4 ₁ Tray 4 ₂ Tray 11a TV Camera 11b TV Camera 12a Image Storage Section 12b Image Storage Section 13a Representative Point Extraction Section 13a Representative Point Extraction Section 14 Representative Point Correspondence Attaching part 15 3D position calculating part 16 Object recognition part G ₁ centroid G ₂ centroid G ₃ centroid G ₄ centroid G ₅ centroid G ₆ centroid

Claims (11)

[Claims] 1. A first image and a second image are picked up by an image pickup means at two different positions with respect to the object from a direction substantially orthogonal to the surface of the object, and the first image and the second image are respectively set to 2 in terms of density. Find two binarized images, 2
By detecting the positions of the same representative point on the surface of the object in the first image and the second image based on the two binary images, respectively, and by associating the representative points in the first image and the second image with each other. Calculate the parallax between the first image and the second image,
By applying the triangulation method, the distance information between the object and the imaging means in the direction orthogonal to the surface of the object is obtained,
A method for recognizing a three-dimensional shape, characterized by recognizing a three-dimensional shape of an object by combining the distance information with the two-dimensional shape of the object obtained from an image or a second image. 2. The three-dimensional shape recognition method according to claim 1, wherein the representative point is the center of gravity of the object in the binary image. 3. The first image based on the similarity in density around each centroid when matching the centroid in the binary image obtained from the first image with the centroid in the binary image obtained from the second image. The method for recognizing a three-dimensional shape according to claim 2, characterized in that the second image and the second image are associated with each other. 4. An inertial principal axis of an object in the first image and the second image is obtained, and a center of gravity in the binary image obtained from the first image and a center of gravity in the binary image obtained from the second image are calculated. The method for recognizing a three-dimensional shape according to claim 2, wherein the first image and the second image are associated with each other based on the degree of similarity in the direction of the principal axis of inertia when they match. 5. The first image and the second image are arranged such that the optical axes of the image pickup means are parallel to each other when the first image and the second image are picked up and the centers of the first image and the second image are connected to each other. Of the first image and the second image so as to be parallel to the horizontal axis of the first image and to match the position of the representative point when the enlargement ratios of the image capturing means for capturing the first image and the second image are set to be equal. The point position is corrected, and the first image and the second image are associated with each other based on the corrected position of the representative point.
The method for recognizing a three-dimensional shape according to claim 4. 6. A first image and a second image, which are grayscale images, are picked up by an image pickup means from two directions different from each other in a direction substantially orthogonal to the surface of the target object, and the first image and the second image are taken. The position of one representative point on the surface of the object in each of the first image and the second image is detected by the respective pattern matching with the reference image, and the representative points in the first image and the second image are associated with each other. By so doing, the parallax between the first image and the second image is obtained, and by applying the triangulation method, the distance information between the object and the imaging means in the direction orthogonal to the surface of the object is obtained.
A method for recognizing a three-dimensional shape, characterized by recognizing a three-dimensional shape of an object by combining the distance information with the two-dimensional shape of the object obtained from an image. 7. The second region is limited by limiting the area where the representative point exists in the second image based on the position of the representative point obtained from the first image.
The method for recognizing a three-dimensional shape according to any one of claims 1 to 6, wherein a representative point is extracted from the image. 8. The method according to claim 1, wherein after the first image is captured by the image capturing means, the same image capturing means is moved by a predetermined distance to capture the second image. The three-dimensional shape recognition method described. 9. The method for recognizing a three-dimensional shape according to claim 1, wherein the first image and the second image are imaged by different imaging means. 10. The method according to claim 1, wherein a plurality of representative points are obtained for one object, and the shape of the object is recognized based on distance information of the plurality of representative points.
The method for recognizing a three-dimensional shape according to any one of 1. 11. The three-dimensional shape recognition method according to claim 1, wherein the first image and the second image are images obtained by extracting a specific color component from a color image.