JPH0534602B2 - - Google Patents

Description

[Detailed Description of the Invention] Technical Field of the Invention The present invention relates to an object position and shape recognition method that automatically recognizes the three-dimensional position, posture, and shape of a work target object in an automatic machine such as a robot. .

Background of the technology Conventional methods of this type involve inputting an object as a two-dimensional grayscale image using ITV Camra, etc.
After pre-processing, the object is recognized by interpretation using artificial intelligence, the object is recognized by the principle of triangulation based on the angle of sight using optical sensors such as two ITV cameras, and the laser tracker is used. Method,
There was a method of projecting slit light, and a method of determining the distance to an object using light reflected from the object using a light emitting element and a light receiving element.

Prior Art and Problems Several conventional methods of this type and the problems of each method will be described below.

The method of representing the first object as a two-dimensional grayscale image often requires prior knowledge of the object, and the preprocessing may not be complete, there may be noise or shadows, or the object may have a complex shape. However, in the case of a conventional method, the object cannot be recognized accurately, and the amount of processing is large, which increases the processing time and hardware size. In addition, the method of binarizing an image at a certain density level as preprocessing and using this to recognize objects is widely used industrially, but There are strong restrictions regarding applicable conditions, such as being limited to objects, and it has the disadvantage that three-dimensional objects cannot be accurately recognized in any environment.

FIG. 1 shows the second method using the line of sight angle using two optical sensors. 101 is a light, 102 is a fixed mirror, 103 is a movable mirror, 104 is a light detection sensor, and 105 is a correlator. Light 101 from an object is transmitted to a fixed mirror 102 and a movable mirror 10.
3 and guided to the photodetection sensor 104, the movable mirror 103 is rotated, and the distance to the object can be determined by the principle of triangulation from the point where the correlation between the left and right images is maximum by the correlator 105. The disadvantage is that it is difficult to detect correlations and the accuracy of object recognition is not necessarily high.

A third method using a laser tracker is shown in FIG. 106 is a laser beam, 107 is an object, 10
8 is the image day sector, 109 is the background, 110
is the protrusion of the object. A laser beam 106 is irradiated onto an object 107 from a laser light source (not shown), and the reflected light is received by an image dissector 108, and the reflection point A on the object is determined from the two-dimensional coordinate output and the inclination of the laser beam 106 from the reference axis. (X, Y, Z)
This is to find the coordinates. In other words, if this laser beam 106 is scanned two-dimensionally and the Y coordinate of the reflection point A corresponding to each scan is used, the outline of the object can be distorted due to the discontinuity of the Y coordinate that occurs when the background 109 is irradiated. Objects with complex shapes that can be recognized, but become undetectable if there is a protrusion 110 as shown by the dotted line that blocks the laser beam 106, or boundaries or cracks when two objects completely overlap cannot be detected. The drawback was that it was not suitable for recognition.

FIG. 3 shows the principle of the method of projecting the fourth slit light. 111 is slit,
112 is a light source, 113 is a slit light, 114 is an object, and 115 is a TV camera. slit 111
When irradiated with the light source 112, a slit light 113 is obtained.The slit light 113 is irradiated onto an object 114 while changing the angle little by little, and the slit light 113 is
Objects are recognized by inputting a group of patterns from the TV camera 115 and analyzing the breaking points, discontinuities, etc. of the slit light 113, but the recognition accuracy is not high due to the boundaries of overlapping objects and object surfaces. The drawback was that it was not suitable for crack detection or recognition of objects with complex shapes.

FIG. 4 shows a method in which a fifth object is irradiated with light and light is reflected from the object. 116 is a light emitting element, 117 is a light receiving element, 118 is an object, 119 is a detection area, and 120 is an arm. Light emitting element 11
6 irradiates the object 118 with light, and the light receiving element 117 detects the reflected light. Object 118 is detection area 11
9, the light can be detected by the light-receiving element 117, so the arm 120 to which the sensor is attached is controlled so that the object 118 can be detected at all times, but the detection range is limited and the entire object 118 must be scanned. This method has disadvantages in that it takes time and the delicate shape of the object 118 cannot be detected.

Purpose of the Invention In order to eliminate these drawbacks, the present invention extracts candidate outlines of the object from a two-dimensional image of the object input by a visual sensor, and then arranges a plurality of light emitting elements on the same circumference. , a controllable light projection mechanism projects light near a candidate outline of the object so that the emitted light beam passes through the center of the circle and focuses on an arbitrary point on a straight line perpendicular to the circle. Based on the pattern of light spots formed on the object, it is determined whether it is an object or not, and the inclination of the object surface and the distance to the object surface are detected.The drawings will be described in detail below.

Embodiment of the Invention FIG. 5 shows an outline of the configuration of an embodiment for explaining the present invention. 1 is a light projecting mechanism, 2 is a visual sensor such as a TV camera, 3 is an object to be recognized, and 4 is a light spot. 6a and 6b show an example of the structure of the light projecting mechanism 1 including an explanation of its operation, 5 is a light emitting element, 6 is a light beam, and 7 is a circle in which the light emitting element 5 is arranged. is a line perpendicular to the circle on which the emitted light rays of are focused. The light-emitting elements 5 are end faces of a plurality of optical fibers guided from a light-emitting source such as a laser beam or a single light source, and a plurality of light-emitting elements 5 are arranged on the same circumference. The larger the number, the more desirable. The apex angle of the cone and the distance l to the condensing point can be controlled so that the light ray 6 emitted from the light emitting element 5 is condensed at an arbitrary point on the line 7.

In the present invention, first, a grayscale image of an object 3 is captured by a visual sensor 2 such as a TV camera shown in FIG. Furthermore, by operating a differential operator or a high-pass filter, a sudden change in density is extracted as a candidate outline of the object. An example of part A of object 3 is shown in FIG. In the example 10×10 mesh, pixels painted black are candidate pixels. In Figure 7, the right side (part B) and the left side (part C) of the candidate pixel row
You cannot know which one is an object without prior knowledge. Therefore, the light projecting mechanism 1 projects light onto portions B and C in FIG. First, when light is projected onto part B, it generally does not focus on one point, but may overlap with a slight deviation as shown in Figure 8a, or be completely separated as shown in Figure 8b. become. Therefore, the light emitting element 5 is controlled to focus the light on one point.
In this process, a suitable laser light filter or the like is installed in front of the visual sensor 2, and only spot light is taken into the visual sensor 2, and the light emitting elements 5 are controlled so that the sum of the areas is minimized. In this way, the distance to the part shown in FIG. 7B can be detected. Next, while keeping the light emitting element 5 in that state, the light projection mechanism 1 is moved in parallel to the position corresponding to C, and light is projected onto the portion C. In the case of Figure 5, part C corresponds to the background, so it is generally located far away from object 3.
Will the spot light pattern be as shown in Figure 8b?
Or the reflected light will not be detected. Therefore, it can be seen that the contour line candidate pixels in FIG. 7 are the actual contours, and furthermore, it can be seen that B is the object and C is the background. In this way, since the judgment is made based on the sum of the areas of the spot lights, the processing is not affected by the color of the object or the brightness of the illumination. In addition, the contour line candidate pixels are
If it is noise, a boundary line between shadows, characters or symbols written on an object, cracks on the object surface, or a boundary line where objects overlap, the pixels on both sides of the outline candidate pixel are The spot pattern does not change, and it can be determined that the contour candidate pixels are not contours. Conversely, even if the outline candidate pixels are interrupted, it is possible to confirm whether or not the outline candidate pixels are truly interrupted.

The present invention can be used to know the direction of the object plane. FIG. 9 shows outline candidate pixels of the A' portion of object 3 in FIG. 5. 8 to parts B', C', and D' in Figure 9.
When the light spot group shown in Figure b is projected, Figure 10 a~
A spot pattern as shown in c is obtained. That is, since patterns with different shapes can be obtained depending on the direction of the surface, it is possible to use this to know the direction of the surface. Furthermore, even for an object composed of a curved surface such as the cup shown in FIG. 11, by generating the spot pattern described above, the local direction of the surface can be determined, and the overall curved surface can also be recognized.

As described above, based on the grayscale image information of the object,
By focusing on the areas where the concentration suddenly changes and emitting the light as described above only to the necessary areas or areas to be confirmed, it is possible to know the distance to the object, the direction of the object surface, etc., so you can know the object beforehand. Even without it, objects can be recognized at high speed. Furthermore, since a sudden change in the density image of an object is used, it is also possible to detect a boundary where two objects completely overlap or a crack on the object surface.

FIG. 12 is a diagram showing the main parts of an embodiment of the light projecting mechanism 1 of the present invention. The same reference numerals as in FIG. 6 indicate the same parts. 8 is a ball screw, 9 is a casing, 1
0 is the motor, 11-1, 11-2 are the joints,
12 is a link, 13 is a link equipped with a light emitting element, 14
1 is a mounting pin, 15 is a cover, 16 is an optical fiber, and 17 is a light source. In operation, first, the casing 9 on the ball screw 8 is moved to the corresponding position by rotating the motor 10 based on the information shown in FIG. At this time, a link 12 is connected to the casing 9 via a joint 11-1, and the angle of the light projecting element mounting link 13 can be changed about the pin 14 via another joint 11-2. The link 13 is attached to the cover 15 via a number of pins 14, as shown in FIG. A light emitting element 5 is attached to each link 13, and the light emitting elements 5 are arranged on the same circumference. In FIG. 12, the end of the optical fiber 16 is used as the light emitting element 5, and the optical fiber 16 is connected to a light source 17, so that a large number of light emitting elements 5 can appropriately project light.

Effects of the Invention As explained above, according to the present invention, the position and shape of an object can be determined from a gray-scale image obtained from a visual sensor such as an ITV camera, distance information obtained from a light projection mechanism, and information about the direction of the object surface. Objects can be recognized regardless of noise, shadows, image deterioration, symbols on the object surface, etc. By combining a dedicated circuit that extracts contour line candidates from grayscale images, object recognition with short processing time is realized. can,
By configuring the visual input system with an image guide made of a bundle of optical fibers, the light projecting mechanism can be made compact, so there is an advantage that the overall device structure can be made small and lightweight.

The present invention is effective as a visual sensor for automatic machines, especially industrial robots, intelligent robots, etc. That is, it can be effectively used as a visual sensor separate from the robot's arm or as an arm-mounted sensor.

[Brief explanation of the drawing]

Figure 1 shows an object recognition method using two visual sensors, Figure 2 shows an object recognition method using a laser tracker, Figure 3 shows an object recognition method using slit light, and Figure 4 shows an object recognition method using reflected light. The detection sensor method is a conventional object recognition method, FIG. 5 is a schematic diagram of the configuration of an embodiment explaining the present invention, and FIGS. 6 a and b.
7 shows an example of the configuration of the light projecting mechanism according to the present invention, FIG. 7 shows an example of object outline candidate pixels, FIG. 8 a and b shows an example of a spot pattern, FIG. 9 shows an example of object outline candidate pixels, and FIG. a, b, and c are examples of spot patterns corresponding to the directions of the object surface, FIG. 11 is an example of detecting the direction of a curved surface by changing the spot pattern of a curved object, and FIG. 12 is an example of a light projecting mechanism. 101...Light, 102...Fixed mirror, 103
...Movable mirror, 104...Light detection sensor, 10
5... Correlator, 106... Laser light, 107...
Object, 108... Image desector, 109...
... Background, 110 ... Protrusion of object, 111 ... Slit, 112 ... Light source, 113 ... Slit light, 114 ... Object, 115 ... TV camera, 1
16... Light emitting element, 117... Light receiving element, 118
...Object, 119...Detection area, 120...Arm, 1...Light projection mechanism, 2...Visual sensor, 3...
Recognition target object, 4... light spot, 5... light emitting element, 6... light ray, 7... straight line passing through the center of the circle in which the light emitting element 5 is arranged and perpendicular to the circle, 8 being a ball screw, 9... Casing, 10...Motor, 11
-1, 11-2...Joint, 12...Link, 13...Light emitter 5 mounting link, 14...Mounting pin, 15...Cover, 16...Optical fiber, 17...Light source.

Claims (1)

[Claims] 1 In order to automatically recognize the position, posture, and shape of the work target object in automatic machines such as robots,
Using a plurality of light emitting elements arranged on the same circumference, a conical beam is irradiated so that the apex is on a straight line perpendicular to the circle, and a pattern of light spots formed on the surface of the object is detected. In a method for recognizing the distance to an object, the shape and orientation of the object,
A casing that moves on a straight line perpendicular to the circle and a link mechanism to which the light emitting element is attached are coupled via a rotatable joint, and by moving the casing, the apex angle of the conical beam is controlled. A method for recognizing the position and shape of an object, which comprises measuring the distance from the apex angle to the object that minimizes the area of a light spot formed on the surface of the object.