JPS5988297A - Visual device for robot - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed Description of the Invention] [Field of Application of the Invention] The present invention detects an optical image of a target object and analyzes it to detect the position and orientation of the target object.
The present invention relates to a robot visual device in which a robot's arm mechanism is controlled based on the detection results.

[Prior art]

Conventionally, the method of detecting the position of an object is to uniformly illuminate the entire object, capture this optical image with a 'vV camera, and obtain II! II image is binarized, and this two
Methods for processing value images have been widely used. but,
In this method, the color of the object was significantly different from the background, and it was necessary that only the object could be separated and extracted from the image through binarization. Additionally, this method cannot detect the position in the depth direction, so when a robot must grasp an object or attach parts to it, such as in assembly work, the object must be located at a known distance from the detector. Therefore, the scope of work was limited.

In addition, for automatic welding by robots, the 11th International Symposium on Industrial Robots (11th Int, Symp, on In
Industrial Robots (1981)), pages 151 to 158, an optical cutting detection head is attached to the robot's hand, and the detected optical cutting waveform is processed to automatically determine the welding position. There was a way to detect it. However, this method cannot be used for assembly work because the work target is limited to groove-shaped objects.

Furthermore, an attempt was made to apply this method to assembly work at the 9th International Symposium on Industrial Robots (<J th Int, 8 years
rpon Industrial also robots (
1979) ) published on pages 213 to 230
(G, J, Van der Brug) and others.

A Vision 5ystetn for
Real Time Control of IL
NBS proposed in a paper entitled robots
There is a method. In order to completely detect the position and orientation of an object three-dimensionally using this method, it is necessary to move the robot arm carrying the detector and view the object from several different square boxes. However, there was a drawback that detection took a long time.

The object of the present invention is to eliminate the above-mentioned drawbacks of the prior art and to provide a robot hand that can rapidly detect the three-dimensional position and predominance of an object without being affected by the object's color. The objective is to provide a compact and universal viewing device.

[Summary of the invention]

In order to achieve the above object, the robot vision device according to the present invention includes a light source that generates two non-parallel flat light beams, a light source switching device that switches between the two flat light beams to emit light, and a light source that generates two flat light beams that are not parallel. Converts the optical image of the line of intersection between the light beam and the surface of the target object into an electrical signal, and is on a plane that bisects the angle formed by the two planes formed by the two flat light beams, and An imaging device mounted on the robot's arm that is not on the intersection line of two planes, a device that separates and extracts bright lines on the number of strokes obtained by the imaging device, and a woman that is separated and extracted in this way. The gist of the present invention is to have a device for storing the correspondence between the position on the image and the actual position, and a calculation means for comparing and N-analyzing the separated and extracted lines and the stored correspondence. In an advantageous embodiment of the present invention, the intersection line of the plane formed by the two flat light beams coincides with the rotation center axis of the robot's hand or the geometrical symmetry axis, and the optical axis of the imaging device also coincides with the intersection line of the plane formed by the two flat light beams. It passes through the above intersection.

The present invention is based on the following knowledge of the inventors.

As a method for detecting the position and orientation of an object without being affected by its color, there is a method that applies light sectioning.

As shown in FIG. 1, this uses a detection 'a' consisting of a slit light source 1 and a secret imaging device 6'2. Is the flat element-3 projected from the slit light source 1 (hereinafter referred to as the slit book) and the iJ clothing object, that is, light cutting?
If the IM 4 is taken from an oblique direction with an angle that makes the optical axis of this light beam 1.2ζ, then rM Iff, the front surface of the object as shown in FIG. 2 is obtained. The point that can be seen far from the detector is placed 1)7 above the image area, so this light cutting waveform 81111 is output,
Detect the positions of points a and a' on the 21st ¥1 on the number of strokes.

By processing, the distance to the object and the 1q position in the left and right direction can be detected. By calculating the angle θ by 5, we can detect the rotation angle of the object around the axis perpendicular to the plane created by the slit light.

Figure 3: As shown in Figure 3, two slint lights 1a and lb are combined to form a cross-shaped strip IJ 71-light 3a that intersects with each other.
.

3b, and for each sulin
, 2b to image the optical cutting line. With such a configuration, since the rotation angle of the object on two non-parallel planes can be detected, the attitude of the object can be detected three-dimensionally without moving the detector.

In addition to the distance to the object, the size of the object in two directions,
Position can also be detected.

However, in such a configuration, two slit light sources and two
Since 11!1 imaging devices d are required, the detector becomes large and difficult to attach to the robot hand. Therefore, it cannot be used for applications in which the relative position 1 & relationship between the work object and the robot hand is detected by the hand detector, feedback is given to the robot, and the robot hand is precisely positioned on the work work object.

Therefore, in the present invention, as shown in FIG. 8, two slit lights are rotated in the same direction around their intersection line,
Light cutting of two beams with one photographing device
The detector is designed to be small enough to detect tp. 2) When it is possible to install a light source, install two IJs in order to extract the two light cutting lines independently.
y) Switch the light sources and emit light one by one to simplify subsequent processing and speed up the detection time.

In the present invention, the core-shaped slit light shown in FIG. 4 is further obtained by arranging the slit light sources 211 in an inverted V-shape as shown in FIG. That is, in Fig. 4, the IJ spot light is on the flat @3aab,
In addition, as long as the angle between the intersection line of these planes 3a and 3b and the optical axis 8 of the imaging device is smaller than 90'', it can be moved and placed anywhere.

Hereinafter, the present invention will be explained in more detail using Examples with reference to the drawings, but these are merely illustrative, and various modifications and improvements may be made without going beyond the scope of the present invention. It's obvious.

[Embodiments of the invention]

Here, in order to make the explanation easier to understand, we will explain an example 1 of a device configuration in which the X axis and y4ilII are symmetrical with respect to a plane passing through the intersection of the imaging device shown in FIG. The same is true for the case shown in FIG. 5, which is a configuration of the inventive device.

In the principle diagram of the detector shown in Fig. 4, when two solid light sources are emitted one by one, the light cutting line detected by the imaging device and the object The correspondence with the actual position is shown in Figure 6 12), 1))) i
As shown here, it forms a diagonal grid. Figure 6 (a) shows the correspondence on the xz plane, and (b) shows the correspondence on the yz plane, with the caustic line C'
i Equal width dashed lines indicate equidistant lines. C) Find the correspondence relationship using an object whose dimensions are known in advance while changing its position from the detector, and store δ, size, and shape as 23<. And 670 of the object. When detecting the posture, the position of the optically amputated woman on the painter detected by the imaging device is compared with the corresponding VA staff who is stored in advance (/-). ], the actual b7 position and inclination of the object can be determined.In the present invention, two S1 knot lights are emitted, and the distance to the object on the two parallel planes is calculated. 1 and the inclination are determined, and from these, the three-dimensional position and appearance of Tsukuda's body can be determined -i--wise.

In addition, in the present invention, the intersection line 5 of two IJ y l- elements
It is assumed that the detector or the object is roughly aligned in advance so that the surface of the object intersects with the surface of the object. shall be detected.

Next, a method for determining the correspondence between the position of the detected light cutting line on the drawing and the actual position of the object will be explained with reference to FIG. Here, only the case of one of the IJ 7 light sources shown in FIG. 4 1a will be described. The same applies to the case of FIG. 4 1b and the case of FIG. 7th
As shown in the figure, a coordinate system xyz is assumed in which the lines of intersection of two slit lights coincide with the two axes and the slit light from the slit light source la coincides with the xz plane. First, as shown in Fig. 7, a plane 6 perpendicular to the two axes and wider than the field of view of the imaging device.
It is placed on a scaled rail 7 so that it can be moved in parallel in the z-axis direction. If the position of the light cutting line on the detected image is determined while moving the plane ii[i6 away from the detector in steps of, for example, 1 cm, an equidistant factor as shown in FIG. 8 will be obtained. Next, the plane 6 is a rectangle with a constant width, and such objects are prepared in several widths, and they are also erected perpendicularly to the two axes on the rail 7 and moved in the two axes directions. When ll@ measured in the X direction of plane 6 is stitched in 2 cm increments, for example, and each plane is moved in two axial directions,
When the locus of the end points of the detected light cutting line is determined, a monospaced line diagram (regarding the X coordinate) as shown in FIG. 9 is obtained. In the equidistant line diagram shown in Fig. 8 and the equispaced line diagram shown in Fig. 9, if we look at the changes in distance and position in the X direction along certain horizontal gh and h' lines on the line diagram, we can see that It will look like Figure m10 and Figure 11. If these are expressed as a distance function z8 at the vertical coordinate j on the diagram and a function X of the position in the X direction, and the horizontal coordinate i on the diagram is expressed as a variable, zz = t (+) = aJo + aj1i+
aJ21 +a, 31 +・(t)xj =
g(1=bjo+')j+'+bjz'+bja
'+ ''' (can be approximated as 21. The degree of approximation is determined by the required detection degree. For all J on the diagram (for example, j = 1 to 256),
a J n.

If bjn (n 1,000 + I + 2 +...) is determined in advance using the above method, the inverse ζ, its two directions, and The position in the X direction can be determined.Also, when determining the inclination of a plane constituting the detection target, the actual coordinates (XI2) (X'lZ) of the end point of the light section line determined as above
') can be used to find the angle between that plane and the X-axis.

Next, a method for extracting a light section line from a detected image will be explained in detail. In this case as well, the case of the one-sided slit light 3a shown in FIG. 4 will be explained. (The optical cutting line imaged by the imaging device t2 generally has a certain width due to the width of the slit light, blurring of the optical system, etc.) Therefore, the horizontal scanning direction of the imaging device is shown in Fig. 8. 9th
The coordinates I of the peak tff11 (i.e., the brightest point) of each horizontal scanning signal are aligned with the directions h and h' (horizontal direction) of the diagram shown in the figure, and the coordinates I of the peak tff11 (i.e., the brightest point) of each horizontal scanning signal are set to the valley j (the optical cutting line at this point). Let the position be Ni.

(FIG. 12) In this way, the position tS+- of the optical cutting line for calculating equations (1) and (2) is extracted.

Next, the process of finding out the three-dimensional position and posture of the object using the device of the present invention will be explained with reference to the detection of a cylindrical object 9 shown in FIG. 13 and a circular hole 10 formed on a plane shown in FIG. Explain.

First, a method for detecting the position and orientation of the cylindrical object 9 will be explained. For example, when the slit light shown in FIG. 5 3a is projected, it is assumed that a light cut image as shown by the broken line in FIG. When the position of the brightest point in the direction of the lateral force is detected and the shape of the light section line is extracted as described above in this figure, the result is as shown in FIG. 16. This waveform is approximated by a broken line, and the coordinates (i, j) of the end points of each line segment are calculated in advance using the equation (1
) (2) ') to convert into actual coordinates (x, z), it becomes as shown in Fig. 17. In this figure, the most 2
An approximate center position M in the X direction of the upper surface of the cylindrical object is determined from the line segment A1 with the smallest coordinates, that is, the midpoint of the line segment closest to the detector. The inclination θ8 of the object in the xz plane can be found using equation (3). Next I
By switching the J91-light source and performing exactly the same detection on the light section image (broken line in Figure 15) obtained for the other IJ7 light in Figure 5 3b,
Find the nearness of the upper surface of the cylindrical object in the X direction, iI:iJ, and the inclination of the object in the yz plane.

Roughly based on the above detection results, the distance z from the detector to the center position of the top surface of the object is determined by 2 for the top surface of the cylinder. Here, xl, x2 are the X coordinates of the end points of the optical section image (1513/iA,) of the xz plane on the top surface of the cylinder, yl
, y2 is the X coordinate of the end point of the optical section image of the yz plane (Fig. 15B), zl"2Iz3.z4
0) Represents the two coordinates of the end point. From the detection results of
If the slopes of the xz plane ik+ and yz plane curves of the top surface of the cylinder are significantly different from 90', the method of determining the center position from the midpoint of the cross section line and the calculation of distance resistance using equation (4) or (5) are not possible. If the difference is judged to be non-negligible,
Detection of inclination W: The robot arm is moved accordingly so that the two axes, that is, the line of intersection of the slit beams are approximately perpendicular to the plane, and the center position and distance are detected again.

Next, a method for detecting the center position of a circular hole perpendicular to a plane and the direction of the hole C will be described. 1st
Figure 8 shows an example of a light cutting team detected from imaging position #C. In this case, as in the previous example, two sets of light cutting groups in the X2 plane and the yz plane are independently detected by switching the slit light sources. The solid line and the broken line in FIG. 18 respectively indicate the two sets of light cutting edges obtained in this way. In the figure, a line segment connecting points a, , a2 and points b1, . By performing processing in which the line segments connecting b2 correspond to the line segments A and H in FIG. 15 of the previous example, the center position & and the direction in which the hole is placed are detected in exactly the same way.

Hereinafter, one embodiment of the detection device according to the present invention will be explained with reference to FIG. I will clarify.

Detection H'i c, TV camera 18, slit light source 1a,
It is made up of 1.5 lbs., and is placed on the robot's arm in a similar arrangement.

That is, 1) The intersection line 5 of the plane formed by the two slit lights 3a and 3b is made to coincide with the rotation axis 12 of the hand of the robot.

2) Optical axis 8a + 8b ts of slint light source 1a + 1b
Yohi '11' V camera 18 light a! +8c intersect at approximately one point on the intersection line 5 of the slit lights.

3) Arrange the slit light sources 1a, lb and the TV camera 18 so that they are lined up horizontally.

4) The angle formed by the slit lights 3a and 3b is 90''.

The entire apparatus is controlled by a microfun beater (or minicon beater) 15. That is, the slit light switching device 1f13 is controlled, and the slit light sources la, l are
Let b emit light one by one, and then light cut i#!
is detected by the TV camera 18. The detected 'rv image signal is inputted to the optical cutting line extraction circuit 14, where the optical cutting waveform is extracted and outputted to the microwave beater 15. This photo-cleavage! As described above, the extraction circuit 14 compresses two-dimensional image data into one-dimensional waveform data, and is, for example, a device disclosed in Toku Mll Sho 57-146427. The two light-cutting waveform data obtained in this way are processed in the microwave beater 15, and as explained earlier, the waveform data is calculated using the polygonal line approximation formula ((5)). The three-dimensional position and orientation of the target object are detected, and the data 17 is output to the control device of the robot arm mechanism.

The coefficient ajn+bjl(J
= 1.2.3..., n = 0.1゜2,...
) is determined in advance by the method described above and stored in the storage device 16, and is appropriately taken out and used when necessary.

Next, the operation of the robot vision device according to this embodiment will be explained.

Before detecting the target object, as explained above, it is necessary to store the coefficients shown in equation (11(2)) (and the coefficients of yj obtained in the same way) in the storage device. Using the device shown in Fig. 7, the robot is operated and fixed so that the line of intersection of the IJ shot beams is parallel to the rail 7, and the ζ equidistant line diagram (Fig. ) and a monospaced line diagram (Fig. 9).This work is switched by the slit light switching device 13, and the two slit lights 3a
, 3b respectively. Next, based on these data, polynomial approximation is performed by the conbeater 15, and the coefficients of equation (2) for each 1 are obtained and stored in the storage device 16. As explained using FIGS. 13 and 14 as examples, the conbeater 15 controls the slit light switching device 13, and the extracted waveform of the light cutting line when the two sets of slit light sources emit light is extracted by the light cutting line extraction circuit. 14, and after approximating this waveform to a broken line using the conbeater 15, the actual position coordinates of each line segment are calculated using the equation (2) (2) based on the data stored in the storage device 16.
and a similar formula for determining yj). Among the line segments when two sets of slit light sources are emitted on the conbeater 15, the one on the target object is selected one by one using the distance 2 as a criterion. The position and orientation of the target object are detected by calculating equations (3) to (5) and the midpoint of the line segment from each of the four end points of the line segment (near the center), and controlling the robot arm mechanism. Output to device.

In this embodiment, since the TV camera and the IJ 71-light source are arranged as shown in 2) to 3), the detector can be made small and compact. In addition, since the slit light is arranged as shown in 1) to 4), the two axes always face in the same direction with respect to the rotation of the robot hand, and the coordinate system xyz is 1.
This is an α-angle seating system, and it is easy to convert the detection results output to the opening side 1 device to the robot arm mechanism into the coordinates of the robot arm mechanism thread. Furthermore, since the equidistant line diagram and the equispaced line diagram are approximated by functions as shown in equation (2), the amount of data is smaller than if each line diagram is stored as is.

(For example, i = l ~ 256. When performing a fourth-order approximation, approximately i) [Effects of the Invention] As explained above, according to the present invention, the light source is arranged so that the two slit lights intersect with each other, and each By analyzing the optical cutting line, the position and orientation of the target object in two non-parallel planes can be detected without moving the detector or the target object, so the three-dimensional position of the object,
This has the effect of allowing posture to be detected at high speed. In addition, since the cross-shaped slit light is obtained from two sets of slit light sources arranged in an inverted V-shape, the dust pickup device and the slit light source can be integrated, making it compact and lightweight.
There is an effect that detection is possible without being affected by an object grabbed by the brim, IJ & bar. Since the optical cutting line in two non-parallel planes is detected using a single imaging device, there is a benefit in making the detector smaller and lighter. moreover,
The effect of the light cutting method, which is the premise of the present invention, makes it possible to detect the position and orientation of an object without being affected by its color or surface condition.

[Brief explanation of drawings]

Figure 1 is a perspective view of a detector for explaining the light sectioning method, Figure 2 is a diagram showing an example of a detected image obtained by the detector shown in Figure 1, and Figure 3 is a diagram showing two sets of the first image. FIG. 4 is a perspective view of a detector based on the principle of the present invention. FIG. 5 is a perspective view of the basic configuration of a detector according to the present invention. Fig. 4 is a diagram showing equidistant lines and equispaced lines by the detector, Fig. 7 is a perspective view of the device for obtaining equidistant 1llIf/s and equispaced lines, Figs. 8 and 9. Figure 7 shows an example of an equidistant line diagram and an equispaced line diagram obtained by one cycle of the apparatus shown in Figure 7, Figure 10, and M
11 is a graph showing the changes in the values in FIGS. 8 and 9, respectively. FIG. 12 is a graph for explaining the method of extracting the optical section line, and FIGS. 15, 16, and 17 are perspective views of the target example, and FIG. 18 is a diagram for explaining the detection process of the detection target shown in FIG. 13, and FIG. 18 is a diagram showing the detection process of the detection target shown in FIG. 14. The figure showing the image, FIG. 19, is a perspective view of the detector according to the present invention attached to the hand of the robot, including a block diagram showing the configuration of the image information processing device. 1, la, Ib... slit Light sources 2, 2a, 2b...Imaging device 3+ 3a, 3b...S IJ Tsu I-light 4...
・Light cutting line 5...intersection of slit light @6...flat plate 7・
・Rail with scale 8a, 8b, 8c...
・Optical axis 9...Cylindrical object 10...Circular hole 1
1...Robonoto 1 death 12...Robot minion 1! -! 1 rotation axis 13...S IJ 7 Light switching device 14... Light cutting yoke extraction circuit 15... Conbeater 16... 1i12 storage device
17...Detection output Fig. 7 Fig. 2 Fig. 3 Ordinance 4th revision No. 5 Fig. 2 Fig. 7 ? f] Fig. 70 Fig. 1I Fig. 72 Fig. 13 Fig. 14 Fig. 15 Port 632- Fig. /6 Fig. 77 Fig. /2

Claims (1)

[Claims] 1. A robot visual device that detects an optical image of a target object, analyzes it to detect the position and orientation of the target object, and controls the arm mechanism of the robot based on the detection results. Here, a light source that generates two non-parallel flat light beams, a light source switching device that switches between the two flat light beams to emit light, and an optical system for the intersection of the flat light beams and the surface of the target object. Convert the image into an electrical signal and place it on the robot's arm on a plane that bisects the angle formed by the two planes formed by the two flat light beams, and that is not on the intersection line of the two planes. An onboard imaging device, a device that separates and extracts the bright lines in the second image obtained by the imaging device, and a correspondence relationship between the positions of the lines thus separated and extracted on the image and their actual positions. A robot visual device characterized by having a storage device and a calculation means for comparing and analyzing the separated and extracted wins and the stored correspondence. 2. The robot visual device according to claim 1y4, wherein the intersection line of the plane formed by the two planar light rays coincides with the rotation center axis of the robot's hand or the geometrical symmetry axis. 3. The robot visual device according to claim 1, wherein the optical axis of the imaging device passes through the intersection line.