JPH01177615A - Offline programming method for robot - Google Patents

Abstract

PURPOSE:To program a robot action path with an offline by obtaining a variable-density image from the same viewpoint to measure a distance image, automatically preparing the robot action path from a work starting point up to a completing point on a variable-density screen, and preparing a three- dimensional action path corresponding to the action path on the variable-density screen from the distance image. CONSTITUTION:A distance image 12 of a work object 11 is measured, it is stored into a picture memory MEM, simultaneously, a variable-density image 13 is obtained from the same viewpoint to measure the distance image, and it is plotted on a display screen CRT. The work starting point and work completing point are indicated by a cursor 14 on the variable-density image, simultaneously, a proceeding direction is instructed, and thereafter, a robot action path 15 is automatically prepared on the variable-density image 13 from the work starting point in the proceeding direction. Thereafter, a three-dimensional robot action path 16 corresponding to the action path 15 on the variable-density image 13 is prepared from the distance image 12. Thus, by a small-sized system constitution and, moreover, even when the work object has a complex shape, the robot action path can be programmed with the offline.

Description

DETAILED DESCRIPTION OF THE INVENTION <Field of Industrial Application> The present invention relates to an offline programming method for a robot, and more particularly to an offline programming method that can generate a robot motion path by offline control using a visual sensor.

<Prior art> Conventional offline programming systems for robots have a simple internal modeler, so if the shape of the workpiece is simple, this modeler can be used to define the shape and the robot's motion path. Can be done.

In addition, large-scale offline programming systems are connected to CAD systems, and the object model defined in the CAD system can be obtained and used, so even if the object to be worked on has a complex shape, If the motion path also matches the model shape, the motion path can be defined off-line.

<Problem to be solved by the invention> However, a large-scale offline programming system combined with a CAD system is expensive, and if the workpiece does not match the object (model) designed with the CAD system, There is a problem that the robot movement path cannot be programmed offline. In particular, the shapes of objects handled in arc welding, laser welding, sealing, etc. often include free-form surfaces, and many of the objects processed by sheet metal processing are not designed using CAD, and such work objects require large scale. Even with advanced systems, it has been difficult to program robot movement paths offline.

Further, although a small-scale offline system has no problem in terms of cost, there is a problem in that the robot movement path cannot be programmed offline if the object to be worked on has a complicated shape.

From the above, it is an object of the present invention to provide a method that allows offline programming of a robot motion path with a small-scale system configuration, even when the workpiece has a complicated shape.

〈Means for solving problems〉 FIG. 1 is an explanatory diagram of the present invention.

Reference numeral 11 denotes a workpiece to be arc welded, which includes members 11a,
Arc welding is performed between llb and llb. 12 is a distance image, 13 is a grayscale image of the object drawn on the display screen CRT, 15 is a robot movement path, and 16 is a three-dimensional robot movement path.

<Operation> The distance image 12 of the work object 11 is measured and the screen, Nomo I
At the same time as storing it in JMEM, a grayscale image 13 is obtained from the same viewpoint from which the distance image was measured and drawn on the display screen CRT. Then, use the cursor 14 to indicate the work start point and work end point on the gray scale image, as well as the direction of movement.
5 is automatically generated on the grayscale image 13. After that, the three-dimensional robot motion diameter Fi! corresponding to the motion path 15 on the grayscale image 13 is determined. 616 is generated from the distance image 12.

<Example> FIGS. 1 and 2 are explanatory diagrams for explaining the offline robot programming method according to the present invention,
11 is a workpiece to be arc welded, and a member 11a
, llb are arc welded. 12 is screen memory ME
Distance image stored in M, 13 is display screen CR
This is a grayscale image of the object drawn on T. In addition, 14 is a cursor, 15 is a robot movement path on the grayscale image, and 16 is a cursor.
are the three-dimensional movement path of the robot l-, P, PE (Fig. 2) are the work start and end points specified by the cursor 14, p m and p +i are the intermediate points, and TDS-
T.D.

is the direction of tools such as 1 to 1 held by the robot.

FIG. 3 is a block diagram of an offline programming system implementing the method of the present invention.

1 is an image generation device that generates a distance image 12 and a gray scale image 13 of the work object 11 (FIG. 1); 2 is a main processor that executes offline programming according to a control program; 3 is a main memory (RAM); 4 is a graphic display device, and a cathode ray tube (C
RT) 4a and a display controller 4b,
The display controller 4b is provided with a screen memory for storing grayscale images.

5 is a keyboard, 6 is a disk controller, 7 is a hard disk that stores generated robot movement paths, etc.
8 is an X-Y plotter. Although not shown, necessary input/output devices are connected to the bus line to constitute a robot offline programming system.

The image generation device 1 includes a projector 1a, a camera 1b, and an image processing processor 1c, and is configured to output a distance image and a grayscale image of a work object.

It should be noted that each pixel of the distance image represents the distance from the viewpoint to the object, and is generated by the following methods (il to (iii). Namely, (1) Projecting the G-L-ecode pattern light with the projector 1a. (11) By projecting slit light and expressing the space as a code, and capturing the image with camera 1b to obtain three-dimensional information of the object using the principle of triangulation, or (11) projecting slit light and similarly using the principle of triangulation. By determining the three-dimensional position of the part illuminated by the light and scanning the slit light with a rotating mirror to obtain three-dimensional information for one screen, It is generated by calculating the distance to the object from the phase difference of the reflected waves and scanning the laser beam two-dimensionally to obtain three-dimensional information for one screen.

FIG. 4 is a flowchart of the offline programming knob method according to the present invention, and FIG.
FIG. 6 is a flowchart of a process for determining the orientation (posture) of a tool. Off-line programming for arc welding will be described below in accordance with the flowcharts of FIGS. 4 and 5, but the present invention can also be applied to other processes such as laser welding and sealing.

(81 Image generation device 1 (Fig. 3)
A distance image 12 and a grayscale image 13 shown in FIG. 1 are generated.

That is, first, the image generation device 1 generates a distance image 12 of a scene including the surface of the workpiece 11 to be welded, and stores it in the main storage device 3.

Then, a gray scale image 13 of the same scene is generated at the same camera position where the distance image was generated and is stored in the main memory bag @3. The grayscale image 13 is also sent to a graphics display device (monitor television) 4 and displayed on a display screen C.
RT (see Figure 1).

Here, the reason why the gray scale image is used is to make it easier for the operator to understand when displaying the gray scale image on the display screen and displaying the motion path superimposed on it. Unless there is a step in the direction, it is difficult for the operator to visually understand.

(In a state where the BL grayscale image 13 is displayed on the screen L-, the screen CRTI), the cross cursor 14 (first
(see figure) by operating the cursor shift key on the keyboard 5, welding starting point p, (Figure 2 (a)),
Enter the welding end point PE.

Also, the traveling direction of the welding line (operation path) at the welding start point P5 is set, for example, by −1 according to the two-dimensional coordinate axes on the screen.
Enter -X, -X, +Y, -Y.

Furthermore, (1) whether the plane that serves as a reference for determining the posture of the tools () and -chi) held by the robot is on the right or left side in the direction of welding progress; robot?
/) The angle a, (iii
Enter the angle β between the tool and the base milling surface.

(c) After the welding line 15 is traced on the grayscale image 13 from the work start point P to the work end point PE in the advancing direction, the obtained welding line 15 is traced on the grayscale image 13.
(see Figure 2 (bl)).

In addition, tracing of the welding line (carry out as follows, i.e.,
Generally, a weld line is a place where metal plates are overlapped, so on a grayscale image, it is a darker line (curve) than the surrounding area. So, starting point P! , to the end point P5, the location where the brightness is minimum on the grayscale image is tracked.

Specifically, for example, if the welding line's advancing direction is +X, we will extract a total of 7 pixels, including the adjacent pixel on the +X side of the latest tracked pixel and 3 pixels above and below it in the Y-axis direction. The pixels whose brightness is minimal are stored as pixels on the welding line, and the same process is repeated thereafter to generate the welding line 15. However, the advancing direction of the welding line is determined, for example, from the coordinates of the welding point 5 pixels before and the coordinates of 1 pixel before, which is the advancing direction +x, -x, +y, or -y.

(By dl Ju, the welding line on the grayscale image (Robo ν1
・Operation route)] 5 is determined, a three-dimensional welding point array (three-dimensional mouth button 1-operation route) 16 is calculated from the distance image 12 based on the coordinates of the welding point array (welding line) on the grayscale image. (1st
Figure).

That is, the three-dimensional coordinate values of the object displayed at the pixels in m rows and n columns of the grayscale image are m in the distance image 12.
Since the data is stored in correspondence with the pixels in rows and n columns, the three-dimensional welding point sequence (three-dimensional robot motion path) 16 can be easily determined.

Once the three-dimensional welding line 16 is determined, it is approximated by a polygonal line to extract the teaching point and the welding point (see FIG. 6). That is, first, find the intermediate point 'P11 where the distance to the straight line LL (FIG. 6(a)) connecting the starting point P, . . . If the value is greater than or equal to 7, the intermediate point P11 is stored as the teaching point.

After that, a polygonal line L2 (Fig. 6(b)) is generated that connects the starting point P, the intermediate point p++, and the end point PE in order, and the intermediate point pH with the maximum distance to the polygonal line is determined, and the maximum distance is Rd
Check if 9 is above the allowable value, if it is above, intermediate point PI
Store I as a teaching point.

Next, similarly generate a polygonal line T-3 (Fig. 6 (C)) connecting the start point, each intermediate point, and the end point until there is no intermediate point where the maximum distance to the polygonal line is greater than or equal to the allowable value. Perform similar processing. Finally, the starting point P5, each intermediate point p
,, pN1 end point PE becomes the welding point to be taught (second
Figure (c)).

When the tel fold approximation is completed, each teaching point P obtained.

Determine the posture of the torch at p, , p, , p- (assumed to be a-90° and β-45°). FIG. 5 is a flowchart of a routine for determining the postures of 1.

(Find the three-dimensional straight line that indicates welding is from the three-dimensional welding point array before and after the il teaching point.

In addition, use the least squares method (using the seven welding point arrays in front and back including the teaching point) to find a straight line that indicates the welding line.As in the case where the teaching point is the start point or end point, three points in the front and back. If the above welding points do not exist, for example, the starting point is taken to include the starting point and the following six points.

Fitting a straight line using the method of least squares A straight line can be expressed as an intersection line of two planes perpendicular to the coordinate plane. Therefore, here, by fitting two planes perpendicular to the coordinate plane to the three-dimensional point sequence, a straight line is found as the intersection line. Now, (1) Compare the starting point and ending point of the point sequence and
When the width of change in the component is larger than the width of change in the X component, the two planes are expressed as follows.

The former is a plane perpendicular to the xy plane, and the latter is a plane perpendicular to the zx plane.

(2) If the width of change in the X component is greater than the width of change in the X component when comparing the start point and end point of the point sequence, the two planes are expressed as follows.

Now, in the case of (1), the normal equation by the method of least squares of the plane equation% is
= g Σ
yl [xx g+ [xl p= [xy
] Solving this for gtT' gives g-([xl [yl -1c Cxyl)/([x
l”-k [xx]lp-([X co Exy]
[yl [xx] / + [X]
2-1c [XXko]. And similarly for the plane equation % formula % h-([X] [y, co-k [Z X] )
/ + [X] 2-k [XX])
q-([X ko [xx ko - [Zl [xx]
)/ + [X] ”-k [XX] 1
becomes.

(11) Once the three-dimensional straight line indicating the welding line is found, a plane passing through the teaching point and perpendicular to the straight line is found.

Calculated point (x, ,
Y, p Zl) through the vector N (A.

The plane perpendicular to the directions B and C) can be expressed as follows.

A (x-x,) +B (y-y,) +C(zz-z
, ) -〇 (11) Assuming that the straight line is obtained by fitting the straight line to a three-dimensional point sequence using the method of least squares, this straight line can be expressed as follows, for example.

This straight line can be expressed as x-(y-p)/g-(z-q)/h, so the vector of this straight line is (1, g
, h). Substituting this into equation (1), (x-x,) +g (y-y,) +h (z-zl
) becomes −0.

(iii) Next, find the plane equation of the plane that will be the reference for determining the posture of 1.-chi.

Several tens of points on a reference plane near the teaching point are extracted from the distance image, and a plane is fitted to these points using the least squares method.

The point used to find the plane by the least squares method, -15
− is extracted as follows, for example. That is, (
From the two-dimensional coordinates of the starting point and ending point of the point sequence extracted in 1),
Find the welding direction (10X, -X, +Y, -Y).

For example, if the traveling direction is +X and the reference plane is on the left side of the traveling direction, shift the two-dimensional point sequence of the 7 points extracted in (1) from 2 pixels to 8 pixels in the +Y force direction.
Three-dimensional points corresponding to a total of 7×7=49 points are extracted.

Fitting the plane to the front point using the least squares method The equation of the plane is A-x+B -y+C-z+D=0 z=-A -x/C-B -y/C-D/C (C≠0)
Expressed as In addition, C=O means a plane parallel to the Z axis,
Therefore, it is assumed that the plane expressed by the above formula is not parallel to the Z axis. That is, -A/C=a1. -B/C=b1. -D/C=d, then the plane equation is z=a, x+b, -vld.

It can be transformed into As a result, the normal equation of the least squares method is as follows. Note that k (-49) is the number of pixels in the small area,
(X, , yl, Zl) are the coordinate values of the first pixel.

Σz, = a, Σx, +b, Σy, + to -d, (
i=1-k) Σx, z, -a, Σx'', +b, Σx, y
, +d, Σx, (i = 1~k)Σy, z
, = a, Σx, y, +b, Σy? +a,Σy,
(i = 1~k) Expressed using Gauss symbols, (
ΣX, y, -1-[X y ], etc.) as follows.

[X a, + [yl b, +k - d, =
[zl[xxko a, + [xy] b, + [x
l dl-[zx][xy] a, + [yy] b
, + [yl d1= [yz] This simultaneous equation is a,
, b,, dl, we get a,=(u,,・
uZX-uXy□u,,,)/'/.

b, = (uxx ・uyz-uyy ” 2X) /
vld, −([zl −[xl a, −[yl b,
) / to find the plane. However, t xx yl ×yu,,y
. [xl 2-k [xx] , u, , -[xl
[yl-k[xy], etc.

EX]-Σx,, [xx] =Σx? , [xy]−
Σx, y, et c.

(iv) Once the reference plane (reference plane) for determining the posture of the h-chi is found, find the line of intersection between the reference plane and the plane perpendicular to the welding line (determined in step (11)). .

(v) Next, if we find a straight line rotated by β (-45°) in the direction away from the plane that is the reference for determining the torch posture in a plane perpendicular to the intersecting welding line, we can find that the straight line is 1. - It becomes a straight line that represents the posture of Chi.

That is, the straight line direction is the direction in which 1--chi faces (Fig. 2 (
(See TDS-TDE in d)). However, the reference planes are the planes BS1 and BS2 on the left side of the welding line in the direction of travel.
shall be.

As described above, if the direction of l·-chi is determined, this direction is output together with the teaching point, and thereafter, the direction of 1·−1,000 is determined for all the teaching points and outputted together with the teaching point, and the off-line programming is completed. Note that the true teaching point is a point that is a certain distance away from the object in the straight line direction determined by (■).

In the above, when finding the posture of l・-chi, a-90
Although we have explained the case of a = 90' and β-45°, this is a special case, and it is not necessary that a = 90' and β-458.In this case, the attitude of 1. To do this, find a plane that includes the welding line (three-dimensional robot motion path) and makes an angle β with the reference plane, then find a straight line on this plane that makes an angle a with the welding line, and set the direction of the straight line to 1. It may also be set in the direction of .

<Effects of the Invention> According to the Ueki invention, a distance image of the work object is obtained along with a measurement distance, and a grayscale image is obtained from the same viewpoint from which the distance image was measured, and the work is performed from the work start point indicated on the grayscale image. The welding line (robot motion path) up to the end point is automatically generated on the grayscale image, and then a three-dimensional motion path corresponding to the welding line (motion path) on the grayscale image is generated from the distance image. With this configuration, it is now possible to program the robot motion path offline even when the workpiece has a complicated shape.

Further, according to the present invention, since the image generation device is simply added to the conventional off-line programming system, it is possible to provide a small-scale and inexpensive off-line programming system for robots.

=19-Furthermore, the present invention is very useful because it not only teaches the positions of tools such as 1 and 1 but also automatically calculates the orientation of the tools.

In addition, since the obtained welding line is approximated by a polygonal line and the number of teaching points is reduced, the number of points at which the posture should be calculated is reduced, and the processing time can be shortened.

[Brief explanation of the drawing]

Figures 1 and 2 are explanatory diagrams of the present invention, Figure 3 is a block diagram of a robot in-programming system that implements the present invention, Figure 4 is a flowchart of the processing of the present invention, and Figure 5 is a FIG. 6 is a flowchart of the posture determination routine of steps 1 and 1. 11... Work object to be arc welded, 12... Distance image, 13... Grayscale image, 14... Carnol, 15... Welding line (motion history #!I), 16... Three-dimensional welding line (Three-dimensional robot movement history f?a
) Patent Applicant Fanuc Co., Ltd. Agent Patent Attorney Chiki Saito Figure 5 Figure 6

Claims (5)

[Claims] (1) A first step of measuring a distance image of the work object and obtaining a gray scale image from the same viewpoint from which the distance image was measured;
The second step is to automatically generate the robot movement path from the specified work start point to the work end point on the grayscale screen, and to create a three-dimensional movement path corresponding to the movement path on the grayscale screen by distance. 1. An offline programming method for a robot, comprising a third step of generating data from an image. (2) Set the angle α between the tool gripped by the robot and the motion path, and the angle β between the tool and the reference plane, find a plane that includes the motion path and forms an angle β with the reference plane, and
2. The offline programming method for a robot according to claim 1, further comprising a fourth step in which the tool direction is a linear direction forming an angle α with the motion path on the plane. (3) In the second step, the brightness of a pixel adjacent in the traveling direction to a known pixel on the robot motion path and several pixels above and below adjacent to the adjacent pixel in the traveling direction is minimized. 3. The offline programming method for a robot according to claim 1, wherein the motion path is generated assuming that each pixel is a pixel on the motion path. (4) In the third step, find the intermediate point where the distance to the straight line connecting the robot movement path start point and end point is maximum, and when the maximum distance is greater than or equal to the allowable value, store the intermediate point, and then Find the intermediate point where the distance to the polygonal line that connects the intermediate point and the end point in sequence is the maximum, and when the maximum distance is greater than or equal to the allowable value, store the intermediate point; 4. The offline programming method for a robot according to claim 3, wherein the robot motion path is approximated by a polygonal line by performing polygonal line approximation processing until no points exist. (5) In the third step, find the intermediate point where the distance to the straight line connecting the robot movement path start point and end point is maximum, and when the maximum distance is greater than or equal to the allowable value, store the intermediate point, and then Find the intermediate point where the distance to the polygonal line that connects the intermediate point and the end point in sequence is the maximum, and when the maximum distance is greater than or equal to the allowable value, store the intermediate point; 2. The robot offline offline method according to claim 2, wherein the robot motion path is approximated by a polygonal line by performing polygonal line approximation processing until no points exist, and in the fourth step, the tool direction is determined only at the approximate points. programming method.