JP3538506B2 - Automatic image processing program generation method for robot, device thereof and robot program automatic correction method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a robot for performing an operation involving contact with a work such as a welding operation or a chamfering operation, and more particularly to an automatic image processing program generating method for such a robot, an apparatus therefor, and a robot. It relates to a program automatic correction method.

[0002]

2. Description of the Related Art Conventionally, robots have been used in various forms on factory production lines and the like to perform various operations. The main applications of robots include welding in automobile lines and assembling electronic components.
In addition, it is used in various operations such as material handling, painting, and chamfering. The basic usage of the conventional robot is to ensure that the work is accurately positioned with respect to the robot using a jig, etc. It was to repeat it.

[0003] However, in recent years, the price of computers has dropped, the computing power has increased sharply, and CCDs (Charge c
Due to the price reduction and performance improvement of cameras (oupled devices), image processing has become available at a price that is incomparable to conventional ones. Along with this, in the field of robots, an example of actively using image processing is increasing. Typical methods of use include a method of detecting the type and accurate position of a work in palletizing for transporting an object to a predetermined position on a pallet, and a method of detecting the position of an IC in a chip mounter. In conventional image processing for robots, binarization processing is usually performed by hardware dedicated to image processing due to severe restrictions on processing time. Accordingly, image processing apparatuses that perform pattern matching of grayscale images and the like are increasing. It is generally said that pattern matching of a grayscale image is more stable with respect to changes in illumination conditions than binarization processing.

An image processing apparatus for a robot is incorporated in a robot controller for controlling the operation of the robot, or can exchange data with the robot controller, so that the result of the image processing can be reflected in the robot program. It has become. Further, the image processing apparatus is usually provided with a dedicated image processing language.
The user needs to write a program for the processing content in advance. Many of the conventional image processing languages have been devised to make the program work by the user as easy as possible. However, the functions provided by the image processing apparatus have rapidly increased with the development of image processing technology. It can be said that the knowledge of image processing required by users is rather increasing.

[0005] As a specific work of a robot that can use image processing, for example, in a work in which a tool attached to the end of the robot moves along a work ridgeline such as a chamfering work or a welding work, the work of the work is not performed. It is necessary to accurately grasp the position and shape of the work ridgeline that changes due to individual differences, and it is effective to obtain the actual position of the teaching point that characterizes the position and shape of the work by image processing. Become.

[0006]

As described above, it is effective to use image processing for various operations performed by a robot. However, in order to use image processing, it is necessary to use image processing unique to each image processing apparatus. After learning the language, it was necessary to write an image processing program for each task. Further, it is necessary to describe an image processing program for each work and to create or modify a robot program so that the robot operates using the image processing result.

[0007] For this reason, in the conventional robot, it takes a lot of time to describe the image processing program. In addition, a great deal of time is required to acquire a language specific to the image processing apparatus and basic knowledge about image processing, and it is difficult to use the image processing apparatus unless a person who is familiar with image processing is used.

However, if the image processing required for the work involving contact with the work such as the welding work and the chamfering work performed by the robot is limited, the position and shape of the work ridge line that changes depending on the individual difference of the work. Are already included in the robot program that has been taught for one master work, as a type of operation command and a teaching point that is an argument of the operation command. Here, a teaching point for which an accurate position is required by image processing is an operation for which positioning with a work ridge line or the like is required.

On the other hand, in the robot program, in addition to the teaching points that require an accurate position directly related to the operation at the time of welding operation and the operation at the time of chamfering operation, the robot program merely changes the posture of the robot. It also includes teaching points related to motions that do not need to consider individual differences between workpieces, such as motions. However, as for the welding operation, for example, the teaching points related to the operation at the time of the welding operation that requires high-precision positioning by selecting only the operation command of the robot sandwiched between the welding start command and the welding stop command. It is possible to extract only

[0010] Further, as for the chamfering operation by the robot having the force control function, the welding operation can be performed by selecting only the operation command that comes into contact with the work, such as a linear movement instruction or an arc movement instruction that comes into contact with the work. As in the case described above, it is possible to extract only the teaching points related to the operation at the time of the contact work with the work requiring high-precision positioning. As described above, the robot program taught for the master work includes all information necessary for creating an image processing program.

The present invention has been made in view of the above points, and it is possible to automatically generate an image processing program for coping with individual differences between works from a robot program taught for a master work. It is an object of the present invention to provide a method and an apparatus for automatically generating an image processing program for a robot that can be used. It is another object of the present invention to provide a robot program automatic correction method capable of automatically correcting the coordinate values of each teaching point previously stored in a robot program using an image processing program generated by such a method. I do.

[0012]

SUMMARY OF THE INVENTION The present invention relates to a method for automatically generating an image processing program for a robot which performs an operation (such as a welding operation or a chamfering operation) involving contact with a workpiece.
Extract a motion command including a teaching point based on the robot coordinate system when performing a contact work with a work from a robot program that has been taught in advance for a master work, and derive a pre-installed teaching point from the extracted motion command. The first step to be extracted and, for each teaching point extracted in the first step, what curve (straight line, arc, etc.) each teaching point belongs to based on the type of the operation command including each teaching point. And reading out, from the robot program, coordinate values of teaching points (end points, intermediate points, etc.) for specifying the range of the determined curve.
And the coordinate values in the robot coordinate system of each teaching point read out in the second step are defined by a displacement between the coordinate system of the video camera for photographing the workpiece and the robot coordinate system defined in advance. A third step of converting the coordinates into coordinates within the screen of the video camera using the quantity, and a coordinate value of each teaching point converted by the third step and a type of a curve to which each teaching point belongs. Generating an image processing program for detecting a position of a curve (a straight line, a circular arc, or the like) corresponding to each of the teaching points from an image taken by the video camera, and obtaining an actual position of each of the teaching points as an intersection thereof. And a fourth step of automatically generating an image processing program for a robot.

According to the present invention, there is provided an image processing program automatic generation device for a robot which performs a work involving contact with a work (welding work, chamfering work, etc.), comprising: a video camera for photographing a work to be worked; Extract a motion command including a teaching point based on the robot coordinate system when performing contact work with the work from the robot program taught for the master work, and extract pre-built teaching points from the extracted motion command Means, and for each teaching point extracted by this means,
From the type of the operation command including each teaching point, it is determined which curve (straight line, circular arc, etc.) each teaching point belongs to, and a teaching point (end point or halfway point) specifying the range of the determined curve is determined. Means for reading the coordinate values of the teaching point from the robot program, and the coordinate values in the robot coordinate system of each teaching point read by the means are converted into the coordinate system of the video camera and the robot coordinates defined in advance. Means for converting into coordinate values within the screen of the video camera using the amount of displacement with the system, and based on the coordinate values of each teaching point converted by this means and the type of curve to which each teaching point belongs, An image processing program that detects the position of a curve (a straight line, an arc, or the like) corresponding to each of the teaching points from an image captured by a video camera and obtains the actual position of each of the teaching points as an intersection of these points. Further comprising a means for generating a to provide an image processing program automatic generation device for a robot according to claim.

Further, the present invention uses the image processing program generated by the above-described method for automatically generating an image processing program for a robot to calculate the coordinate values of the teaching points of the actual work from the image taken by the video camera. Determining a coordinate value of each teaching point previously stored in the robot program based on the coordinate value of the teaching point of the actual work determined by this step. A method for automatically correcting a robot program is provided.

According to the method and apparatus for automatically generating an image processing program for a robot according to the present invention, a teaching point for which an accurate position is to be obtained by image processing is extracted from a robot program that has been previously taught for a master work. Based on the type of the operation command including the extracted teaching points and the teaching points included in other operation commands, the coordinate values of these teaching points in the robot coordinate system and the type of the curve to which each teaching point belongs are determined. Then, the coordinate values of these teaching points in the robot coordinate system are converted into coordinate values in the video camera screen. Then, based on the converted coordinate values of the taught points and the type of the curve to which the taught points belong, the positions of the curves (straight lines, arcs, etc.) corresponding to the taught points are determined from the image taken by the video camera. An image processing program for detecting and finding the actual position of each teaching point as these intersections is generated. This makes it possible to automatically generate an image processing program for coping with individual differences between works from the robot program taught for the master work.

Further, according to the robot program automatic correction method of the present invention, an actual image processing program generated by the image processing program automatic generation method for a robot of the present invention is used to convert an image taken by a video camera into an actual image. The coordinate values of the teaching points of the work are calculated for each work, and the coordinate values of the teaching points pre-installed in the robot program are automatically corrected based on the obtained coordinate values of the teaching points of the actual work. Therefore, it is possible to automatically modify the robot program so as to absorb individual differences for each work only by performing teaching for one master work and creating a robot program.
For this reason, even if a person who does not have sufficient knowledge about the image processing can easily operate the robot using the image processing in a form absorbing the individual difference of the work.

[0017]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 and FIG. 2 are diagrams for explaining an embodiment of an automatic image processing program generation method and apparatus for a robot according to the present invention.

First, an overall configuration of an apparatus for realizing an image processing program automatic generation method for a robot will be described with reference to FIG. As shown in FIG. 2, the apparatus includes a video camera 1 for photographing a work 6 to be worked, and a personal computer 3 into which an image input board 2 for capturing an image photographed by the video camera 1 is inserted. ing. Here, the image processing program generated in the personal computer 3 is executed on the personal computer 3, and the robot program taught for the master work is corrected based on the execution result. The modified robot program is Robot 5
The operation is performed on the robot controller 4 that controls the operation of the work 6, and the robot 5 performs the operation in consideration of the individual difference of the work 6. Note that the robot program can be corrected on the personal computer 3 or the robot controller 4. When the correction is performed on the personal computer 3, the corrected robot program is transferred to the robot controller 4.

Next, the procedure of an image processing program automatic generation method for a robot will be described with reference to FIG. FIG.
As shown in the figure, first, from the robot program that was previously taught for the master work, an operation command based on the robot coordinate system when performing work involving contact with the work to be worked, such as welding work or chamfering work, is given. A teaching point requiring accurate position measurement is extracted from the extracted operation command (step S101).

Subsequently, for each of the taught points extracted in step S101, it is determined from the type of the operation command including each of the taught points what kind of curve the straight line or the arc belongs to. At the same time, the coordinate value of the teaching point for specifying the range of the curve such as the end point or the halfway point in the determined curve is read from the robot program (step S102).

Thereafter, the coordinates of the teaching point extracted in step S101 in the robot coordinate system and the coordinates of the other teaching points extracted in step S102 in the robot coordinate system are displayed on the screen of the video camera. The value is converted to a value (step S103). That is, since the coordinate values described in the robot program are coordinate values in the robot coordinate system, these coordinate values are calculated using the displacement between the robot coordinate system and the camera coordinate system having the origin at the projection center of the video camera. To a coordinate value in a screen coordinate system which is a coordinate system in the screen of the video camera. The displacement between the robot coordinate system and the camera coordinate system is set in advance by the user.

Then, based on the coordinate values of each teaching point converted in step S103 and the type of curve to which each teaching point belongs, position detection for detecting the position of a curve such as a straight line or an arc on the screen of the video camera. A program is generated (step S104), and the teaching point extracted in step S101 and step S102 from the position of a curve such as a straight line or an arc detected by the position detection program generated in step S104 is displayed on the video camera screen. And a coordinate conversion program for converting the coordinate values in the screen coordinate system of each teaching point calculated by the position calculation program into the coordinate values in the robot coordinate system. (Step S105).

Hereinafter, taking the work shown in FIG. 3 as an example, a procedure for generating an image processing program for the work will be described in detail with reference to FIG. FIG. 4 is a diagram showing a robot program for performing the edge chamfering work of the work shown in FIG.

In the robot program shown in FIG.
“Speed 10” on the fifteenth line means that the operation speed of the robot is set to “10”. Further, “force 1” on the 16th line means that the pressing force at the time of pressing work by gmove or the like among the operation commands is set to “1 kg”. Further, “dset on line 17
“1” indicates the digital output of channel 1 as “1 (O
N) ”means that“ dres
“et 1” sets the output of the digital output channel 1 to “0”.
(OFF) ”. Note that“ ds
"et 1" and "dreset 1" correspond to the start and stop of rotation of a conical tool or the like, respectively.

On the other hand, "mov
`` el '' is a linear movement command that does not involve pressing on the work, and is used to linearly move a conical tool etc. without pressing on the work from the position at the start of command execution to the teaching point specified by the argument. is there. On the other hand, “gmove” in the 19th line and the like is a linear movement command accompanied by pressing on the work, and a cone tool or the like is pressed from the position at the start of command execution to the teaching point specified by the argument while pressing the work on the work. For linearly moving. “Gmovec” in the 20th line or the like is an arc movement command accompanied by pressing on the work, and is performed on an arc passing through the position at the start of command execution and the two teaching points specified by the arguments at the start of command execution. Through the teaching point specified by the first argument from the position
This is for moving the conical tool or the like while pressing the work to the teaching point designated by the argument of.

When a robot program as shown in FIG. 4 is given, in step S101 in FIG. 1, "gmove", "g
movec "is selected from the robot program, and the teaching points specified as the arguments of these operation instructions and the teaching point of the immediately preceding operation instruction that determines the position at the start of execution of these operation instructions By examining the
The teaching points related to the operation during the chamfering operation are extracted. For example, the argument of the operation instruction “gmove p3” on the 19th line of the robot program shown in FIG. 4 is “p3”, and the position at the start of execution of this operation instruction is determined on the 18th line immediately before "P2" which is an argument of "move p2" which is the operation instruction of "1". Similarly, if all the teaching points related to the operation during the chamfering operation are extracted from the robot program shown in FIG. 4, p2, p3, p4, p5, p
6, p7, p8, p9, p10, p11, p12, p1
It becomes 3.

In step S102 of FIG. 1, for each of the teaching points p2 to p13, the teaching point belongs to any of curves such as a straight line or an arc based on the type of the operation command including the teaching point. Is determined, and a coordinate value of a teaching point for specifying a range of the curve such as an end point or an intermediate point in the determined curve is read from the robot program. In the case of the work shown in FIG. 3, since the work ridgeline is limited to a straight line and an arc, the types of the teaching points are (1) the intersection of the straight line and the straight line, (2) the boundary between the straight line and the arc, and ( 3) It is limited to three types of points in the middle of the arc. In addition, the teaching points for specifying the range of the curve are both end points in the case of a straight line, and both end points and a halfway point in the case of a circular arc. Therefore, in the case of the robot program shown in FIG. 4, the teaching points for specifying the type of the teaching point and the range of the curve are as follows.

[0029] In step S103 of FIG. 1, step S102
The coordinate values in the robot coordinate system of the teaching points related to the operation during the chamfering operation and the teaching point that specifies the range of the curve determined up to this point are converted into the coordinate values in the camera coordinate system and further in the screen coordinate system I do. In the case of the robot program shown in FIG. 4, the teaching point requiring coordinate conversion calculation is p
2, p3, p4, p5, p6, p7, p8, p9, p1
0, p11, p12, and p13. Here, the amount of displacement between the camera coordinate system and the robot coordinate system can be specified by the amount of positional displacement between the origins of the two coordinate systems and the amount of change in posture of the coordinate systems. For these values, the user calibrates the positional relationship between the video camera and the robot accurately in advance, and sets the values in the setting file of the program that implements the automatic image processing program generation method for the robot according to the present embodiment. Keep it.

The coordinate transformation matrix from the robot coordinate system to the camera coordinate system calculated from the displacement between the camera coordinate system and the robot coordinate system is T, the coordinate value of each teaching point in the robot coordinate system is X _ri , and the camera coordinates Assuming that the coordinate value of each teaching point in the system is _Xci , conversion to the camera coordinate system is performed by the following equation (1).

[0031]

(Equation 1) In the above equation (1), it is assumed that the matrix T is represented by homogeneous coordinates. Here, the camera coordinate system has an origin at the projection center of the video camera, and in this embodiment, the lens optical axis passes through the origin, the axis corresponding to the horizontal direction of the screen passes through the origin, and the X axis passes through the origin. The axis corresponding to the vertical direction of the screen is the Y axis. The direction corresponding to the right side of the screen is defined as the positive direction of the X axis, the direction corresponding to the upward direction of the screen is defined as the positive direction of the Y axis, and the direction from the front of the video camera to the video camera is defined as the positive direction of the Z axis. After converting the coordinate values of the teaching point in the robot coordinate system into coordinate values in the camera coordinate system, which is a coordinate system in a three-dimensional space having the origin at the projection center of the video camera, Conversion to a screen coordinate system, which is a coordinate system in a two-dimensional space, is performed by the following equation (2).

[0032]

(Equation 2) In the above equation (2), X _c = [x _c , y _c , z _c ] ^t is the coordinate value of the teaching point in the camera coordinate system, and [x _f ,
y _f ] ^t is the coordinate value of the teaching point in the screen coordinate system. The screen coordinate system has an origin at the center of the screen. In this embodiment, an axis passing through the origin and corresponding to the horizontal direction of the screen is defined as an X axis, and an axis passing through the origin and corresponding to the vertical direction of the screen is defined as a Y axis. .
The direction corresponding to the right side of the screen is defined as the positive direction of the X axis, and the direction corresponding to the upward direction of the screen is defined as the positive direction of the Y axis.

A function or command for converting the coordinate value of the teaching point from the robot coordinate system to the camera coordinate system and a function or command for converting the coordinate value of the teaching point from the camera coordinate system to the screen coordinate system are as follows: xc = TrnRobCam If two functions or commands of the form (xr, T), xp = Persp (xc) are prepared in advance, in the case of the robot program shown in FIG. 4, steps S104 and S105 in FIG.
The program shown in FIG. 5 is executed as a program preceding the image processing program generated in the step (1). 5A is a program for converting the coordinate value of the teaching point from the robot coordinate system to the camera coordinate system, and FIG. 5B is a program for converting the coordinate value of the teaching point from the camera coordinate system. This is a program for converting to the screen coordinate system.

In step S104 of FIG. 1, based on the coordinate values of each teaching point converted in step S103 and the type of curve to which each teaching point belongs, a curve such as a straight line or an arc in the video camera screen is displayed. Generate a position detection program for detecting a position. In the case of the work shown in FIG. 3, since the work ridgeline is limited to a straight line and a circular arc, a routine for detecting a straight line existing in a limited area on the screen, and a routine for detecting a straight line existing in a limited area on the screen. A routine for detecting an existing arc is prepared in advance, and a program for sequentially detecting the straight line and the arc determined in step S102 in FIG. 1 from the screen is generated. That is, in the case of the robot program shown in FIG. 4, the dedicated image processing routine prepared as described above prepared in advance is called, and the straight line L1 to the straight line L4 and the arc C
A program for sequentially detecting the arc C4 from 1 is generated.
In the image processing routine according to the present embodiment, the fact that the brightness sharply changes at the work ridge line is used, and a straight line, a circular arc, or the like is selected from an area in which the range of the individual difference of the work specified in advance is considered. The exact position of the curve is detected.

The details of the routine for detecting a straight line or an arc present in a limited area on the screen will be described below.

First, a routine for detecting a straight line existing in a limited area on the screen will be described with reference to FIG. Here, the limited area in the screen is an area in which both end points have a width of a specified distance around a specified straight line (line segment), and its shape is, for example, a rectangle. Note that the user specifies the range in advance in consideration of the individual difference of the work.

When a limited area in the screen is set, first, as shown in FIG. 6, edge enhancement processing is performed on the area (step S201). FIG. 7 shows a specific example of the edge enhancement processing. FIG. 7A is a diagram showing an entire image captured by a video camera, and FIG.
7A shows the result of performing the edge enhancement processing on the entire image shown in FIG. Note that step S in FIG.
In 201, for example, edge enhancement processing is performed on a region surrounded by a dotted line in FIG. Here, the edge enhancement processing means performing a filtering calculation as in the following equation (3) for each pixel in the area.

[0038]

[Equation 3] In the above equation (3), filter [i] [j] is a filter pattern. In the present embodiment, a filter obtained by differentiating a Gaussian function inclined in a direction perpendicular to the edge inclination (DoG: Derivative of Gaussian) I do. The inclination of the edge is calculated from the coordinate values of both end points of the line segment calculated up to step S103 in FIG. By performing such edge enhancement processing, the value of the pixel on the edge increases as shown in FIG. 7B, and the values of the other pixels decrease.

When an image on which edge enhancement processing has been performed is obtained, the equation of a straight line is obtained by Hough transformation.
The Hough transform is one of the typical methods for obtaining equations such as a straight line and an ellipse, and has a feature that a straight line, an ellipse, and the like can be detected stably even on a screen having noise.

The Hough transform proposed by Hough for straight line detection expresses a straight line as y = ax + b (4) using two parameters (a, b) of a slope a and an intercept b, and expresses a feature point (X _i , Y _i ),
A locus represented by b = Y _i −aX _i (5) is drawn in a parameter space spanned by the straight line parameters (a, b). Then, after trajectories are drawn for all the feature points, a point where many trajectories intersect is extracted in the parameter space, and a straight line corresponding to the point is regarded as existing in the image space.

When the Hough transform is actually performed by a program, the parameter space is decomposed into a large number of elements called "cells", and when a trajectory is drawn for each feature point, the trajectory passes in the parameter space. By increasing the voting frequency of each cell by the value of the result of the edge emphasizing process, points that are more edge-like are given greater weight. Such processing is performed in step S2 of FIG.
02.

After the voting for all the feature points in the limited area on the screen is completed, the cell having the largest voting frequency is extracted to obtain the straight line parameters (a, b). (Step S203). In addition,
In step S203, voting is not performed for all the feature points in the region, but is limited to only the feature points that have been subjected to the edge enhancement process in step S201 and are considered to be more likely to be edges. You may do so.

As shown in the above equation (4), the parameters (a, b)
When the straight line in the image space is represented by, the values of these parameters become infinite as the straight line approaches the vertical in the screen, so that the parameter space is not bounded. A general method for solving this problem is to express a straight line in the image space as ρ = xcos θ + ysin θ (6) using a perpendicular angle θ and a signed distance ρ from the origin. (ρ, θ) space. FIG. 8A shows (ρ, θ) for a straight line.
FIG. 8B shows how to set a parameter space (ρ, θ) by Hough transform. In the present embodiment, for each point in a rectangular area around a straight line (line segment) whose end points are specified,
The value resulting from the edge enhancement processing at that point is added for each cell on a straight line in the (ρ, θ) parameter space calculated by the above equation (6). As a result, a straight line that passes through more points having a large value as a result of the edge enhancement processing is selected.

Next, a routine for detecting an arc present in a limited area on the screen will be described with reference to FIG. 6, as in the case of a straight line. In the case of an arc, three unknown values are the coordinate value (x0, y0) of the center of the arc and the radius r of the arc. Therefore, the coordinates (x0, y0) and the value of the radius r of the arc center are used as parameters. (X0, y0,
r) By voting in the parameter space in the same manner as in the case of the straight line, the arc equation, that is, the position of the arc is determined.

In the edge enhancement processing, as in the case of the straight line described above, a filter in which the DoG filter is set in a direction perpendicular to the inclination of the edge calculated from the coordinate values of both ends of the arc calculated up to step S103 in FIG. (Step S201). However, in the case of a circular arc, unlike the case of a straight line, the slope of the edge changes continuously. Therefore, when performing the edge enhancement processing of each point, the coordinate values of the teaching point calculated up to step S103 are used. The gradient of the circular arc near a specific point on the circular arc is calculated using the circular arc equation that can be calculated from the above equation, and the gradient of the DoG filter is set.

After the edge enhancement processing is performed, the following processing is performed on each point P in the limited area. That is, (x0, y0) in the (x0, y0, r) parameter space
0) For each cell on the plane, if the point P is a point on an arc, the radius is calculated by the following equation (7), and the cell (x0, y0,
The voting frequency of r _o2 ) is increased by the value of the result of the edge emphasis processing at the point P (step S202). Note that r _o2
Is the representative value of the cell containing _ro calculated by the following equation (7) in the (x0, y0, r) parameter space.

(Equation 4)

After the voting for all the feature points in the limited area on the screen is completed, the cell having the largest voting frequency is extracted, so that the parameter (x0, y0, r) of the arc is changed. Is obtained (Step S20)
3). FIG. 9A shows how to set the (x0, y0, r) parameter space for the arc, and FIG. 9B shows how to determine the arc parameter (x0, y0, r) by Hough transform.

The above-described method of calculating the parameters of the arc is a method in the case where the video camera is installed directly above the work, and the method in the case where the video camera is not installed directly above the work. Since it looks like an ellipse, the parameters of the arc are obtained as follows.

First, from the coordinate values of the three points on the arc in the robot coordinate system calculated in step S102 of FIG. 1, an equation of a plane on which three points exist, that is, parameters a, b, and Calculate the values of c and d.

Ax + by + cz + d = 0 (8) In the above equation (8), the values of the parameters a, b, c, and d are values in the robot coordinate system. Convert to the value in the system. Also, since the center position of the arc passing through the three points described above must exist on the work plane described above, two equations expressing the relationship that the distance from the center position of the arc to the three points are equal, and the center of the arc Can calculate the coordinate value of the center of the arc by an equation representing the relationship that the arc is located on the known work plane, whereby the radius of the arc can be easily calculated. The coordinates of the center of the arc are represented by (x _cp , y _cp ,
z _cp ), the coordinate value of the center of the arc can be calculated by the following equation (9).

(Equation 5)

From the arc equation thus calculated, the coordinate values of the points on the arc are calculated for 360 points each degree, and each point is converted into a coordinate value in the screen coordinate system, and an array xfcirle is obtained. To save.

Next, the three-dimensional (x0, y0, r) parameter space using the coordinates (x0, y0) of the center of the arc in the camera coordinate system as unknown values and the radius r of the arc as parameters is described above. By performing the same voting as in the case of the straight line and the arc, the equation of the arc, that is, the position of the arc is determined.

In the edge enhancement processing, as in the case of the straight line and the arc described above, a filter in which the DoG filter is set in a direction perpendicular to the inclination of the edge calculated from the coordinate values of both ends of the arc calculated up to step S103 is used. Apply (step S201). The setting of the slope of the DoG filter corresponding to the continuously changing slope of the edge is as follows.
The array xf described above when setting the filter inclination at each point
The coordinate value closest to the point is searched for in the circle, the slope of the arc is calculated from the values of the array elements before and after the coordinate value, and the inclination is set at a right angle to that.

After the edge enhancement processing is performed, the following processing is performed for each point P in the limited area. That is, for each point P, a three-dimensional coordinate value in the camera coordinate system is calculated from the coordinate value in the screen coordinate system and the equation of the plane. (X0, y0, r) (x0, y) in parameter space
0) For each cell on the plane, if the point P is a point on an arc, the radius is calculated by the following equation (10), and the cell (x0, y0,
The vote frequency of r _o2 ) is increased by the value of the result of the edge emphasizing process at the point P (step S202). Note that r _o2
Is a representative value of a cell including _ro calculated in the (x0, y0, r) parameter space by the following equation (10). Also,
The value of z0 in the following equation (10) is calculated from the value of (x0, y0) and a plane equation.

(Equation 6)

After the voting for all the feature points in the limited area on the screen is completed, the cell having the largest voting frequency is extracted to obtain the parameter (x0, y0, r) of the arc. Is obtained (Step S20)
3).

The routine for detecting a straight line and the routine for detecting an arc described above are as follows: l = line (x1, x2, area) and c = circle (x1, x2, x3, area), respectively. A function or command of the format is prepared, and when a position detection program is generated in step S104 in FIG. 1, a program for sequentially calling the function or command by specifying a corresponding argument is generated. In these functions or commands, l and c are arrays for storing the calculated parameters of the straight line and the parameters of the arc, respectively. Also, x1, which is an argument of line (),
x2 is the (x, y) coordinate value in the screen coordinate system of both end points of the line segment calculated up to step S103, and circcl
x1, x2, and x3, which are the arguments of e (), are the (x, y) coordinate values in the screen coordinate system of the both end points and the intermediate points of the arc calculated up to step S103. Ar
ea is the width of an area where image processing around a line segment or an arc is performed, and for each work from the coordinate value of a teaching point in a robot program taught for a master work in consideration of individual differences of the work, etc. The user sets by considering how much the position of the teaching point may vary.

In the case of the robot program shown in FIG. 4, the straight line L1 to the straight line L4, and the circular arc C1 to the circular arc C4
FIG. 10 shows a position detection program for detecting the position.

In FIG. 10, xp2, xp3, etc.
Teaching points P2 and P3 calculated up to step S103
Are coordinate values in the screen coordinate system. FIG. 12 shows a specific example of an area (edge detection area 1 and edge detection area 2) for detecting a straight line set when obtaining the intersection of two straight lines.
Shown in

In step S105 of FIG. 1, the video camera screen of each teaching point extracted in steps S101 and S102 from the position of a curve such as a straight line or an arc detected by the position detection program generated in step S104. Generates a position calculation program that calculates the actual position in the robot, and a coordinate conversion program that converts the coordinate values in the screen coordinate system of each teaching point calculated by the position calculation program into the coordinate values in the robot coordinate system. I do. Here, the type of the teaching point is, as described above,
It is limited to three types: (1) the intersection of a straight line and a straight line, (2) the boundary between a straight line and an arc, and (3) a point in the middle of the arc. Therefore, for each of these cases, xo = clines (l1, l2, xp1, xp2, x
p3), xo = clinecircle (l1, c1, xp1,
A function or a command of a format such as xp2, xp3, xp4), xo = pcirle (c1, xp1, xp2) is prepared. In these functions or commands, xo is the (x, y) coordinate value of the calculated teaching point in the screen coordinate system. l
1, 12 are the parameters (a, a) of the straight line detected by the position detection program generated in step S104.
b) or an array containing (ρ, θ), and c1 is an array containing the center position (x0, y0) and radius value r of the arc also detected by the position detection program. Also, xp1, xp2, xp3 in the clines ()
Is the coordinate value in the screen coordinate system of the end point of the two line segments specified by the argument, and x in clincircle ()
p1, xp2, xp3, xp4 are coordinate values in the screen coordinate system of both end points of the line segment specified by the argument, both end points of the arc, and intermediate points, and xp1, x in pcirle ()
p2 is the coordinate value in the screen coordinate system of both ends of the arc specified by the argument, and the coordinate value specified by all the arguments uses the value obtained by conversion in step S103. These arguments determine in step S102 from the type of the operation command that includes each teaching point, to what kind of curve each of the teaching points belongs to a curve such as a straight line or an arc.
Since the coordinate value of the teaching point for specifying the range of the curve such as the end point or the halfway point in the determined curve is obtained, the coordinate value can be easily set. In the case of the robot program shown in FIG. 4, position calculation for calculating the actual position on the video camera screen of each teaching point read from the robot program from the position of a curve such as a straight line or an arc. The program is as shown in FIG.

In step S105 of FIG. 1, the values of xf2 to xf13 in the screen coordinate system calculated by the position calculation program shown in FIG. 11 are further converted to values in the camera coordinate system, and these converted values are converted. Generates a coordinate conversion program for converting into a robot coordinate system. Here, a function or command for converting the coordinate value of the teaching point from the screen coordinate system to the camera coordinate system and a function or command for converting the coordinate value of the teaching point from the camera coordinate system to the robot coordinate system are as follows: xc = TrnCam (xf , Plane), xr = TrnCamRob (xc, T) Two functions or commands of the following format are prepared in advance. In these functions or commands,
plane is a coefficient vector of the equation of the plane on which the point is placed. If the point is a part of an arc, the value of the plane equation is calculated as described above, and the value is used. On the other hand, if the point is a part of the line segment, three points including both end points of the line segment and adjacent points among the teaching points obtained in step S101 are used, and a plane equation is calculated and used. T is a transformation matrix of a homogeneous representation from the camera coordinate system to the robot coordinate system. In the case of the robot program shown in FIG. 4, the coordinate conversion program for converting the coordinate values of the teaching point in the screen coordinate system into the coordinate values in the robot coordinate system is as shown in FIG. The part (E) of FIG. 13 is a program for converting the coordinate value of the teaching point from the screen coordinate system to the camera coordinate system, and the part (F) of FIG. 13 converts the coordinate value of the teaching point from the camera coordinate system. This is a program for converting to the robot coordinate system.

FIG. 14 is a diagram for explaining an embodiment of the robot program automatic correction method according to the present invention. As shown in FIG. 14, the robot program automatic correction method according to the present invention uses an image processing program generated by the above-described image processing program automatic generation method for a robot to convert an image captured by a video camera into an actual image. The coordinate value of the teaching point of the work is obtained, and further, based on the obtained coordinate value of the teaching point of the actual work,
This is for correcting the coordinate value of each teaching point previously stored in the robot program. Step S301
Steps S305 to S305 are the same as the corresponding steps S101 to S105 in FIG. 1, and a detailed description thereof will be omitted.

As shown in FIG. 14, first, based on a robot coordinate system for performing a contact work with a work to be worked, such as a welding work or a chamfering work, from a robot program which has been previously taught for a master work. An operation command is extracted, and a teaching point requiring accurate position measurement is extracted from the extracted operation command (step S30).
1).

Subsequently, for each of the teaching points extracted in step S301, it is determined from the type of the operation command including each of the teaching points to which curve among the curves, such as a straight line and a circular arc, each teaching point belongs. At the same time, the coordinate value of the teaching point for specifying the range of the curve such as the end point or the halfway point in the determined curve is read from the robot program (step S302).

Thereafter, the coordinate values in the robot coordinate system of the teaching point extracted in step S301 and the coordinate values in the robot coordinate system of the other teaching points extracted in step S302 are converted into coordinate values in the screen coordinate system. Conversion is performed (step S303). That is, since the coordinate values described in the robot program are coordinate values in the robot coordinate system, these coordinate values are converted into coordinate values in the screen coordinate system using the displacement amount between the robot coordinate system and the camera coordinate system. Convert. The displacement between the robot coordinate system and the camera coordinate system is set in advance by the user.

Then, a position detecting program for detecting the position of a curve such as a straight line or an arc on the video camera screen based on the coordinate value of each teaching point converted in step S303 and the type of the curve to which each teaching point belongs. Is generated (step S304), and the teaching point extracted in steps S101 and S102 from the position of a curve such as a straight line or an arc detected by the position detection program generated in step S304 is displayed on the video camera screen. And a coordinate conversion program for converting the coordinate values in the screen coordinate system of each teaching point calculated by the position calculation program into the coordinate values in the robot coordinate system. Step S305).

Finally, by executing the image processing program generated in steps S304 and S305, the coordinate values of the actual work teaching point are obtained from the image taken by the video camera. Then, based on the coordinate values of the teaching points of the actual work thus obtained, the coordinate values of each teaching point built in the robot program are corrected in advance (step S306).

In the case of the robot program shown in FIG. 4, as shown in FIG. 15, the generated image processing programs (A) to (F) ( The coordinate values of the teaching points xr2 to xr13 of the actual work obtained by executing the image processing program are stored in the corresponding teaching points p2 to p13. (See (G) in FIG. 15). The image processing program generated as shown in FIG. 15 is directly incorporated into the robot program, and the image processing program is executed independently of the robot program, and the coordinate values obtained as a result of execution execute the robot program. The coordinate values of the teaching points stored in advance may be corrected.

As described above, according to the embodiment of the robot program automatic correction method of the present invention, step S30 is performed.
1 to use the image processing program generated in step S305 to determine the coordinate values of the teaching points of the actual work for each work from the image taken by the video camera;
Further, based on the obtained coordinate values of the actual teaching points of the work, the coordinate values of the teaching points incorporated in the robot program as shown in FIG. 4 are automatically corrected in advance (step S306). The robot program can be automatically modified so as to absorb the individual differences between the works only by performing teaching on one master work and creating a robot program.

[0069]

As described above, according to the present invention,
An image processing program for coping with individual differences for each work can be automatically generated from the robot program taught for the master work. Also,
Using the image processing program generated in this way, the coordinate values of each teaching point previously stored in the robot program are automatically corrected, so that teaching is performed for one master work and a robot program is created. Can automatically modify the robot program to absorb individual differences for each work, so even if a person who does not have enough knowledge about image processing can easily use the The robot can be operated in a form absorbing the individual difference of the robot.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining an embodiment of an image processing program automatic generation method for a robot according to the present invention.

FIG. 2 is a diagram showing an embodiment of an apparatus for realizing an automatic image processing program generation method for a robot according to the present invention.

FIG. 3 is a diagram illustrating an example of a work and teaching points of the work.

4 is a diagram showing a robot program for performing a chamfering operation of an edge of the work shown in FIG. 3;

FIG. 5 is a diagram showing a program for converting a coordinate value of a teaching point from a robot coordinate system to a camera coordinate system and a program for converting the coordinate value of a teaching point from a camera coordinate system to a screen coordinate system.

FIG. 6 is a diagram for explaining a routine for detecting a curve such as a straight line or an arc present in a limited area on the screen.

FIG. 7 is a diagram illustrating a specific example of edge enhancement processing on an image captured by a video camera.

FIG. 8 shows how to set a (ρ, θ) parameter space for a straight line (FIG. 8A) and how to determine a straight line parameter (ρ, θ) by Hough transform (FIG. 8B). FIG.

9 is a diagram showing how to set an (x0, y0, r) parameter space for an arc (FIG. 9A) and how to determine an arc parameter by Hough transform (FIG. 9B). .

FIG. 10 is a diagram showing a position detection program created for the robot program shown in FIG.

FIG. 11 is a diagram showing a position calculation program created for the robot program shown in FIG.

FIG. 12 is a diagram showing a specific example of a region for detecting a straight line set when obtaining an intersection of two straight lines.

FIG. 13 is a diagram showing a coordinate conversion program created for the robot program shown in FIG. 4;

FIG. 14 is a diagram for explaining an embodiment of a robot program automatic correction method according to the present invention.

FIG. 15 is a diagram showing a robot program incorporating an image processing program generated by an automatic image processing program generation method for a robot according to the present invention.

[Explanation of symbols]

1 Video camera 2 Image input board 3 Personal computer 4 Robot controller 5 Robot 6 Work

Claims (5)

(57) [Claims] 1. A method of automatically generating an image processing program for a robot performing a work involving contact with a work, wherein a robot coordinate system for performing a work of contacting the work from a robot program which has been previously taught for a master work. A first step of extracting an operation instruction based on the first instruction and extracting pre-stored teaching points from the extracted operation instruction, and including each teaching point for each of the teaching points extracted in the first step. A second step of determining, from the type of the operation command, what curve each teaching point belongs to, and reading out the coordinate value of the teaching point specifying the range of the determined curve from the robot program; The coordinate values in the robot coordinate system of each teaching point read in the second step are taken by taking the previously defined workpiece. A third step of converting into a coordinate value in the screen of the video camera using a displacement amount between the coordinate system of the video camera and the robot coordinate system for performing the teaching, and each teaching point converted by the third step Based on the coordinate value of and the type of the curve to which each teaching point belongs, the position of the curve corresponding to each of the teaching points is detected from the image taken by the video camera, and the actual position of each of the teaching points as an intersection thereof is detected. A fourth step of generating an image processing program for obtaining a position; and a method of automatically generating an image processing program for a robot, comprising: 2. An image processing program generated in the fourth step includes: a position detection program for detecting a position of a curve on a screen of the video camera from an image taken by the video camera; A position calculation program for calculating an actual position on the video camera screen of each of the teaching points read from the robot program from the position of the curve detected by the program; A coordinate conversion program for converting a coordinate value of each of the teaching points on the screen of the video camera into a coordinate value in a robot coordinate system. 3. The program according to claim 1, wherein the position detection program is configured to execute the program based on a coordinate value of each teaching point read in the second step and a type of a curve to which the teaching point belongs from an image captured by the video camera. 3. The automatic image processing program generation method for a robot according to claim 2, wherein a position of a curve is detected within a limited predetermined area in a screen of a video camera. 4. An image processing program automatic generation device for a robot performing a work involving contact with a work, wherein a video camera for photographing a work to be worked, and a robot program previously taught for a master work. A means for extracting an operation command based on a robot coordinate system when performing a contact operation with a workpiece, extracting a pre-built teaching point from the extracted operation command, and for each teaching point extracted by this means, From the type of the operation command including each teaching point, it is determined which curve each teaching point belongs to, and the coordinate value of the teaching point specifying the range of the determined curve is read out from the robot program. Means and a coordinate value in the robot coordinate system of each teaching point read out by this means. Means for converting the coordinate system of the video camera into a coordinate value in the screen of the video camera using the displacement amount between the coordinate system of the video camera and the robot coordinate system, and the coordinate value of each teaching point converted by the means and Image processing for detecting the position of a curve corresponding to each of the teaching points from an image taken by the video camera based on the type of the curve to which the teaching point belongs, and obtaining the actual position of each of the teaching points as an intersection of these An image processing program automatic generation device for a robot, comprising: means for generating a program. 5. An image processing program for a robot according to claim 1, wherein the image processing program automatically generates an actual workpiece from an image captured by the video camera. A step of obtaining coordinate values of the teaching points; and a step of correcting the coordinate values of the teaching points preliminarily incorporated in the robot program based on the coordinate values of the teaching points of the actual work obtained in this step. An automatic correction method for a robot program.