JP2925072B2 - Movement control method for robot position teaching - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a movement control method for teaching a position of a robot, and more particularly, to a method for quickly moving a robot to a desired teaching position using a visual target and a visual sensor for recognizing the target. It relates to a movement control method.

[0002]

2. Description of the Related Art The most commonly used position teaching method for a robot is a method using jog feed (a robot operation by manual input). According to this teaching method, the operator operates the jog feed button on the teaching operation panel to move the robot to the desired teaching position, and teach the robot the position at that time.
When the offline teaching is performed, when the rough position teaching is completed, or when the teaching position is corrected, the teaching position is finely adjusted by jog feed.

In any case, the operation of moving the robot to a desired position is performed while visually confirming the relative position / posture relationship between the robot hand (or end effector) and the desired teaching point. Cost. In addition, since the number of points (positions / postures) requiring teaching is generally large, it is not uncommon to spend a great deal of time on teaching work. In addition, in the teaching operation relying on the visual observation of the operator, the teaching accuracy tends to vary, and there is also a problem in reliability.

[0004]

SUMMARY OF THE INVENTION The present invention has been made to overcome such a situation. That is, an object of the present invention is to provide a movement control method for teaching a position of a robot, which can reduce a burden on an operator during a teaching operation and improve the efficiency and reliability of the teaching operation.

[0005]

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems of the prior art, and has a visual sensor and an appropriate visual sensor.
By using a combination of visual target means,
This has solved the above problem by enabling the bot to move to the desired teaching position autonomously . Included in the scheme of the present invention
The various steps include a desired teaching position reaching state storing step of storing desired teaching position reaching state expression data representing a desired teaching position reaching state of the robot in a control means of a system including the robot and the visual sensor. Moving the robot to the desired teaching position by controlling the movement of the robot; and moving the robot to the desired teaching position.
When it reaches, the position data corresponding to the desired teaching position is controlled.
And a teaching position storing step of storing the teaching position in the control means .

[0006] At this stage of moving the desired teaching position, the teaching is performed.
Desired teaching position stored in the indicated desired position reaching state storage stage
Includes autonomous movement stage using arrival state expression data
It is an important feature of the present invention which are.

[0007] To make this autonomous movement possible, the teaching
Desired position reaching state expression data is taught as seen from the visual sensor
Visual target hand in a state equivalent to the desired position arrival state
The steps are obtained through recognition with a visual sensor .

The visual target means is an autonomous moving stage.
When the floor is executed, the robot is guided to the desired teaching position.
Provide visual guidance indicators for

The software processing executed in the control means for the autonomous movement is performed in a storage state of a desired position to be taught.
State represented by stored desired teaching position reaching state expression data
The visual sensor uses the visual guidance index
The robot is guided so as to guide the robot until it reaches the position where the robot is to be moved. The visual target means may take the form of a mark means including a mark coordinate system recognizable by a visual sensor. In that case, the desired teaching position reaching state expression data stored in the desired teaching position reaching state storage stage includes data representing the position and orientation of the mark coordinate system on the image acquired by the camera of the visual sensor. Become. On the other hand, the marking means is used to provide a visual guidance index for the autonomous movement of the robot, and to provide a desired teaching position.
The mark coordinate system at a position with a certain relative relationship to
It is used for. And, by movement control based on software processing executed in the control means ,
Recognition state of mark coordinate system by
Log to a position that matches the recognition status corresponding to the current data.
The robot is guided to reach the bot.

As the mark means, a mark member which can be fixed on the surface of the representative work to be taught is used. On this mark member, a mark coordinate system is drawn in a dot pattern or the like.

As another form of usable visual target means, a light spot formed on a light projecting surface by a light beam projecting means supported by a robot and a desired teaching position are projected.
There is a combination of marks represented on the light surface. this
Is used as the visual target means, in the teaching desired position reaching state storing stage, in order to prepare the teaching desired position reaching equivalent state for the visual sensor, the light spot is set to the robot's teaching target point (corresponding to the teaching desired position). A reference light projecting surface to be formed at a position that coincides with the point to be attempted, which is usually a tool tip point; TCP).

The light beam projection direction is adjusted to form a light spot at the position of the teaching target point on the reference light projection surface. This state corresponds to a teaching position reaching equivalent state for the visual sensor. Accordingly, the light spot is recognized by the visual sensor, and the desired teaching position reaching state expression data is acquired so as to include data representing the position of the light spot on the image.

In the autonomous movement stage, the reference light projecting surface is removed,
And a light beam when a light spot is formed at the teaching target point.
This is executed while maintaining the light projection direction . Autonomous movement stage
The robot movement control in
A light spot formed by projecting a light beam on a surface
The mark indicating the desired teaching position on the light emitting surface
It is used as a mark. And the movement of the robot
Software processing for control is performed by the visual sensor
The desired teaching position reached in the desired position reaching state storage stage
Visual recognition that matches the state represented by the reached state expression data
The robot reaches a position that it does with the guidance index
Until the robot is guided .

Prior to the autonomous movement to the desired teaching position, jog feed may be performed to perform a preliminary approach to the desired teaching position. At that time, it is preferable that the transition from the jog feed stage to the autonomous movement stage is automatically performed by a process in the control means based on the output of the visual sensor.

The software processing at the stage of executing the autonomous movement to the desired teaching position is performed by a visual sensor.
Present on the camera image obtained when recognizing the guidance index
At the stage of storing the data representing the position and the desired teaching position reached state
Processing for comparing with the stored desired position reaching state expression data
Control each axis of the robot based on the
And whether the robot has reached the desired teaching position
The process of determining whether to reach the desired teaching position is performed.
May be repeated successively until it is completed. in this case,
In the teaching position storage stage, it is determined that the desired teaching position has been reached.
Will be executed later.

In the present invention, data of the position on the image of the visual target means which is observed when the robot reaches the desired teaching position is input in advance, and when the robot autonomously moves to the desired teaching position, the visual target means is used. The visual sen
Using the robot as a navigation index
To the desired teaching position.

This feature imposes a burden on the operator.
The robot to reach the desired teaching position quickly without
Position teaching (robot position data when reaching the desired teaching position)
Data storage). In addition, if the transition from jog feed to autonomous movement is performed by automatic switching in the system ,
The convenience of the robot during the position teaching operation can be further improved. Therefore, there are many teaching desired positions.
In such a case, the burden of teaching work can be reduced.

[0018]

FIG. 1 is a diagram conceptually illustrating an overall image of a movement control system for teaching a position of a robot according to the present invention. Reference numeral 1a denotes a desired teaching position reaching state expression data input means for inputting data representing the desired teaching position reaching state. The desired teaching position reaching state expression data input means 1a is a data (for representing the relative positional relationship between the visual target B1 and the robot hand 2a in a state where the movement of the robot 2 with respect to the visual target B1 and others is completed. Hereinafter, the data will be referred to as “taught desired position reaching state expression data”) in the memory 4 in advance.

The desired teaching position is exemplified by four symbols A1 to A4 on the surface 3a of the representative work 3. A1 to A4
B1 to B4 drawn in the vicinity of the above are visual target means representing the positions represented by A1 to A4, and are given in a certain relationship with the corresponding desired teaching positions A1 to A4. Specific examples of the visual target means will be described later.
In a normal form, each visual target position B1 -B
4 and the desired teaching positions A1 to A4 are provided so as to be close to each other but do not coincide with each other. In some cases, however, it is also possible to make them coincide with each other.

Reference numerals 2a and 2b represent a hand position of the robot and a point to be taught (usually a tool tip point; TCP). In the following description, the robot hand position 2a
Is represented by the origin of the coordinate system set on the flange at the end of the final arm.

At the time of actual work, the operator manually operates a robot manual operating means such as a jog feed button (robot operation manual input means; teaching operation panel, etc.).
5, the axis movement control means 6 controls each axis of the robot 2 to start the movement to the desired teaching position. The movement control for autonomous movement to the desired teaching position is performed from the start of the movement or from a certain point after the start of the movement. The movement control for the autonomous movement includes a desired teaching position reaching state expression data input by the desired teaching position reaching state expression data input means 1a, a robot hand (a camera fixed thereto) at each time point, and visual information. This is performed based on relative position recognition data indicating a relative position relationship with the target means.

In the movement control process, the latter data is used as a means for recognizing the relative position between the visual targets of the robot's hand.
given at least once by b. In the desired teaching position reaching state, the TCP 2b naturally coincides with one of the desired teaching positions A1 (or one of A2 to A4) (exemplified by reference numeral 2b ').

As will be described later, the teaching desired position reaching state expression data input means 1a and the relative position recognizing means 1b are embodied using a visual sensor. The visual target means B1 to B4 are embodied in a mark coordinate system recognizable by a visual sensor or in the form of a light spot formed by a laser beam. In the following description, a form in which the visual target means is embodied in the form of a mark coordinate system is referred to as "scheme 1", and a form in which the visual target means is embodied in the form of a light spot formed by a laser beam is referred to as "scheme 2".

FIG. 2 is a schematic block diagram showing a hardware configuration of a system used in the present embodiment, mainly showing a robot controller. A processor board 3 is provided in the robot controller designated by reference numeral 30 as a whole.
The processor board 31 includes a central processing unit (hereinafter, referred to as a CPU) 31a including a microprocessor, a ROM 31b, and a RAM 31c.

The CPU 31a controls the entire robot controller according to a system program stored in the ROM 31b. In the RAM 31c, in addition to the created operation programs and various set values, a program defining the robot-side processing necessary to execute the autonomous movement to the desired teaching position in accordance with the method 1 or the method 2, and related set values, etc. Is stored. A part of the RAM 31c is used for temporary data storage for calculation processing executed by the CPU 31a. Note that a hard disk device or the like prepared as an external device is appropriately used for storing the program data and the set values.

The processor board 31 is connected to a bus 37, and commands and data are exchanged with other parts in the robot controller 30 via the bus connection. First, the digital servo control circuit 32 is connected to the processor board 31, and the CPU 31a
, And drives the servomotors 51 to 56 via the servo amplifier 33. The servo motors 51 to 56 for operating the respective axes are built in the mechanism of each axis of the robot 2.

A serial port 34 having a built-in communication interface is connected to a bus 37 and connected to a teaching operation panel 40 having a liquid crystal display, an image processing device 20, and a laser oscillator 60. However, the laser oscillator 60
Are used in scheme 2 and are not required in scheme 1.

The teaching operation panel 40 has a size and weight that can be carried by an operator, and a jog feed button or the like used as a robot manual operation means is provided on a panel thereof. In addition, a bus 37 has an input / output device (digital I / O) for digital signals.
O) 35, an input / output device for analog signals (analog I / O)
O) 36 are bonded. When it is necessary to exchange signals with the end effector, the control unit of the end effector is connected to the digital I / O 35 or the analog I / O 36. In an example described later, an application of an arc welding robot is considered, so that a power supply device of an arc welding torch is connected to the digital I / O 35.

The image processing apparatus 20 is an ordinary one in which a CPU is connected to a program memory, a frame memory, an image processor, a data memory, a camera interface, and the like via a bus. The image processing device 20 is connected to a camera 21 via a camera interface. This camera is used for photographing for acquiring an image of the visual target means in a manner described later. The program memory stores program data for image analysis required in the method 1 or the method 2.

FIG. 3 is a diagram showing a schematic configuration of a panel surface of the teaching operation panel 40. The display screen 41 is, for example, a liquid crystal screen, and switches and displays detailed data of the movement instruction program. The function key 42 is a key for selecting a menu displayed at the lower end of the display screen 41. The teaching operation panel valid switch 43 is a switch for switching whether the operation of the teaching operation panel 40 is valid or invalid.

The emergency stop button 44 is a button for stopping the operation of the robot 2 in an emergency. Cursor keys 45
These keys are used to move a cursor displayed on the display screen 41. The numeric keypad 46 is provided with numeric keys and other keys, and can input and delete numerical values and characters.

A series of jog feed buttons 47 (J1 to J
6) is a button for inputting a movement command by designating the translation / rotation direction and the + -direction in the normal mode in which the conventional jog feed is performed. In the present embodiment (autonomous movement mode), the button is described later. In this way, it is used as an autonomous movement command input means to a desired teaching position. On the premise of the above, details of the embodiments of the method 1 and the method 2 will be described below.

[Method 1] [1] Outline FIG. 4 is a diagram illustrating the operation of autonomous movement in an embodiment according to method 1. As shown in FIG. 1, desired teaching points A1 to A4 are placed on the surface 3a of the representative work designated by reference numeral 3.
(In this case, four) mark members MK1 corresponding to the number of
To MK4 are attached. On each mark member, the same mark coordinate system is drawn in a dot pattern as described later. The mark member MK1 is affixed with a position / posture that exactly corresponds to the position / posture of the desired teaching point A1. Similarly, the mark members MK2 to MK4 are affixed with positions and postures that exactly correspond to the positions and postures of the desired teaching points A2 to A4, respectively.

The robot showing only the vicinity of the hand has the welding torch 2c and the camera 21 mounted on the hand,
CP2b is set at the tip of welding torch 2c. In the present embodiment, a case is described in which the robot starts from the movement start position Ps, autonomously moves the TCP 2b set at the tip of the welding torch 2c to the desired teaching points A1 to A4, and teaches the position at each desired teaching position. Think. Note that the start position Ps of the autonomous movement is generally arbitrary as long as the first mark MK1 is within the field of view of the camera 21.

FIG. 5 is a view for explaining the configuration of the mark member used in the embodiment according to the method 1. Referring to the figure, the mark coordinate system ΣM has a distance a on the mark member MK.
, Five circular dots D0, D1 arranged in a grid pattern
, D-1, D2, and D-2. Dot interval a
Is a positive constant value. Therefore, the center position of each dot is
D0 (0,0,0), D1 (a, 0,0), D-1 (-
a, 0, 0), D2 (a, a, 0), D-2 (-a, a,
0). Note that the mark coordinate system may be configured with another pattern as long as it can express a three-dimensional orthogonal coordinate system.

A hole MH is provided at an appropriate fixed position of the mark member MK.
Is provided. The hole MH indicates a desired teaching position. When the hole MH is attached on the surface 3a of the representative work 3,
The sticking position is selected such that the representative point (for example, the center) matches the desired teaching position. Attachment posture is mark coordinate systemΣ
Select by referring to the direction of M. That is, if the sticking posture is different,
The attitude taught later will also be different. For example, since the sticking postures of MK1 and MK4 in FIG. 4 are different by 90 degrees, the postures taught later are also different by 90 degrees.

[2] Preparation (Calibration of Camera and Acquisition of Expression Data of State of Attainment of Desired Desired Position) Before starting the work, the camera is calibrated by applying an appropriate calibration method. Various methods are known for camera calibration, but the mark members shown here can also be used. The outline will be described later for convenience of explanation.

2. Acquisition of teaching desired position reaching state expression data is performed. First, the mark member MK is fixed at an appropriate position. Of course, one of the mark members attached to the representative work 3 can be used as it is. First, the robot is moved by the jog operation in the normal mode (conventional method), the tool tip 2b is made to coincide with the representative point MA of the hole MH of the mark member MK, and the desired teaching posture is set. Thus, the state of reaching the desired teaching position with respect to the mark member MK is realized.

3. When the desired teaching position reached state is realized for the mark member MK, the camera 21 is caused to take an image via the robot control device 30 to obtain an image for creating the desired teaching position reaching state representation data. 4. The acquired image is subjected to image processing in the image processing device 20, and the mark coordinate system に 対 す る with respect to the camera coordinate system of the camera 21.
Data representing the relative position and orientation of M is created (details will be described later).

[3] Processing for autonomous movement of desired teaching position in method 1 Movement from the approach start position Ps (expressed as a flange position) shown in FIG. 4 to the desired autonomous teaching position is started, and the desired teaching position Pt ( (Represented by the flange position) will be described. In the following description, the symbol <> is used to represent a vector. 1. Algorithm Skeleton FIG. 7 shows an algorithm skeleton necessary for autonomous movement to a desired teaching position. Current position of flange (position at each time during movement to desired teaching position) T0
, The current relative position of the camera coordinate system and the mark coordinate system (relative position at each point in the process of moving to the desired teaching position) M0, the flange position Tg in the reaching desired teaching position, and the camera coordinate system in the reaching desired teaching position. The geometric relationship with the relative position (target relative position) Mg in the mark coordinate system is as shown in FIG.

Mg is equivalent to the desired position reaching state expression data acquired in the preparation operation. C is taught by camera calibration. The basic equation defining the relationship between T0, M0, Tg, and Mg is as follows. T0 CM0 = Tg CMg (1) Accordingly, the following equation (2) obtained by solving this for Tg is a basic equation representing the movement target position of the robot on the orthogonal coordinate system. Tg = T0 CM0 Mg ^-1 C ^-1 (2) If a position command with this Tg as the final movement target point is given to the servo, the robot can autonomously move to the desired teaching position. I can do it. Therefore, determining the algorithm of the autonomous movement to the desired teaching position results in a problem of specifically obtaining the right side of Expression (2).

In the right side of the equation (2), T0 represents the current position data of the robot, and is data of a property that can be obtained in the robot controller as needed. The data of C is separately acquired by appropriate camera calibration (an example of calibration will be described later). Therefore, the method of obtaining M0 and Mg will be described.

2. The equations M0 and Mg of M0 and Mg both represent the position and orientation of the mark coordinate system ΣM as viewed from the camera coordinate system (current and when the approach is completed).

The mark coordinate system ΣM used in this embodiment
(See FIG. 5), the circular dots D0, D1, D-1,.
Since the arrangement interval a of D2 and D-2 is a constant positive value, a> 0 (3). M0 and Mg are 4 × 4 homogeneous transformation matrices expressing the relationship between the position and orientation between the three-dimensional orthogonal coordinate systems, and can be placed as in the following equation (4).

[0045]

(Equation 1) Here, RM is a 3 × 3 matrix representing rotation, and 1M is a 3 × 1 matrix (vector) representing position. Hereinafter, M0 and M
As an equation satisfied by g, consider finding an equation satisfied by RM and 1M. First, vectors <e1> and <e2> are defined by the following equations (5) and (6).

[0046]

(Equation 2) The information obtained from the visual sensor with respect to the mark coordinate system ΣM is the direction in which the center of each of the circular dots D0, D1, D-1, D2, and D-2 can be seen from the origin of the camera coordinate system (viewing direction). These directions are, as shown in FIG. 6, the five unit vectors <d0>, <d1>, <d-1>, <d2>, <
d-2>.

For the inner product of these vectors, δij defined by the following equation (7) is introduced. δij = <di>. <dj> (i, j = -2, -1,0,1,2) (7) where <d0>, <d1>, <d-1>, <d2>,
Since the lengths of <d-2> are all 1, the absolute value of δij does not exceed 1. | Δij | ≦ 1 (the necessary and sufficient condition for establishing the equal sign is i = j) (8) Further, the distance from the origin of the camera coordinate system to the center of each dot is unknown, but this is represented by ti (i = − 2, -1,0,1,2).
Of course, these take positive values.

Ti> 0 (i = -2, -1,0,1,2) (9) Then, the following equations (10) to (12) are obtained. t0 <d0> = lM (10) t1 <d1> = lM + aRM <e1> ... (11a) t-1 <d-1> = lM-aRM <e1> ... (11b) t2 <D2> = t1 <d1> + aRM <e2> (12a) t-2 <d-2> = t-1 <d-1> + aRM <e2> (12b) Accordingly, ti is obtained. While these equations are expressed as RM and 1M
By solving for, M0 and Mg can be determined. For the specific solution, see 4. A description will be given in the section on how to determine M0 and Mg, and a brief description will be given of C representing the position and orientation of the camera coordinate system.

3. Equation C is a matrix representing the position and orientation of the camera coordinate system viewed from the flange, and its data is obtained by calibration of the camera 21. Various methods are known for calibration and can be used. However, even if the mark coordinate system ΣM is used, the camera 2
1 can also be performed. Here, only the main points are briefly described.

At three different points indicated by reference numerals CB1 to CB3 in FIG. 4, photographing by the camera 21 is executed, image processing is performed, and a relative position between the mark coordinate system ΔM and the origin of the camera coordinate system is obtained. Data Pi (i = 1,2,
3) get. When these are combined with the data Ti (i = 1, 2, 3) of the robot position at the time of each photographing, the following three sets of equations are obtained. The way to find Pi (i = 1,2,3) is M
Same as 0 or Mg. T1 CP1 = T2 CP2 = T3 CP3 (13) This equation can be solved for C under conditions where the robot postures at the three points CB1 to CB3 are different from each other. This condition is defined as (14). Condition: Each posture component of T1, T2, and T3 is different from each other (14) The obtained data of C is stored in the robot control device or the image processing device. The method of solving the equation is described in 5. Supplementary explanation will be given on how to find C.

4. Determination of M0 and Mg In solving equations (10) to (12) to obtain RM and 1M, first consider obtaining ti (i = -2, -1,0,1,2). For this purpose, the following equation (15) is replaced.

[0052]

(Equation 3) Then, the equations (10) to (11) are converted into the following equation (1)
Deforms as shown in 6) to (18). <RM> = t0 RM ^-1 <d0> (16) <rM> + a <e1> = t1 RM ^-1 <d1> (17) <rM> -a <e1> = t-1RM ^-1 <d-1> (18) Here, note that RM ^-1 is a matrix representing rotation, and the following equation (19) is established. (RM- ¹ <di>). (RM- ¹ <dj>) = <di>. <Dj> (i, j = -2, -1,0,1,2) (19) Expression (19), Expressions (5) to (8) and Expressions (16) to (18)
Using the equations, the following relational expressions (20) to (23) are obtained.

[0053] || <rM> || ² = t0 ² ··· (20) || <rM> || ^{2 + 2arMx + a 2 = t1} 2 ··· (21a) || <rM> || ^{2 -2arMx + a 2 = t1 2} · · · (21b) || <rM> ‖ ^{2 + arMx = t0 t1 δ0,1 ···} (22a) || <rM> ‖ ^{2 -arMx = t0 t-1δ0,} -1 ··· (22b) δ0,1 2 <1, δ0, −1 ² <1 (23) From these equations, the conditions (3) and (9) and the equations (22a) and (22)
From b), ‖ <rM> ‖ ² + arMx and aδ0,1, ‖ <rM
> ‖ ² -ArMx and Eideruta0, if organize note that where the -1 becomes each same sign, the following equation (24a) ~ (25b) is obtained.

[0054]

(Equation 4) Equations (24a) and (24b) are further transformed into the following equations (26),
(27) is obtained. ‖ <RM> ‖ ² = (a ² β + ² ) / (1 + β− ² ) (26) rMx = (aβ + β −) / (1 + β− ² ) (27) where β + , Β- are defined by the following equations (28a) and (28b). β + = (α ++ α −) / 2 (28a) β − = (α + −α −) / 2 (28b) Then, the equations (20), (21a) and (21b) are used. (26) (27)
Is substituted and the conditions (3) and (9) are used, t0, t1
And t-1 are obtained by the following equations (29), (30a), and (30b).

[0055]

(Equation 5) Using these, t2, t-2, <1M> and RM
Is required as follows.

[0056]

(Equation 6) 5. How to determine C Under the above-mentioned condition (14), the above-mentioned equation (13) is solved.
By transforming equation (13), the following equations (31) and (32) are obtained. C ^-1 T1 ^-1 T3 C = P1 P3 ^-1 ... (31) C ^-1 T2 ^-1 T3 C = P2 P3 ^-1 ... (32) Here, the following equations (33) to (37) are used. Replace according to

[0057]

(Equation 7) Then, the equations (31) and (32) are converted into the following equations (38) to (41).
Is decomposed into RC- ¹ R1 RC = R2 (38) RC- ¹ R3 RC = R4 (39) (I-R1) <lc> = <l1> -RC <l2> (40) ( I-R3) <lC> = <l3> -RC <l4> (41) Accordingly, the goal is to determine RC and <lC>.
Here, from the forms of equations (38) and (39), it can be seen that the similarities R1 to R2 and R3 to R4 between the rotation matrices and the similarity transformation matrices expressing the similarities between them are RC. Therefore, a vector representing the rotation direction of Ri (i = 1, 2, 3, 4) is defined as <vi> (i = 1, 2, 3, 4), and the following equations (42), (43)
Is obtained. Here, <vi> (i = 1, 2, 3, 4) can be uniquely determined from the condition (14) and the definition expressions (34) and (36). <V1> = RC <v2> (42) <v3> = RC <v4> (43) Furthermore, RC can be obtained by using the following equation (44). (<V1>, <v3>, <v1> × <v3>) = RC (<v2>, <v4>, <v2> × <v4>) (44) Equations (40), (41) <LC> can be obtained by applying the least squares method to the form of the following equation (45).

[0058]

(Equation 8) 6. Outline of processing flow of movement control by method 1
~ 5. Based on the description items described above, an outline of a processing flow for preparation work and movement to a desired autonomous teaching position in the method 1 will be described. First, the supplementary work is as follows. (1) Prior to the start of the work, the robot is sequentially moved to the three positions CB1 to CB3 as described in 3 above. (2) At each of the positions CB1 to CB3, an image of the mark coordinate system ΣM (one) of the mark member MK1 on the representative work 3 is acquired, and the line-of-sight direction of each dot center is represented by image processing in the image processing device 20. Get the data. (3) The above 5. The processing according to the algorithm described in (1) is performed in the image processing apparatus 20, and data (RC, <lC> data) of the camera coordinate system C is calculated and stored.

(4) When the camera calibration is completed, the robot is moved to the position Pt shown in FIG. 4, and the state of reaching the desired teaching position with respect to the mark MK1 appears.

(5) Obtain an image in the mark coordinate system ΣM,
Data representing the line-of-sight direction of each dot center is acquired by image processing in the image processing device 20. (6) The above 1.2.4. The processing according to the algorithm described in (1) is performed in the image processing device 20 to calculate and store the desired position attainment state representation data (Mg data).

The autonomous movement to the desired teaching position is executed by a processing cycle including the following steps as shown in the flowchart of FIG. [K1] <d0>, <d1>, <d-1>, <d2>, <
d-2> is obtained by image processing, and M0 is calculated. [K2] The stored Mg data and <d0>, <d1>, <d-1>, <d2>, <d
-2> is compared with M0 to determine the match / mismatch between the two. If they match, this means that the robot has reached the desired teaching position, and the process proceeds to step K6. If they do not match, it means that the desired position has not been reached, and the process returns to step K1 via steps K3 to K5. Various algorithms can be used to evaluate the coincidence between Mg and M0. For example, the determination index Δ is calculated by the following equation (46), and if Δ <ε (ε is a sufficiently small positive value), the approach is completed, and if Δ ≧ ε, the approach is not completed.

[0062]

(Equation 9) [K3] A matrix T0 of 4 rows and 4 columns representing the position and orientation of the robot's hand (the origin of the coordinate system set on the flange) is calculated and stored. [K4] The right side of equation (2) is calculated using the data (C and Mg data) acquired in the preparatory work and the above-described algorithm to determine the orthogonal movement target position Tg.

[K5] A movement command for moving to the orthogonal movement target position Tg is created and passed to the servo to move the robot. If the system is perfect with no error, it should reach the desired teaching position in one cycle of the above steps K1 to K5. However, due to errors in the visual sensor, calculation errors, etc. The teaching desired position reaching state is successively and approximately achieved. [K6] The robot is stopped, and the process ends.

[Method 2] [1] Outline In the embodiment according to the method 2, a light spot formed by a laser beam is used as a visual target means. First, an outline of the preparation work and the teaching work is shown in FIG.
14 to 14 will be described in addition to the reference drawings. FIG. 9 is a diagram for explaining PR1 to PR3 among various stages of the preparation work,
FIG. 10 is a diagram for explaining the remaining steps PR4 to PR6. FIG. 11 shows TH1 of various stages of the teaching operation.
FIG. 12 is a diagram for explaining steps TH4 and P3.
It is a figure for explaining R5.

[2] Preparatory work (acquisition of teaching desired position reaching state expression data and camera calibration) (PR1) First, in addition to the camera 21, the laser head 22 ( The irradiation head of the laser oscillator 60 shown in FIG. 2) and the reference tool 25 are attached. The laser head 22 is mounted via an appropriate adjusting mechanism 23, and the laser beam 2
4 can be adjusted. The portion indicated by the arrow 27 near the tip of the reference tool 25 has a TCP
A mark 26 representing (2b) is provided. However,
It should be noted that this mark 26 is for visual observation. As shown in the figure, when only the laser head 22 is mounted, the laser beam 24
Does not enter the mark 26 on the reference tool 25.

(PR2) Adjust the direction of the laser head 22 so that the laser beam 24
6. Thus, a light spot is formed at the position of the mark 26, that is, at the position 2b of the TCP. Thereafter, the laser head 22 is fixed in this state.

(PR3) In this state, the camera 21 is caused to take a picture via the robot control device 30 to obtain an image for creating the desired teaching position reaching state expression data. The image of the acquired light spot is stored in the image processing device 20.
The image is processed in the camera coordinate system of the camera 21 (the origin O
Data of a vector <et> representing the direction of the light spot (located at the TCP position 2b) viewed from c) is created and stored. This data will be used later as the desired teaching position reaching state expression data.

(PR4) When the data of <et> is obtained, the reference tool 25 is removed. Since the laser head 22 remains fixed, the laser beam 24 passes through the TCP (2b) in the air as shown.

(PR5) Data of a vector <eh> is created and stored as a vector representing the direction in which the laser beam 24 exists as viewed from the origin Oc of the camera coordinate system. <E
h> is a vector satisfying the following two conditions. Condition 1: the plane <et> and the laser beam 24 extend within a plane. Condition 2: A vector closer to the direction in which the laser head 22 is viewed from the camera 21 among <eh> satisfying <eh> ⊥ <et>. <Eh> can be obtained, for example, by the method shown in FIG.

That is, the robot is moved above a suitable plane (here, the representative work surface 3a is used) by jog feed to form a spot F of the laser beam 24. Then, from the state where the TCP position 2b is above the surface 3a as shown in (1), when the robot approaches the surface 3a, the TCP position 2b comes below the surface 3a as shown in (2). Make a state. In other words, the position of the spot F is made to come to the opposite side from the state of (1) beyond the intersection N of the line of sight 21a of the camera 21 and the surface 3a viewing the TCP position (2b).

In this state, photographing is performed by the camera 21 to obtain an image of the spot F. Then, image processing is performed in the image processing device 20 to obtain data of a vector <es> representing the direction of the spot F viewed from the camera coordinate system. The vector <eh> is calculated by the following equation (47) using this <es>.

[0072]

(Equation 10) (PR6) The camera 21 is calibrated.
This is for obtaining the relationship between the flange coordinate system (2a is the origin) set on the flange surface of the robot hand and the camera coordinate system. As the camera calibration method, various methods can be used as described in the description of the method 1 (a method using a mark member may be used).

[3] Teaching work by method 2 (TH1) First, the robot is moved by jog feed above the representative work surface 3a where the first desired teaching position is located.
A spot F of the laser beam 24 is formed on the representative work surface 3a. The desired teaching position A is preliminarily marked with an appropriate teaching point mark A recognizable by the visual sensor. It is assumed that the flange position 2a in the illustrated state is Ps, and the autonomous movement starts from the position Ps. Note that, as described later, the transition from the jog feed to the autonomous movement can be, for example, automatic switching using a visual sensor output.

As shown, the TCP position 2b is above the surface 3a, and the laser beam 24 forms a spot F on the surface 3a via this point. In this stage TH1, generally, the teaching point mark A, the spot position F and the TC
The three points P (2b) are at different positions. The line of sight 21a viewing the teaching point mark A from the camera 21 is TCP
(2b) does not pass.

(TH2) The robot is operated by the autonomous movement process so that the line of sight 21a viewing the teaching point mark A from the camera 21 passes through the TCP (2b). As the operation of the robot, rotation around the origin Oc of the camera coordinate system is reasonable.

(TH3) The robot is operated by the autonomous movement process, and the spot F is formed close to the teaching point mark A while observing as much as possible the condition that the line of sight 21a passes through the TCP (2b). As the operation of the robot, it is assumed that the rotation in the direction of the line of sight 21a is appropriately combined with the rotation around Oc.

(TH4) When the forward movement in the direction of the line of sight 21a becomes excessive, the line of sight 21a retreats along the direction of the line of sight 21a as appropriate while observing the conditions for passing the TCP (2b) as much as possible, and appropriately combines rotations around Oc. .

(TH5) Thereafter, the operation of TH3 (forward) and the operation of TH4 (retreat) are repeated to converge the spot F formation position to the teaching point mark A. This convergence point is T
Since it is the intersection of the laser beam passing through the TCP (2b) and the line of sight 21a passing through the CP (2b) and looking at the teaching point mark A, the teaching point mark A, the spot position F and the TCP (2b) are ultimately obtained. Will come to the same position. This is the TC to be taught.
This means that P (2b) has reached the desired teaching position.
The position of the flange position 2b at this time is represented by Pt as in the case of the method 1.

Then, if the robot position in this state is stored in the control means (non-volatile memory in the robot controller), the position teaching of the position A is completed. When there are a plurality of desired teaching positions (teaching point marks) (not shown,
For example, see the points A1 to A4 on the representative work 3 in FIG. 1). The above processes TH1 to TH5 may be applied to the second and subsequent teaching point marks.

[4] Processing at Autonomous Movement of Teaching Desired Position in Method 2 According to the process described in the above [2], the autonomous movement start position Ps (expressed as a flange position) is changed to the desired teaching position Pt.
(Similarly, expressed by the flange position) The processing algorithm for achieving the movement will be described. Symbol <> indicates method 1.
A vector is represented as in the case of (1), but the definition contents of the vector are different from those of the method 1. Autonomous movement is shown in FIG.
(L1 to L12) and a processing cycle according to the algorithm outlined in the flowchart of FIG. 16 (L13 to L19). Hereinafter, the algorithm will be mainly described for each step. The meanings of the symbols in the description are as follows.

Σ 0, σ 1; both are flags indicating forward / backward movement of the robot and take a value of ± 1. +1 means forward and -1 means backward. l; a register value <dm> indicating the increment of forward / backward movement of the robot; as shown in FIG. 14, a unit vector (length 1) indicating the line of sight of the teaching point mark A from the origin Oc of the camera coordinate system. ) Is obtained from the image data. <Ds>; as also shown in FIG. 14, a unit vector (length 1) representing the line-of-sight direction of the spot position F (the incident position of the laser beam 24) from the origin Oc of the camera coordinate system, and <dm> Similarly to the above, it is obtained from the image data.

T: matrices δr, δl representing the position and orientation of the robot on the robot coordinate system; both are appropriate thresholds and are positive constants close to 0. R: a rotation matrix that describes the relationship between the directions of <dm> and <et>, and represents the minimum rotation of a 3 × 3 matrix in which R <et> = <dm>. sgn: a sign function defined as follows. sgn (x) = + 1 (x> 0) sgn (x) = 0 (x = 0) sgn (x) = − 1 (x <0) [L1] The flag σ0 is initialized to 1. Also, 1 is initially set to an appropriate initial value (for example, 3 cm). [L2] The register value representing the data of the matrix T is updated based on the current position of the robot. [L3] Shooting is performed using the camera 21, and an attempt is made to obtain the vector <dm> using the image processing device 20. [L4] If the vector <dm> is not obtained,
Since it is considered that the teaching point mark A is not captured in the camera view, the processing is terminated (the operator adjusts the robot position by jog feed to ensure that the teaching point mark A is captured in the camera view. ). Vector <d
If m> is obtained, the process proceeds to step L5.

[L5] Vector <et> and Vector <d
The outer product of <m>, <dr> = <et> × <dm> is determined. [L6] The absolute value of the vector <dr> is compared with the threshold value δr. If smaller, the process proceeds to step L10.

[L7] A rotation matrix R describing the relationship between the directions of <dm> and <et> is calculated by the expression shown. Note that the notation calculation formula is obtained by solving the following simultaneous equations for R. R <et> = <dm> (48) R <dr> = <dr> (49) [L8] A register notation including a new sub-matrix R representing a register value representing data of the matrix T Update to that for.
This matrix solves the following equation (50) for Tg,
Tg → T.

[0085]

[Equation 11] [L9] The robot is operated to perform processing for moving to the position and orientation represented by the matrix T calculated in step L8, and the process returns to step L2. [L10] If it is determined in step L6 that the absolute value of the vector <dr> is smaller than the threshold value δr, this step L10 is executed for the first time. In this step, an attempt is made to obtain a unit vector <ds> representing the line-of-sight direction in which the spot position F is viewed from the origin Oc of the camera coordinate system from the image data.

[L11] At this point in time when the absolute value of the vector <dr> is almost 0 (the line of sight looking at the TCP and the line of sight looking at the teaching point mark substantially match), the fact that the spot F does not fit in the camera's field of view is as follows. It is considered that it cannot occur unless there is a system abnormality or a design error. Therefore, in most cases, the vector <ds> is determined and the step L12
Proceed to. However, if <ds> cannot be obtained, the processing is terminated (the cause is separately investigated).

[L12] It is checked whether <ds> and <et> are in substantially the same direction and have an inner product of 1 (note that δl is a very small positive value). [L13] Flag σ1 indicating forward / backward movement of robot
Σ1 = −1 if the inner product of <ds> and <eh> is positive,
If it is negative, .sigma.1 = 1, and if substantially zero, .sigma.1 = 0. [L14] If .sigma.1 = 0, the process proceeds to step L15. σ
If 1 = 0, the process proceeds to step L16. [L15] σ1 is updated to σ0, and the flow advances to step L18. [L16] If the product of σ1 and σ0 is positive, step L18
Proceed to. If negative, the process proceeds to step L17.

[L17] Update σ0 to σ1 and halve the value of l. [L18] The register value representing the data of the matrix T is updated to the value for the notation matrix. This matrix is obtained by solving the following equation (51) for Tg, and setting Tg → T.

[0089]

(Equation 12) [L19] The robot is operated to perform processing for moving to the position and orientation represented by the matrix T calculated in step L18, and the process returns to step L2. If a determination result of YES is obtained in step L12 at any point in the processing cycle according to the algorithm described above, the state of reaching the desired teaching position as shown in (TH5) of FIG. 12 is realized. And stop the robot to end the processing.

As will be understood from the description of the embodiments of the method 1 and the method 2, according to the present invention, the robot is autonomously controlled so as to attain the position and posture of the pre-input desired teaching position. Is moved to the desired teaching position, thereby improving work efficiency.

Further, in both the first and second embodiments, the control means (the robot controller 30 and the robot controller 30) switches the mode of the autonomous movement toward the desired teaching position from the jog feed mode by manual command input. It is preferable to automate the processing in the image processing apparatus 20).

FIG. 17 is a flowchart showing the operation procedure of the teaching operation panel and the outline of processing when switching from the jog feed mode to the autonomous movement mode is performed inside the system. The processing of each step is briefly described as follows. In this example, the output of the visual sensor is used to determine the timing of mode switching. [S1] It is determined whether or not the jog feed button 47 (any one) has been pressed and a jog feed command has been issued. If so, the process proceeds to step S2.
Repeat 1. [S2] A predetermined axis is controlled in accordance with the movement content (direction, axis, etc.) specified by the jog feed button 47, and the robot 10 starts jog feed movement. For example, J6 axis +
If the movement in the direction is designated, a movement command for moving the J6 axis in the + direction is created and passed to the servo. [S3] It is determined whether or not the jog feed button 47 has been turned off. If the jog feed button 47 has been turned off, the process proceeds to step S8. If not, the process proceeds to step S4. [S4] The presence or absence of a signal indicating that the teaching point mark A (in the case of the method 2 and the spot F) has entered the field of view of the camera 13 from the image processing device 20 is checked. The process proceeds to S5, and if not received, returns to step S3. [S5] Autonomous movement processing is started. Specific examples of the processing contents are as described above, divided into method 1 and method 2. [S6] It is determined whether or not the jog feed button 47 has been turned off. If the jog feed button 47 has been turned off, the process proceeds to step S8, and if not, the process proceeds to step S7. [S7] It is determined whether or not it has reached the desired teaching position. If it has reached, the process proceeds to step S8. If it has not reached, the process returns to step S6. The method of determining whether or not the desired teaching position has been reached is the same as that described in the description of the methods 1 and 2. [S8] The operation of each axis of the robot 2 is stopped.

[0093]

According to the present invention, the data of the position on the image of the visual target means, which is to be observed when reaching the desired teaching position, is input in advance, so that the visual target means can be used as a navigation index. In this manner, the robot can be autonomously moved to the desired teaching position.
Therefore, the burden on the operator required for the position teaching operation of the robot is greatly reduced. In particular, by performing the transition from jog feed to autonomous movement by automatic switching in the system, the convenience of the robot during the position teaching operation can be further improved.

[Brief description of the drawings]

FIG. 1 is a diagram conceptually illustrating an overall image of a movement control method for teaching a position of a robot according to the present invention.

FIG. 2 is a schematic block diagram showing a hardware configuration of a system used in the present embodiment, which is mainly a robot controller.

FIG. 3 is a diagram showing a schematic configuration of a panel surface of a teaching operation panel 40;

FIG. 4 is a diagram illustrating the operation of autonomous movement in an embodiment according to method 1.

FIG. 5 is a diagram illustrating a configuration of a mark member used in an embodiment according to a method 1.

FIG. 6 is a diagram illustrating a vector describing a line-of-sight direction in which a camera looks at a mark coordinate system in an embodiment according to scheme 1.

FIG. 7 is a diagram for explaining an outline of an algorithm required for an autonomous movement of a desired teaching position in the embodiment according to the method 1;

FIG. 8 is a flowchart illustrating an outline of processing of a desired teaching position autonomous movement in the embodiment according to the method 1.

FIG. 9 is a diagram for explaining PR1 to PR3 among various stages of preparation work in the embodiment according to scheme 2.

FIG. 10 is a diagram for explaining PR4 to PR6 of various stages of the preparation work in the embodiment according to the method 2;

FIG. 11 is a diagram for explaining TH1 to TH3 among various stages of a teaching operation in the embodiment according to the method 2;

FIG. 12 is a diagram for explaining TH4 and TH5 among various stages of the teaching operation in the embodiment according to the method 2;

FIG. 13: In the embodiment according to scheme 2, the vector <
FIG. 9 is a diagram for explaining a method for obtaining eh>.

FIG. 14: For the embodiment according to scheme 2, the vector <
dm> and <ds>. FIG.

FIG. 15 is a first half (L1 to L12) of a flowchart describing an outline of an algorithm of a process for controlling autonomous movement in an embodiment according to scheme 2;

FIG. 16 is a second half part (L13 to L19) of a flowchart describing an outline of an algorithm of a process for controlling autonomous movement in the embodiment according to the method 2;

FIG. 17 is a flowchart showing an operation procedure of the teaching operation panel and an outline of processing when switching from the jog feed mode to the autonomous movement mode is performed inside the system in the embodiment according to the method 1 or the method 2; Things.

[Explanation of symbols]

1a Teaching desired position arrival state input means 1b Robot hand tip visual target relative position recognition means 2 Robot 2a Flange representative point (flange coordinate system origin) 2b Teaching target point (TCP; tool tip point) 2b 'Teaching reaching desired teaching position Target point (TCP) 3 Representative work 4 Memory 5 Robot manual operation means 6 Axis movement control means 20 Image processing device 21 Camera 22 Laser head (irradiation part of laser oscillator) 23 Laser head projection direction adjustment function 24 Laser beam 30 Robot Control device 31 Processor board 31a Processor 31b ROM 31c RAM 35 Digital I / O 36 Analog I / O 37 Bus 40 Teaching operation panel 41 Display screen 42 Function key 44 Emergency stop button 45 Cursor key 46 Numeric keypad 47 (J1 J6) Jog feed button 51 to 56 Servo motor 60 Laser oscillator A Teaching point mark (Teaching desired position) A1 to A4 Teaching desired position B1 to B4 Visual target means CB1-3 Calibration position of camera D0, D1, D- 1, D2, D-2 Mark coordinate system dot F spot MK1 to MK4 Mark member ΣM mark coordinate system

──────────────────────────────────────────────────続 き Continuation of front page (72) Inventor Hidetoshi Kumiya 3580 Kobaba, Oshino-son, Oshino-mura, Minamitsuru-gun, Yamanashi Inside FANUC CORPORATION (56) References JP-A-5-233055 (JP, A) 60-262215 (JP, A) JP-A-4-211807 (JP, A) JP-A-60-161627 (JP, A) JP-A-62-20339 (JP, A) (58) Fields investigated (Int. Cl. ^6, DB name) G05B 19/427 B25J 9/10 B25J 9/22 B25J 19/04 G05B 19/19

Claims (6)

(57) [Claims] 1. A robot having a hand and a robot
Camera means attached and obtained through said camera means
Sensor provided with image processing means for processing a selected image
Robot using a system including
Of robot movement control method for teaching robot position
Te, the teachings desired position reaches a state storage step of storing in the control means of the teachings desired position reached state representation data representing the teaching desired position arrival status with respect to at least one of the teachings desired position, the movement control of the robot by the control unit Moving the robot to the desired teaching position; and moving the robot to the desired teaching position.
The position data corresponding to the desired teaching position is recorded in the control means.
And a teaching position storage step of憶, the teaching Willing position moving step, the autonomous based movement to the teachings desired position above teachings desired position reached state representation data
It contains an autonomous moving step to perform the teaching desired position reached state representation data, the visual sensor
In the state equivalent to the state of reaching the desired teaching position
That the visual target means is recognized by the visual sensor.
The visual target means, for guiding the robot to the desired teaching position in the autonomous movement stage.
There is provided a visual guide indicators, movement control of the robot in the autonomous mobile phase is performed by software processing in the control unit, the software processing, the visual sensor, the teaching
The desired teaching position reached in the desired position reaching state storage stage
Visual recognition that matches the state represented by the reached state expression data
The robot arrives at a position to perform with respect to the guidance index.
The above method, further comprising a process for guiding the robot until reaching . 2. The visual target means is a mark means on which a mark coordinate system recognizable by the visual sensor is drawn , and the teaching desired position reaching state expression data is obtained by the camera.
Position and appearance of the mark coordinate system on the acquired image
It includes data representing the energization, the mark means, in the autonomous moving step, the teaching
Mark so that it has a certain relative positional relationship with the desired position.
The movement control method for teaching a position of a robot according to claim 1, wherein the visual guidance index is provided by arranging a robot coordinate system. 3. The mark means is a mark member which can be fixed on a surface of a representative work to be taught,
3. The movement control method for position teaching of a robot according to claim 2, wherein the mark coordinate system is drawn on the mark member. 4. The visual target means includes a light spot formed on a light projecting surface by a light beam projecting means supported by the robot and a desired teaching position on the light projecting surface.
In the storing of the desired teaching position reaching state, the teaching
In order to prepare the desired position arrival equivalent state,
Reference for forming a robot at a point to be taught by the robot
A light projecting surface is arranged, and is provided at the teaching target point on the reference light projecting surface.
Position of the formed light spot on the image by the camera
To reach the desired teaching position so as to include data representing
Expression data is obtained, and the autonomous movement step is performed when the reference light projecting surface is removed and a light spot is formed at the teaching target point.
It is executed while maintaining the light beam projection direction, the movement control of the robot in the autonomous movement stage,
Emits the light beam on the surface where the desired teaching position exists
The light spot formed by
The mark shown on the light surface is used as the guide index.
And the software processing for the movement control of the robot is such that the visual sensor detects that the desired teaching position has been reached.
Attainment desired position reaching state expression data stored in state memory stage
Recognition corresponding to the state represented by the
Robot until the robot reaches a position where
Said method including a process for inducing a cut . 5. The method according to claim 1 , wherein the step of moving the desired teaching position includes the step of
The step of jog feed prior to the step of moving the law is included, and the transition from the jog feed step to the autonomous movement step includes:
The method according to claim 1, wherein the control is automatically performed by the control unit based on an output of the visual sensor.
The movement control method for teaching the position of the robot described in the section. 6. Software in the autonomous movement stage
The processing includes recognition of the visual guidance index by the visual sensor.
Displays the current position on the image obtained by the camera for each
The data to be transmitted and the desired position to be taught position storage state.
Compare with the stored desired position reaching state expression data
Processing and moving each axis of the robot based on the comparison result
Processing for controlling, and the robot
The process of determining whether the position has been reached
Iteratively repeats until the position is determined.
Ri, after the determination of the teachings desired position reached is made, the teaching position
The movement control method for teaching a position of a robot according to any one of claims 1 to 5, wherein a storage step is performed .