JPH1177568A - Teaching assisting method and device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily carry out the process of a teaching data by an operator, in a device to execute the work by attaching a sensor to an effect container. SOLUTION: By using plural sets of cameras, the moving information of an operator, that is, a three-dimensional posture information is measured at the real time (S1), the position and the posture information of an effect container is obtained (S2), the three-dimensional posture information, and the position and posture information of the effect container are combined, the condition of a robot is provided to the operator through a graphic interface (S3), to carry out a teaching support, so as to process a teaching data to satisfy the sensible sensor posture and the posture of the effect container at a time (S4).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a teaching support method and apparatus, and more particularly to an effector (hereinafter referred to as an "effector") held in a manipulator by inputting position and orientation information of a teaching operator and robot hand position and orientation information into a user interface. The present invention relates to a teaching support method and apparatus for creating teaching data necessary for causing a robot (robot) to move along a predetermined path on a work target.

[0002]

2. Description of the Related Art Conventional robot teaching techniques are described, for example, in Robotics Handbook (Corona, 1990, pp. 5).
22-529), a remote teach method using a teaching box tool provided with push buttons corresponding to the operation direction of a robot is often used.

FIG. 6 is a diagram for explaining a teaching data correction method by a conventional remote teaching method using a teaching box. The system in FIG. 1 includes a manipulator 1, an effector 2, a manipulator control unit 5, and a teaching box 6. The operator uses the teaching box 6 to position the effector 2 on the processing path 4 of the work 3 to be worked, at the tip of the effector 2 or at a position separated from the effector 2 by a certain distance, and the position of the tip of the manipulator 1 at that time.・
The posture coordinates are input and sent to the controller 5 as teaching data.

[0004] This processing is obtained by operating the manipulator 1 a desired number of times along the machining path 4 to obtain all teaching data. In addition, for teaching using only the teaching device, a teaching method that uses sensing information, presents the situation regarding the robot system including the sensor to the teaching operator as support information, and reduces the load on the teaching of the operator is performed. ing. For example, as disclosed in Japanese Patent Application Laid-Open No. 2-15987, in arc welding, the operation of an arc vision sensor at the robot's hand and the robot teaching operation can be switched and executed on the same operation panel, and a work projection image by the sensor is used as teaching support information. There is a method of displaying the information on a CRT to reduce the load on the operator.

[0005]

However, in the above-mentioned conventional teaching data correction method using the remote teaching method using the teaching box, if the work path becomes complicated, the number of teaching points increases, the teaching time increases, and the cost increases. It causes increase. Also, the load on the operator increases.

[0006] In the method using sensing information,
If the path becomes complicated, such as following work on a three-dimensional work, its function is insufficient as teaching support information.
It is difficult to reduce the load of teaching work. As described above, in the work by the manipulator, the predetermined work is achieved by causing the effector to follow the predetermined path surface of the target object (work). In the devices for such work, use of a system in which a sensor is introduced has been increasing in recent years in order to improve work efficiency. Conventionally, in many cases, a teaching-reproduction type manipulator is used.However, in comparison with such a conventional device, in a system incorporating a sensor, environmental uncertainty during work execution, for example, solid There is an advantage that a deviation from the teaching data due to the difference can be dealt with online. However, in order to be able to deal with the environmental uncertainty at the time of work, a teaching work must be performed so as to satisfy the operating conditions of the sensor used at the time of teaching. In general, since the position of the sensor is restricted with respect to the effector,
A point that must be particularly considered among the operating conditions of the sensor is that the position and orientation of the effector must be determined so as to satisfy the detection range of the sensor at the teaching position. In other words, the introduction of the sensor causes a new problem that the sensor position capable of sensing must be determined at the teaching position while satisfying the predetermined effector position / posture.

SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and is a teaching support method in which an operator can easily execute creation of teaching data in an apparatus in which a sensor is attached to an effector and work is performed. And an apparatus.

[0008]

FIG. 1 is a diagram for explaining the principle of the present invention. The present invention provides a teaching support method for acquiring position and orientation information of an effector held by a manipulator from a sensor and creating teaching data so that the manipulator operates on a work target along a predetermined path. , The movement information of the operator, that is, the three-dimensional position and orientation information is measured in real time using a plurality of cameras (step 1), and the position and orientation information of the effector is obtained (step 2). By combining the three-dimensional attitude information and the effector position / orientation information, the robot state is presented to the operator through a graphic interface (step 3), and teaching support is performed to enable sensing sensor attitude and effector attitude. Is created at the same time (step 4).

FIG. 2 is a diagram showing the principle of the present invention. The present invention comprises a manipulator, a controller of the manipulator, an effector held by the manipulator, a sensor held near the effector, and monitor means for monitoring information obtained from the sensor. To move along a predetermined path on the object,
A teaching support device for creating teaching data, wherein a three-dimensional position of an operator is measured using a plurality of cameras.
Three-dimensional position measuring means 100, posture measuring means 200 for measuring the posture of the operator, position and posture information of the operator obtained from three-dimensional position measuring means 100 and posture measuring means 200, and hand position and posture of the effector obtained by the monitoring means Display means 300 for graphically displaying the status of the effector as viewed from the teaching operator using the information.

As described above, according to the present invention, the teaching operation is performed by acquiring the position and orientation information of the operator moving in the work area in real time and combining it with the movement information (position and orientation information) of the effector (robot). The user can understand the state of the robot system as seen from the user with a natural interface using a graphic display, so that the teaching data creation operation can be easily performed.

[0011]

FIG. 3 shows the configuration of a teaching support apparatus according to one embodiment of the present invention. The teaching support device shown in FIG. 1 includes an Ethernet 7, an articulated manipulator 8, a manipulator controller 9, a line sensor controller 10, a line sensor 11, a hand effector 12, CCD camera controllers 15, 16, a position / posture measurement / calculation device 17, Gyro controller 18, CCD cameras 19 and 20, infrared transmission filters 21 and 22, LED (infrared LED) 23, gyro (three-axis angle sensor) 24, graphic display 2
5. It comprises a teaching box 26.

In the above configuration, the line sensor 11
Are mounted on the hand effector 12. The infrared transmission filters 21 and 22 are attached to the lenses of the CCD cameras 19 and 20. The LED 23 is assumed to be attached to the head of the operator. Hereinafter, a method for measuring the position and orientation information of the operator and a support method using graphic information in combination with the position and orientation information of the robot for a user interface will be described with reference to FIG.

In FIG. 1, the hand effector 12 is
It is assumed that work is performed along the work path 14 on the work 13 to be worked. In the following specific example, the hand effector 1
2 is a welding torch and the line sensor 11 is a range sensor. The operation of the manipulator 8 in the figure will be described by taking a welding operation as an example. In the welding work,
The hand effector 12 along the work path 14 of the work 13
That is, the welding torch is brought close to the work 13, welding is continuously performed while following the work path 14, and the work 13 is attached to each other.

In the teaching device, an operator inputs points on the welding route using the teaching box 26 in order to determine the operation route of the robot. In the present teaching device, when creating teaching data in the teaching box 26, the position and orientation of the operator are used to inform the operator of the direction of moving the articulated manipulator 8 based on the sensing data of the welding torch 12 from the range sensor 11. A function of creating teaching data that satisfies the sensor field of view using information and robot position / posture information (welding torch position / posture information) using a graphic interface of a three-dimensional model of the hand is realized.

First, a method for measuring the position and orientation of the operator will be described. The position of the operator is executed by two cameras 19 and 20 and stereo processing. In particular,
An image of the LED 23 on the operator's head is acquired by the camera 19 provided with the infrared transmission filter 21. Since the image is a point light beam of only the LED 23, (x1, y1) in the camera coordinate system can be obtained by binarization using fixed values, which is a simple image preprocessing, and arithmetic operation of the center of gravity. Similarly, the coordinates (x2, y2) are obtained by the camera 20 with the infrared transmission filter 22. From these coordinate values, the three-dimensional position (Xr, Yr, Zr) of the operator's robot in the world coordinate system is calculated by performing stereo processing in the position and orientation calculation device 17. In order to perform stereo vision, it is necessary to perform calibration for obtaining parameters of the cameras 19 and 20 in advance. An embodiment of this method will be described later with reference to FIG.

Also, the posture of the operator (yaw, pit)
ch, roll) is measured using a gyro (three-axis angle sensor) 24 attached near the head. This information is
Graphic operation display device 2 via Ethernet 7
5 and is converted into teaching support information. Next, a method of calibrating the cameras 19 and 20 will be described with reference to FIG.

FIG. 4 is a diagram for explaining a camera calibration method according to one embodiment of the present invention. In order to perform stereo vision, it is necessary to obtain the parameters of the cameras 19 and 20 in advance. Thus, Tsai's algorithm ("An Efficient and Accurat
e CameraCalibration Technique for 3D Machine Visio
n ", Roger Y. Tsai, Proceedingsof IEEE Conference on
Computer Vision and Pattern recognition, Miami Be
ach, FL, 1986, pages 364-37).
Calibration of 9 and 20 is performed.

A reference point for associating the robot coordinate system necessary for executing this algorithm with each camera coordinate system is obtained by moving the infrared LED 32 attached to the hand of the robot (manipulator 33). The calibration uses 60 reference points,
The three-dimensional position of the infrared LED 32 in the robot coordinate system at each position, and the infrared LEDs imaged by cameras 27 and 28 with infrared transmission filters 29 and 30
32, two straight lines L _i , 1, L _i , 2 _,.
Can be obtained, and the three-dimensional position of the infrared LED 32 to be determined is the intersection of these two straight lines _Li , 1, _Li , 2 or the two straight lines _Li , 1, _Li , 2 are in a torsional relationship. In this case, it is calculated as the midpoint of the closest point between two straight lines.

Next, a method of generating support information for the teaching operation of the operator using the position and orientation information of the operator and the position and orientation information of the robot will be described with reference to FIG. FIG.
FIG. 5 is a diagram for explaining creation of support information for a teaching operation of an operator according to an embodiment of the present invention. The system shown in the figure includes a graphic operation display device 35, a graphic interface screen 36 for presenting an operator, and a teaching pendant 37.

The graphics operation display device 35 creates a 3DCG model of the robot hand from the geometric model, and matches the actual tool posture as viewed from the operator calculated from the position and posture information of the operator and the hand position and posture information of the robot. In this way, the 3DCG model is view-transformed and displayed. This 3DCG model is redrawn by moving the operator or changing the position and orientation of the robot. Further, the graphic interface screen 36 for presenting the operator displays the direction in which the robot is to be moved to secure the sensor visual field in a direction superimposed on the 3DCG model by an arrow based on the sensor information attached to the robot hand. A method for determining the direction of the arrow is described in, for example, Japanese Patent Application Laid-Open No. H8-19451.
The method in No. 7 shall be applied. Thus, the operator creates teaching data that satisfies the sensor field of view by performing a predetermined key operation while operating the teaching pendant 37 according to the graphic display.

The graphic operation and display device 35
A mobile terminal of a touch operation input type is also available. It should be noted that the present invention is not limited to the above-described embodiment, but can be variously modified and applied within the scope of the claims.

[0022]

As described above, according to the present invention, in a device in which a sensor is attached to an effector to perform a task, a graphical user interface using the position and orientation information of the operator and the motion information of the robot is provided as support information. As a result, the operator can easily create the teaching data.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining the principle of the present invention.

FIG. 2 is a principle configuration diagram of the present invention.

FIG. 3 is a configuration diagram of a teaching support device according to an embodiment of the present invention.

FIG. 4 is a diagram for explaining a camera calibration method according to one embodiment of the present invention.

FIG. 5 is a diagram for explaining the generation of line-of-sight information for a teaching operation of an operator according to one embodiment of the present invention.

FIG. 6 is a diagram for explaining a teaching data correction method by a remote teaching method using a conventional teaching box.

[Explanation of symbols]

 7 Ethernet 8,33 Articulated manipulator 9 Controller (for manipulator) 10 Line sensor controller 11 Line sensor 12,31 Hand effector 13 Work to be worked 14 Work route 15,16 CCD camera controller 17 Position and orientation measurement arithmetic unit 18 Gyro controller 19 , 20 CCD camera 21, 22, 29, 30 infrared transmission filter 23, 32 infrared LED 24 gyro 25 graphic display 26 teaching box 27, 28 camera 34 surface plate 35 graphic operation display device 36 graphical interface screen 37 teaching pendant 100 three-dimensional Position measuring means 200 Attitude measuring means 300 Display means

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Yoshinori Isoda 3-2-1-2 Nishishinjuku, Shinjuku-ku, Tokyo Nippon Telegraph and Telephone Corporation

Claims (2)

[Claims] 1. Acquiring position and orientation information of an effector held by a manipulator from a sensor, and operating the manipulator on a work target along a predetermined path.
In the teaching support method for creating teaching data, the movement information of an operator, that is, 3
Measures the three-dimensional position and orientation information in real time, acquires the position and orientation information of the effector, combines the three-dimensional position and orientation information with the position and orientation information of the effector, and gives the operator a state of the robot. By using a graphic interface to provide teaching support, thereby creating teaching data that simultaneously satisfies the sensor position that can be sensed and the position of the effector. 2. A manipulator comprising: a manipulator; a controller of the manipulator; an effector held by the manipulator; a sensor held near the effector; and monitoring means for monitoring information obtained from the sensor. A teaching support device that creates teaching data so as to move along a predetermined path on a work target, comprising: a plurality of cameras; a three-dimensional position measuring unit configured to measure a three-dimensional position of an operator; Using attitude measuring means for measuring the attitude of the operator, position and attitude information of the operator obtained by the three-dimensional position measuring means and the attitude measuring means, and hand position and attitude information of the effector obtained from the monitoring means. Display means for graphically displaying the state of the effector as viewed from the teaching operator. Support equipment.