JPH0421105A - Stereoscopic teaching device for manipulator - Google Patents

Abstract

PURPOSE:To attain teaching within a short period by means of a visual system by transmitting output information from a spatial position sensor to a stereoscopic animation forming computer and a manipulator control device. CONSTITUTION:Television cameras 1, 2 are arranged approximately in parallel and a stereoscopic image is displayed on a stereoscopic monitor 4 through a stereoscopic controller 3. An observer puts on electronic shutter spectacles 5 to observe the stereoscopic image. The controller 3 synthesizes a stereoscopic animation formed by a rapid image forming device 6 with the image photographed by the cameras 1, 2 and displays the synthesized image on the monitor 4. When the observer moves a magnetic detecting coil 8, the position and direction of a stereoscopic pointer displayed on the monitor 4 can be optionally changed and a manipulator 14 is driven along a track set up based upon the information through a computer 12 and the manipulator controller 13. Consequently, the moving track of the manipulator can be intuitively and rapidly teached even in an unknown environment.

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial Field of Application] The present invention relates to a teaching device for a manipulator, and particularly to a teaching device using vision.

[Conventional technology]

In order to make a robot perform a task, it is necessary to teach the position of the work object and the movement trajectory of the tip of the arm in some way. Traditionally, this teaching process has been widely used in industrial robots by actually guiding the robot to a target point using inching maneuvers and memorizing the angles or positions of each joint at that position. .

In the above method of using a robot to teach directly, the robot cannot teach while it is performing production activities, and production must be stopped while the robot is teaching, which reduces production efficiency. descend. For this reason, offline teaching is performed in which teaching is performed without using a robot. In this case, a model of the object to be worked on or a robot is held in the computer, and the two numbers are displayed on the CRT screen for teaching. Such offline teaching is discussed in the Journal of the Robotics Society of Japan, Vol. 3, No. 2, pages 59 to 73. However, in this case, there is a slight difference between the positional relationship between the model in the computer and the robot and the actual positional relationship between the workpiece and the robot, and adjustments must be made to correct the positional error before performing the work. There is.

On the other hand, master/slave manipulators, which are used to perform tasks in special environments by remote control, are operated by humans and do not require any instruction. Intelligent manipulators have been developed that use computer control to make them intelligent and perform tasks autonomously. Examples of such are discussed in Proceedings of the Society of Instrument and Control Engineers, Vol. 20, No. 1, pp. 78-85. Generally, with such a manipulator, people are not allowed to enter the work place in many cases, and it is difficult to teach directly using the manipulator.

Additionally, the manipulator is brought into an unknown environment and required to perform work on the spot. For this reason, it is necessary to create a model of the environment and work object on the spot while watching a TV video showing the scene of the work place, and to teach the operation of the manipulator to perform autonomous work.

In this case, when a flat television screen is used, it is not easy to identify the three-dimensional position because there is no depth information. Conventional methods have shown how to use a model already stored in a computer and adjust it to match the actual environment, but this method cannot be applied when used in an unknown environment. . There is also a method of using a laser range finder and a method of observing the position of the laser spot with a television camera.

On the other hand, a stereoscopic vision system that can provide a sense of depth using two monitors has been developed. As shown in FIG. 2, this uses two television cameras 1 and 2 whose video signals are synchronized, and a stereoscopic controller 33 generates a composite image in which the video signals are alternately inserted frame by frame or field by field. When created and displayed on a single monitor 4, images from the two television cameras are alternately displayed on the monitor 4, but if the electronic shutters of the electronic shutter glasses 5 are alternately switched in synchronization with this, the observation To the human eye, the images from the two television cameras 1°2 appear to be separated into the right and left eyes. Therefore, when the two television cameras 1 and 2 are arranged in parallel, images with parallax can be observed, and a stereoscopic sensation based on the parallax can be obtained. It is also possible to create an animation image in real time using the computer graphics device W6, and use the video frame memory to synthesize the real image captured by the television camera and the animation created by the computer, and display it stereoscopically on the monitor. For stereoscopic technology, see S.
, 1. D. Conference Minutes, Volume 19, No. 2 (1978), No. 1
Proceedings of the S
,I,D,,Vof219/ 2 5econd
Quarter (1978) pp. 1-5. No one has used this stereoscopic vision system to teach a manipulator.

A sensor that combines an artificial magnetic field and a detection coil has been developed as a sensor that detects a three-dimensional three-dimensional position in a non-contact manner. As shown in FIG. 3, this system is composed of a source 7 that generates an artificial magnetic field 9, a sensor 8 that detects magnetism, a source drying circuit 10, a sensor detection circuit 11, and a computer 12, which excites the source. The sensor detects magnetism, and the spatial position of the sensor can be calculated based on the strength and direction of the magnetic force.

[Problem to be solved by the invention]

Among the above-mentioned conventional techniques, the method of directly teaching the robot using a robot is undesirable from a safety standpoint because a person enters the operating area of the robot. On the other hand, in the method of teaching using flat TV images, since the images are flat,
Detecting stereoposition is difficult.

In the case of using a laser range finder or observing the position of the laser spot with a TV camera, it is necessary to scan the entire environment, which increases the amount of data and takes processing time, and it is necessary to separate the work target from the background. It is difficult to identify the

If the teaching procedure is complicated or the processing time is long in manipulator work, the advantage of autonomy will be lost, and it will be more efficient to perform the work one by one using a master-slave system. Therefore, the difference between whether or not there is an advantage in making the manipulator autonomous depends on how quickly, simply, and easily teaching can be performed in an unknown environment.

An object of the present invention is to provide a teaching method that allows teaching to be performed in a short time using a visual system.

[Means to solve the problem]

In order to achieve the above object, a visual display system that synthesizes and displays a stereoscopic animation created by computer graphics and a stereoscopic real image captured by two television cameras, a spatial position sensor that detects a spatial position, a manipulator, and This system is composed of a manipulator control system, and is configured so that output information from a spatial position sensor is transmitted to a stereoscopic animation creation computer and a manipulator control device.

[Effect]

The animation creation computer creates two spatial pointer images with parallax to express the spatial position.
This is presented to the viewer using a stereoscopic display system. On the other hand, the observer holds a spatial position sensor, and when he moves it,
A pressure signal indicating a change in spatial position information corresponding to the position change is obtained. This signal is sent to the animation drawing computer. Based on this spatial position information, the computer changes the animation so that the position of the three-dimensional image of the spatial pointer changes. Therefore, the observer can arbitrarily change the position of the spatial pointer using the above spatial position sensor. Further, the manipulator control device similarly receives spatial position information and stores it as a teaching point.

Therefore, I! The observer can teach the manipulator using the spatial position sensor. If each element is adjusted through calibration etc. so that the spatial position on the 3D image corresponds correctly to the spatial position in the actual work space, the observer can move the 3D pointer while looking at the object on the 3D image. can be matched with points representing the characteristics of the object, and the spatial position data at that time can be used to teach a model of the object to the manipulator control device, and the spatial path to the work place can be determined using the spatial pointer. It can also be taught.

〔Example〕

The configuration and operation of an embodiment of the present invention will be explained below with reference to FIG. Two television cameras 1.2 are arranged substantially parallel to each other in the work place, and stereoscopic images are displayed on a stereoscopic monitor 4 via a stereoscopic controller 3. An observer wears electronic shutter eyes [5 to observe this stereoscopic image. A high-speed drawing device 6 is connected to the stereoscopic controller 3, and the stereoscopic animation created by this high-speed drawing device 6 is synthesized with images shot by the television cameras 1 and 2 inside the stereoscopic controller 3, and then displayed on the stereoscopic monitor. 4 will be displayed. Furthermore, a spatial position measurement system using a magnetic sensor is connected to the high-speed drawing device 6.

As shown in FIG. 3, this position measurement system is composed of a source 7 that creates an artificial magnetic field, a magnetic detection coil 8, a source interface device 10, a magnetic detection coil interface device 11, and a computer 12. The three-dimensional position and direction of the magnetic detection coil are obtained from the signal 8, and the information is sent to the high-speed drawing device 6 as shown in FIG. Based on this stereoscopic position information, the high-speed drawing device draws a stereoscopic image of a stereoscopic pointer, which will be described later, and displays it on the stereoscopic monitor 4 via the stereoscopic controller 3. I! When the observer holds the magnetic detection coil 8 and moves it, the position of the stereoscopic image of the stereoscopic pointer displayed on the stereoscopic monitor 4 changes in accordance with the change in its position. Therefore, the position and direction of the three-dimensional pointer can be changed arbitrarily using the magnetic detection coil 4. Further, the position information of the magnetic detection coil 4 or the spatial position information of the three-dimensional pointer is transmitted to the computer 12. or
The information is sent from the high-speed drawing device 6 to the manipulator controller 13, and the manipulator 13 is moved along the trajectory set based on this information.

FIG. 4 shows an example of the display on the stereoscopic monitor 4. On the screen, a work object 15 and a background object 16 not directly related to the work are shown. However, since stereoscopic representation is not possible in this figure, it is drawn as a normal two-dimensional image, but it is assumed that all the figures on the stereoscopic monitor are expressed stereoscopically.

Here, the bolt 17 attached to the work object 15
Assume that the manipulator is taught to automatically perform the task of loosening the . The observer holds the magnetic detection coil and moves the three-dimensional pointer 18. Here, 3D pointer 1
8 is in the shape of an umbrella, and its tip represents the spatial position, and the orientation of the handle represents the direction.

The operator grasps the magnetic detection coil, moves the 3D pointer 18, aligns the tip of the 3D pointer 18 with the desired location on the work object, and sends a teaching signal from the keyboard attached to the high-speed drawing device 6. A three-dimensional teaching point 19 is marked at the tip of the three-dimensional pointer 18. The teaching points are connected by a connecting line 20 such as a straight line, circular arc, or spline curve. In this way, the environment model is created only for the parts necessary for the work. Next, the motion trajectory of the tip of the manipulator 14 is taught. Passage points 21 leading to the target bolt 17 are taught using the three-dimensional pointer 18, and the points 21 are connected with lines 22. Thereby, the movement locus of the tip of the manipulator can be taught three-dimensionally.

The teaching procedure is shown in FIG. 5. First, the three-dimensional pointer is placed on the teaching point to be taught, and a key indicating the teaching is inputted from the keyboard. The animation drawing computer stereoscopically displays a teaching point mark (black circle in FIG. 4) at the tip of the three-dimensional pointer. After teaching the required teaching points for the workpiece and the motion trajectory of the manipulator tip, specify two teaching points for linear interpolation or three teaching points for curved interpolation with the 3D pointer, and move between them. Connect them with straight lines or curves for stereoscopic display. This operation is repeated while there are still points to be taught, and after the teaching is completed, the teaching and interpolation data are transferred to the manipulator controller. The manipulator controller performs autonomous work based on this teaching data. Repeat this step until all work is completed. Here, the teaching data may not be transferred all at once, but may be transferred to the manipulator controller each time each point is taught. Further, the line connecting the teaching points may be connected to the previous teaching point by a transition curve or the like immediately after teaching the teaching point.

FIG. 6 shows another embodiment. In this embodiment, the control arm 23 is used instead of a magnetic sensor as means for detecting the spatial position and moving the position of the three-dimensional pointer 18.

This control arm 23 may also be used to control the manipulator 14. Each joint of the control arm is equipped with an angle sensor 24 that detects the joint angle, and the control arm controller 25 can read the angle of each joint at any time.

The control arm controller 25 is equipped with an arithmetic processing function, and calculates the position and orientation of the arm tip from the dimensions and joint angles of each link that makes up the control arm 23, as is done in industrial robots. , this is the three-dimensional pointer 18
The data is sent to the high-speed drawing bag W6 and the manipulator controller 13 as data for determining the position and orientation of the image forming apparatus. The rest is the same as the embodiment shown in FIG. In this case, there is an advantage that the control arm 23 can be used in common to control the three-dimensional pointer 18 and the manipulator 14, and if the control arm has a motor at each joint and can generate force, the contact with the work object can be caused by a reaction force. There are advantages that can be presented as For example, after teaching the contour of the workpiece, the inside of the contour is set as a prohibited area for the three-dimensional pointer 18. It is always confirmed that the three-dimensional pointer 18 does not enter this prohibited area, and when an intrusion into the prohibited area occurs, the position of the control arm is controlled so as to return to the position immediately before the intrusion. Therefore, even if the operator applies force to the control arm, a reaction force is generated and the three-dimensional pointer 18
cannot penetrate inside the area enclosed by this outline.

Therefore, work instruction becomes easier.

FIG. 7 is a display example of a three-dimensional pointer. 3D pointer 2
6 is a three-dimensional arrow, with the tip indicating the position and the direction of the arrow indicating the direction. The three-dimensional pointer 27 indicates the position by the intersection of the crosshairs in the circle, and indicates the direction by the shape of the circle and the direction of the crosshairs. The three-dimensional pointer 28 is a schematic representation of a hand.

〔Effect of the invention〕

According to the present invention, it is possible to intuitively and quickly create an environment model of the environment or work object in a computer even in an unknown environment, and the movement trajectory of the manipulator can be taught. , it becomes possible to have the manipulator perform work autonomously. In addition, since teaching can be performed intuitively, fatigue caused by teaching work can be reduced.
Further, since the time required for teaching is short, the working time does not increase significantly compared to the case where the robot is operated one by one, and the usefulness of autonomy can be increased.

[Brief explanation of drawings]

Fig. 1 is a block diagram showing an embodiment of the present invention, Fig. 2 is a block diagram of a conventional stereoscopic vision system, Fig. 3 is a block diagram showing a conventional magnetic sensor, and Fig. 4 is a block diagram of a conventional stereoscopic vision system. 5 is a work flow chart showing the procedure of the present invention; FIG. 6 is a block diagram showing another embodiment of the present invention; and FIG. 7 is a display example of the three-dimensional pointer of the present invention. It is an explanatory diagram. 1.2...TV camera, 3...3D controller, 4...Monitor, 5...Shutter glasses, 6...High speed drawing device, 8...Magnetic sensor, 12...Computer,
13... Manipulator controller, 14... Manipulator. 18... Three-dimensional pointer, 19... Three-dimensional teaching point, 20
...Connection line, 21... Passing line, 23... Control arm, 24... Angle sensor, 25... Control arm controller, 31J Sheep drawing foil Otobacterium

Claims (1)

[Claims] 1. A manipulator, a manipulator control system, a stereoscopic video display system that displays the working status of the manipulator, a high-speed drawing system that creates computer animation and displays it superimposed on the stereoscopic video, and a sensor system that detects three-dimensional positions. , the high-speed drawing system draws a three-dimensional pointer that displays a three-dimensional position based on the detection data of the sensor system, the stereoscopic pointer is displayed by the stereoscopic video system, and the manipulator moves based on the detection data of the sensor system. and a three-dimensional teaching device for a manipulator, which is characterized by teaching the position and shape of a work object.