JPH11338532A - Teaching device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make precise teaching operation performable for a work object which is poor in shape variation by specifying a part of a figure as a teaching index on a shape model displayed on a screen and calculating and recording the three-dimensional position corresponding to specified coordinates on the screen on the surface of the shape model as teaching data. SOLUTION: A teaching mark 15a is moved on the screen of a display device 15 by operation an input device 17 to indicates the top of the figure 10 as the teaching index or its periphery and on the basis of the coordinates of the indicated teaching point on the screen of the display device 15, a teaching position arithmetic part 18 calculates the two-dimensional coordinates of the teaching point on a virtual image. Further, the three-dimensional coordinates of the corresponding point on the surface of the shape model 8 which corresponds to the two-dimensional coordinates of the teaching point on the virtual image and is uniquely determined is calculated through reverse conversion. This operation is repeated to generate a series of teaching points on the surface of the shape model 8, where an operation track is described, and they are registered as teaching data in a teaching data recording part 19.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a teaching device for teaching the positioning of an automatic working device such as a manipulator, and more particularly to a teaching device effective for teaching the positioning of an automatic working device used for remote operation.

[0002]

2. Description of the Related Art In the teaching of positioning of an automatic working device such as a manipulator used for remote work, images of both the work target and the automatic work device are monitored by a remote TV camera, and the images are automatically directed toward a predetermined work target on the work target. The work device was approached and the work position was taught online. On the other hand, in the field of FA machining / assembly, an off-line teaching method for designating a teaching point on a shape model such as CAD data having a shape similar to the work target has become widespread as a means of teaching positioning. For these technologies, as a teaching means that combines the work efficiency of off-line teaching with the reliability of work confirmation by the actual image of the work target, the stereoscopic real image and the three-dimensional shape model image of the work target are superimposed and displayed. Technology for teaching on the screen (Matsui et al., "Integrated Remote Robot Control Method Using Multimedia Display", Journal of the Robotics Society of Japan, Vol. 6, No. 4, pp 33-41, 1988, etc.)
Is disclosed.

[0003]

With regard to online teaching under the supervision of the first remote TV camera, while checking the monitoring images from multiple directions in order to surely grasp the relative position between the automatic working device and the work target. In order to carefully approach the automatic operation device to the work target, it takes a lot of work time to teach. Further, when the work target is an object having a small change in shape such as a large-sized flat wall surface, there is a problem that a reference for determining a setting position of a teaching point cannot be confirmed on a monitoring screen, and teaching work becomes difficult.

For the second off-line teaching, a shape model that faithfully reproduces the shape of the actual work target is required. If the work content varies widely depending on the situation at the work site, a precise shape model is required. It requires a great deal of production cost. Further, when the data scale is increased due to the refinement of the shape model, there is a problem that the system cost is increased, and the teaching work efficiency is reduced due to a decrease in computer graphic operation speed.

[0005] Regarding the technique of superimposing and displaying the third stereoscopic real image and the three-dimensional shape model image of the work object and teaching on the display screen, since both the real image and the model image are three-dimensional images, each of them is required. There has been a problem that the coordinate and dimension calibration work becomes complicated and the system cost increases.

SUMMARY OF THE INVENTION An object of the present invention is to enable highly reliable off-line teaching using a low-accuracy shape model, to efficiently perform a precise teaching operation on an object having a small change in shape, and to further reduce system cost. It is an object of the present invention to provide a teaching device which is inexpensive and easy to calibrate a shape model.

[0007]

SUMMARY OF THE INVENTION 1) An object of the present invention is to provide at least one monocular surveillance camera for capturing an image of a work target, and camera positioning for arbitrarily setting the position and the line of sight of the surveillance camera with respect to the work target. Means, a model database for storing data of a shape model having a shape substantially similar to the shape of the work target, means for drawing a figure of the teaching index on the surface of the shape model, Virtual image generation means for capturing a shape model from a mutual position and a line-of-sight direction equivalent to the line-of-sight direction and generating a two-dimensional virtual image of the shape model with at least the same angle of view as the imaging performance of the monitoring camera; A display device that superimposes the virtual image generated by the virtual image generation means on the captured real image of the work target and presents it to the operator; Means for designating a part of the figure of the teaching index on the indicated shape model or its vicinity, calculating a three-dimensional position on the surface of the shape model corresponding to the coordinates of the specified screen, and recording as teaching data; This is achieved by configuring the teaching device.

2) An object of the present invention is to provide a camera positioning mechanism for controlling the position and orientation of a monitoring camera in a camera positioning means, and a control device for providing an operator with positioning command information of the camera positioning mechanism. A work simulator for presenting an image of a computer model of the camera positioning mechanism to an operator; updating a visual field of a virtual image generated by a virtual image generating means based on operation information of the camera positioning mechanism operated by the control device; The present invention is characterized in that the position and the shape of the computer model of the camera positioning mechanism with respect to the shape model on the work simulator are updated, and the teaching device according to the means for solving the problem 1) is constituted.

3) As the figure of the teaching index, any one or a combination of a grid figure, a mark indicating the position of a teaching point candidate, a line drawing of a work locus on the surface of the shape model, comment information on the figure, and the like are used. This is achieved by configuring the teaching device described in the means 1) for solving the problem.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to FIGS.

FIG. 1 is a block diagram of this embodiment, and FIG. 2 is an explanatory diagram showing the operation of this embodiment. 1 and 2, the same components are denoted by the same component numbers.

The configuration of this embodiment will be described with reference to FIG.

A teaching device 3 of the present embodiment, which generates and outputs a sequence of teaching point coordinates for forming a work trajectory, to a working arm control device 2 which controls the position and orientation of a working arm 1 which is an automatic working device, A monocular TV camera 5 for photographing the work object 4, a camera arm 6 for controlling the position and the line of sight of the TV camera 5, a camera position / view control device 7 for controlling the TV camera 5 and the camera arm 6, A model database 9 for storing and managing computer data of the shape model 8; a teaching index generator 11 for drawing a figure 10 of a teaching index on the surface of the shape model 8; The shape model 8 in which the figure 10 of the teaching index generated by the teaching index generating unit 11 is drawn is equivalent to the mutual position and the viewing direction of the TV camera 5 with respect to the work target 4. A virtual image generation unit 12 that captures the shape model 8 from the mutual position and the line-of-sight direction and generates a two-dimensional virtual image of the shape model 8 in which imaging conditions such as the angle of view of the TV camera 5 are equalized. Work target 4
Model 8 generated by the virtual image generator 12 on the real image of
An image synthesizing unit 13 that generates a synthesized image in which virtual images of the above are overlapped, a display device 15 that presents the synthesized image to the operator 14, and the operator 14 that provides operation information to the camera position / view control device 7. , An input device 17 such as a mouse for indicating a teaching position on the screen of the display device 15, and a teaching position specified by operating the input device 17 as a teaching mark 15a.
5 on the surface of the shape model 8 uniquely determined based on the virtual image generation conditions in the virtual image generation unit 12 from the two-dimensional coordinates of the teaching position on the screen of the display device 15 specified by the teaching mark 15a. A teaching position calculating section 18 for calculating three-dimensional coordinates of corresponding points; and a teaching data recording section for holding a calculation output result of the teaching position calculating section 18 in time series as teaching point coordinates for forming a work path of the working arm 1. 19, an image of the camera arm model 20 linked to the operator 14 based on the operation information of the camera arm 6 together with the bird's-eye image of the shape model 8, and a sequence of teaching point coordinates stored in the teaching data recording unit 19. Shape model 8 based on
And a work simulator 22 for presenting a virtual playback operation of the work arm model 21 with respect to.

Next, the operation of this embodiment will be described.

The coordinates 5a of the TV camera 5 based on the base coordinates 4a of the work 4 are converted from the position / direction conversion information between the base coordinates 4a and the base coordinates 6a of the camera arm 6 and from the base coordinates 6a to the coordinates 5a. It is calculated based on the position conversion information. The position conversion information from the base coordinates 6a to the coordinates 5a changes with the operation of the camera arm 6. The virtual image generation unit 12 includes a camera model 23 simulating the TV camera 5, and the coordinates 23 a of the camera model 23 based on the base coordinates 8 a of the shape model 8 are the base coordinates 8.
a and the base coordinate 23b from the position a to the base coordinate 23b which is the origin of the movement of the camera model 23
Is calculated from the position conversion information from to the coordinate 23a.
Here, the position / direction conversion information between the base coordinates 8a and the base coordinates 23b includes the base coordinates 4a to the base coordinates 6a.
The calibration is performed in advance in the initial stage of the work so as to match the position / direction conversion information between “a” and “a”. Under these conditions, base coordinates 2
The position conversion information from 3b to coordinates 23a is sequentially updated so as to match the position conversion information from base coordinates 6a to coordinates 5a obtained from the camera position / view control device 7.

With such a configuration, the condition and the line of sight of the camera model 23 with respect to the shape model 8 are set so as to always match the position and the line of sight of the TV camera 5 with respect to the work 4. Further, the camera model 23 has its own position, line-of-sight direction and camera position / view control device 7.
3D shape data of the shape model 8 based on the imaging conditions such as the angle of view of the TV camera 5 obtained from the camera model 2
3 and converted into two-dimensional image data on an imaging surface (not shown). With such a configuration, the virtual image generation unit 12
A virtual image of a shape model 8 equivalent to capturing the work target 4 from the TV camera 5 is generated.

As for the operation of the camera arm 6, the TV camera 5 is moved to an arbitrary position and a line of sight by operating the operation device 16. At this time, the operator 14 proceeds with the operation while checking the two images. One is work simulator 2
2, the relative positional relationship between the actual work object 4 and the camera arm 6 is shown by a bird's-eye view image of the camera arm model 20 with respect to the shape model 8 shown on the screen of the work simulator 22. Check the position. The other image is a composite image presented on the display device 15 and sees detailed information on whether a predetermined teaching position has been reached.

The teaching operation is performed in the following procedure.

As shown in FIG. 2, the display device has a real image 15b of the work object 4 captured by the TV camera 5, and a virtual image 15c of the shape model 8 generated by the virtual image generation unit 12.
Are composited, and a composite image 15d is displayed. Virtual image 1
5c displays only the outline 15h of the surface and the figure 10 of the teaching index viewed from the monitoring direction of the shape model 8, and the other portions are transparently displayed. Therefore, the composite image 15d is the real image 1
An image is obtained by overwriting the outline 15h and the figure 10 of the teaching index on 5b.

With such a screen configuration, the shape model 4
The teaching work can be realized while confirming the changed portion 15f of the work object 4 that did not exist at the time of creating the detailed structure 15e and the shape model 8 omitting the expression. Further, since the teaching point can be determined based on the figure 10 of the teaching index, the setting of the teaching point position can be easily performed even when there is no feature as a structure on the real image 15b.

As the figure 10 of the teaching index, in this embodiment, a lattice figure is presented as an example. For example, if the drawing intervals of the vertical lines and the horizontal lines of the lattice figure are displayed on a prescribed scale, the coordinates of the teaching point can be defined for the work object 4.
Further, as shown in the present embodiment, a line image 1
If 5 g is drawn, the teaching point can be set along this. Further, it is also possible to draw comment information such as the numerical value of the coordinate position and the identification name of the work locus as the figure 10 of the teaching index.

The designation of the teaching point is executed by moving the teaching mark 15a on the screen of the display device 15 by operating the input device 17 and pointing on or near the figure 10 of the teaching index. The teaching position calculator 18 calculates two-dimensional coordinates of the teaching point on the virtual image 15c based on the coordinates of the designated teaching point on the screen of the display device 15. Further, based on the imaging conditions such as the position, the line-of-sight direction, and the angle of view of the camera model 23 itself obtained from the virtual image generation unit 12, the shape model 8 uniquely corresponding to the two-dimensional coordinates of the teaching point on the virtual image 15c is determined. The three-dimensional coordinates of the corresponding point on the surface are subjected to an inverse transformation operation. By repeating this work, a series of teaching points on the surface of the shape model 8 describing the work trajectory is generated,
The teaching work is completed by being registered in the teaching data recording unit 19.
In this processing, the teaching of the position coordinates of the hand of the working arm 1 is performed, but the teaching of the direction of the hand is performed by setting the position of the teaching point such as setting the normal direction of the surface near the teaching position of the shape model 8. Automatic setting based on a specific standard linked to the information, or projecting the work arm model 21 on the composite image 15d equivalently to the shape model 8, and confirming the presence or absence of interference with the work target 4 by virtual steering operation It can be determined by a method such as setting a direction.

After the teaching work is completed, the work arm model 21 is operated on a work simulator based on the teaching data registered in the teaching data recording unit 19 to check whether or not there is interference during the work. Then, the work by the work arm 1 is performed.

According to the embodiment described above, the monocular T
A virtual image of the same shape model as the one taken by the TV camera can be superimposed on the real image of the work target taken by the V camera and presented to the operator, and furthermore, the figure of the teaching index is displayed and operated on the virtual image To the user. Further, a three-dimensional position corresponding to a taught point on the surface of a shape model can be automatically obtained only by specifying a taught point position on a two-dimensional screen of the display device. further,
The field of view of the real image can be set while checking the operation of the camera arm and the photographing range of the TV camera on the work simulator, and a virtual image equivalent to the real image can be generated in conjunction with this.

[0025]

According to the present invention, it is possible to obtain work instruction information on a remote monitoring image as well as to directly draw work instruction information such as a work locus on the surface of a work object. Offline teaching is possible on a virtual image that overlaps the real image, making it difficult for direct workers, especially in nuclear power plant preventive maintenance, space robots, and medical teleoperation, to approach the work target and perform setup work. It has the effect of improving the efficiency and reliability of remote work. Furthermore, since offline teaching is performed on a virtual image that overlaps with the real image of the work target, even if the shape model used for offline teaching is low in accuracy, it can be complemented with the real image, thereby speeding up computer graphics and reducing system cost. This has the effect of reducing the cost and manufacturing cost of the shape model.

Further, since the three-dimensional position on the surface of the work can be automatically taught only by specifying the taught position on the monocular two-dimensional image, the operation is simple, and the efficiency of the teaching operation is improved. Furthermore, since the teaching system is based on a monocular image, it is easier to calibrate the position / view of the TV camera with respect to the work target than the stereoscopic-based teaching system, and the system configuration is also simplified. The system cost can be reduced.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a teaching device according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram showing the operation of FIG. 1;

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Work arm, 2 ... Work arm control device, 3 ... Teaching device, 4 ... Work target, 5 ... TV camera, 6 ... Camera arm, 7 ... Camera position / view control device, 8 ... Shape model,
9: Model database, 10: Graphic of teaching index, 11
... Teaching index generating unit, 12 ... Virtual image generating unit, 13 ... Image synthesizing unit, 14 ... Operator, 15 ... Display device, 16 ... Control device, 17 ... Input device, 18 ... Teaching position calculating unit, 19
... Teaching data recording unit, 20 ... Camera arm model, 21
... Work arm model, 22 ... Work simulator, 23 ...
Camera model.

 ────────────────────────────────────────────────── ─── Continuing from the front page (72) Inventor Nobuhiko Sugawara 3-1-1 Sachimachi, Hitachi-shi, Ibaraki Pref. Hitachi, Ltd. Hitachi Plant

Claims (3)

[Claims] In a teaching device for performing positioning teaching of an automatic working device such as a manipulator, at least one monocular monitoring camera for capturing an image of a working object, and a position and a line of sight of the monitoring camera with respect to the working object are arbitrarily set. Camera positioning means, a model database for storing data of a shape model having a shape substantially similar to the shape of the work object, means for drawing a figure of a teaching index on the surface of the shape model, and monitoring of the work object Virtual image generation means for capturing a shape model from a mutual position and a line-of-sight direction equivalent to the mutual position and line-of-sight direction of the camera and generating a two-dimensional virtual image of the shape model with at least the same angle of view among the imaging conditions of the monitoring camera And superimpose the virtual image generated by the virtual image generation means on the real image of the work target captured by the surveillance camera and present it to the operator. Designate a display device and a part or its vicinity of the figure of the teaching index on the shape model displayed on the screen of the display device, and calculate the three-dimensional position on the surface of the shape model corresponding to the coordinates of the specified screen Means for recording as teaching data. 2. The camera according to claim 1, further comprising: a camera positioning mechanism for operating the position and orientation of the monitoring camera in the camera positioning means; and a control device for providing an operator with positioning command information of the camera positioning mechanism. A work simulator for presenting an image of a computer model to an operator, and updating a visual field of a virtual image generated by the virtual image generating means based on operation information of a camera positioning mechanism operated by a control device, 2. The teaching device according to claim 1, wherein the position and orientation of the computer model of the camera positioning mechanism with respect to the shape model on the work simulator are updated. 3. The graphic of the teaching index may be any one or a combination of a grid graphic, a mark indicating a position of a teaching point candidate, a line drawing of a work locus on the surface of the shape model, comment information on the graphic, and the like. The teaching device according to claim 1, wherein the teaching device is used.