JPH0355194A - Remote operating device for robot - Google Patents

Abstract

PURPOSE:To correct the simulation environment model easily by furnishing a display means to display the real image of the robot working environment received by a communication means and an simulated image produced by a simulated in such an arrangement that the two images are overlaid, and providing a correcting means to make identical the working environment image of the real image on the display means with the simulated image. CONSTITUTION:A real environment image photographed by a photographing means 36 for monitoring the works located in a remote place and an simulated image produced by a simulator 31 are displayed overlaid on a display means 33. If the real environment image becomes identical to the simulated image in this overlaid image, it is judged that there is no difference in the environmental position and attitude. If they are not identical, on the other hand, the object with difference in the position and attitude of the simulated image is indicated using a correcting means 34, and model of the simulated image is overlapped on the real environment image by shifting this object. Then this is noticed to the simulator 31, and correction is made to the position and attitude of the model data of simulator 31.

Description

[Detailed description of the invention] [About Tsutsumi] A remote control system for a robot that remotely controls a robot placed in an unconventionally remote place, such as in space, along with a photographing means for photographing the beam-forming environment, from the ground, etc. The purpose of this project is to create a simulated image of the robot work environment in real time, with the aim of easily correcting any discrepancies between the position and orientation of the simulation environment model and those of the actual environment regarding the operating device. A simulator that performs a simulation to operate the robot while displaying it; a communication means that sends the robot operation information generated by the simulation to the robot II1 and receives real image information of the robot working environment from the robot side; and a communication means that receives the information by the communication means. Real images and simulations of the robot work 'IST5t'! a display means for overlaying and displaying the constructed simulated image;
In order to match the working environment images of the real image and the simulated image on the display means, the position and position of the simulator regarding the environment model are adjusted.
A correction means for correcting the posture data is also provided.

[Field of Application of Industry-L] The present invention enables robots located in remote locations such as outer space to
Concerning remote control devices for robots that are remotely controlled from E, etc.
In particular, the present invention relates to a remote control device for a robot equipped with a function for correcting the position and orientation of an environmental model.

[Background Art] With recent space development, there is a need to have robots perform work in space, where the environmental conditions are harsh. For example, this includes work for building a space station in outer space, or work for various experiments inside an unmanned space station. Since these tasks are performed in space, where humans cannot easily approach them, robots are generally operated remotely from the ground or a space station.

FIG. 7 is a block diagram showing the conventional advantages of such a robot remote control system. In FIG. 7, a robot 20 installed in outer space is equipped with a robot arm l4 controlled by a robot controller l3, and the robot arm 14 is equipped with a television camera l5 and a lighting device 16 for monitoring the working status of the arm. A communication device 12 is provided for transmitting and receiving &Fffl information between the robot controller 20 and the ground. Furthermore, a television camera 17 for capturing the overall appearance of the robot 20 is also installed if possible.

The ground-side device 23 sends and receives robot operation command signals generated by these devices, monitoring image signals from the robot controller, etc. to the space robot 20. The space robot 20 is configured to include a communication device 7 for communication with the space robot 20, a plurality of monitor televisions 8 for displaying real images for monitoring taken by the cameras 16 and 17 of the space robot 20 on the ground side, and the like.

In this remote control system, the operator on the ground can
Using the robot control device 19, the robot arm l4 is remotely controlled to perform a desired task while viewing the actual image of the robot displayed on the television monitor 8.

However, operations using this conventional robot remote control system have various problems as described below.

For example, the orbit of the Space Shuttle, in which a robot may be mounted, is several hundred kilometers high, and its orbit ■.
One period is about 1 to 2 hours. For this reason, there are times when the Space Shuttle's position is on the other side of the earth relative to the ground station, so direct communication between the ground station and the Space Shuttle is not always possible. Communication between ground stations and space robots is carried out via satellite communications.

As a result, the propagation length of the radio waves becomes longer, so the radio wave propagation delay becomes very large. Moreover, since the robot operation by the ground-side device 23 is performed while checking the robot operation on the monitor television 8, after the robot operation command is issued from the ground-side device 23, it is monitored whether the robot 20 in space actually performs the desired movement. The time it takes to confirm on TV 8 is at least twice the propagation delay between the ground equipment and the robot.
Therefore, operating the robot becomes much more difficult than when operating the robot while checking its movements in real time.

Furthermore, satellite communication lines generally have a limited communication capacity, and it is necessary to transmit various control information in addition to robot operation command information through these satellite communication lines. Therefore, the capacity of the communication line that can be assigned to transmit robot operation command information is limited. On the other hand, image information for monitoring the movement of a robot generally has a large amount of information. For this reason, it is impossible for the robot to transmit continuous video signals of the robot's working status to the ground equipment through a small satellite, and the images for robot monitoring are, for example, one per second. It is inevitable that the image will be a semi-still image. Since such semi-still images can only show the movement of the robot intermittently, it is not easy to operate the robot while looking at these still images.

Furthermore, when remotely controlling a robot while watching a television monitor, the camera 15 not only captures an image of the robot arm 14, but also the overall posture of the robot relative to the workpiece. It is necessary to photograph the dolphin with the camera 17 in order to make remote control easier. This is especially true in outer space, where the robot's posture is not fixed.

However, in the case of space robot 20, the robot arm l
The camera 15 for photographing the four parts can be installed on the robot itself, but the camera 15 for photographing the posture of the entire robot can be installed on the robot itself.
7 may not necessarily be available depending on the work environment. Therefore, if the camera 17 is not available, the robot arm 14 is
Remote control will be performed based only on images of the surrounding area,
This operation is generally difficult and requires considerable skill on the part of the operator to perform satisfactorily.

In this way, when remotely controlling a robot in a remote location such as space while viewing a monitor image, semi-still images with movement delays, and in some cases images viewed only from a limited direction such as the arm part, are required. The problem is that it is difficult to operate unless you are very skilled, as you will have to rely on this to operate the aircraft.

Therefore, as a remote control system for robots that alleviates this difficulty in robot operation, we use a graphic display and a computer to check the operation in advance, and then transfer the operation data to the actual robot to perform the same operation. A system has been proposed to do this. This robot remote control system will be explained below.

FIG. 2 shows a block configuration of the robot remote control device described above. This device supports robot 20
This is the case when installed in

In FIG. 2, the robot 20 on the space side has the same configuration as that explained in FIG. It consists of '. Here, the television camera 15 is provided in the robot apparatus so as to monitor the movement of the robot arm 14. In addition, the camera 17 is a camera for photographing the overall position and posture of the robot device and the work object, and there may be multiple cameras for viewing from various directions, but it may not be possible to prepare one depending on the work environment. .

FIG. 3 is a diagram showing an example of a working environment including the space robot 20 and the work object 2l, and this corresponds to an external image of the robot apparatus as viewed from the aforementioned television camera 17. As shown in FIG. 3, this robot device has four wooden arms 14 that perform work while bending with multiple joints, and moves so that the work of these arms 14 can be monitored from an appropriate position. It is equipped with 9 and 15 capable cameras. 2l is a work object such as a space structure, and this assembly is performed by remote control using a robot 20.

The ground-side device 22 includes a control device 1 such as a teaching box, and a robot control device controller 2 that generates operation command signals to the robot by operating the control device l.
, a space robot simulator 3 that simulates the motion of the robot while generating a simulated image (simulation image) through computer processing in accordance with operation command signals from the controller 2; and an image display device for displaying the simulated image obtained by the simulation. (High-definition graphic display) 4. Communication device 7 that sends and receives information to and from the space robot 20 via a satellite line, and the robot 20
one or more monitor televisions 8 for displaying the real images of the robot side received from the robot, a frame scan converter 9, and an overlay for superimposing the robot real image signals and the identification image signals generated from the simulated images on the same screen. The images superimposed by the device 10 and the overlay device 10 are
0 It is configured to include a monitor television 11 for monitoring the pain state of the robot work to be displayed.

The simulator 3 includes a robot data file 5 that stores various data regarding, for example, the structure of the robot 20;
It is equipped with a space environment data file 6 that stores detailed data regarding the space environment in which the robot works and the work object 21, and the controller 2 uses this data based on this data.
The movement of a robot installed in outer space is determined through simulation in response to robot operation command signals from
This is displayed as a simulated image on the image display device 4 in real time.

The display mode of the mock-up image on the image display device 4 is as follows:
For example, as shown in Fig. 4, one screen is divided into four parts, and a perspective image A (
(equivalent to the image taken with camera 17), robot l-20's side image B, and its real image C. Alternatively, a method such as displaying an image D (corresponding to an image taken by the camera 15) that displays the working state of the arm 14 is possible. It is also possible to enlarge and display only one of these 1 1 divided screens to fill the entire screen. Note that FIG. 5 shows a specific example of a simulated image created by simulation of the robot and space structure having the external configuration shown in FIG. 3.

A simple example of the vertical drying device I is shown in FIG. In Figure 6, lOO is the 0 that drives the six axes of the robot arm.
1 to 06 drive buttons, 101 is a hand opening/closing button, 102 is an editing button, 103 is an LED display, 104 is an emergency stop button for stopping the robot at random, 105 is a speed control switch between XYZ mode/multiple station mode, 107 is a speed changeover switch that changes the speed in three stages. Edit button 1 here
02 transmits the robot position information during teaching to the robot operating device controller 2 by pressing these buttons.
It is for storing in memory within.

The operation of this robot remote control device will be explained below. The operator uses the control device 1 to control the movement that the space robot 20 wants to perform, and sends a robot operation command signal to the simulator 3 via the controller 1 2 and the roller 2 . The simulator 3 reads data from the robot data file 5 and the space environment data file 6, and based on these data, calculates the robot motion corresponding to the robot operation command signal through computer processing simulation, and calculates the 4-I of the simulation result. The pseudo image is displayed on the image display device 4 as a continuous moving image in real time.

In this case, since the operator can operate the robot 1 while viewing the simulated images that continuously operate in real time, the operator can easily perform the simulation operation of the robot 1. Furthermore, as the simulated images, a perspective image A of the robot's overall posture, a side image B, a front image C, an image D seen from the top, etc. are arbitrarily obtained through calculation processing.
Since it can be displayed as shown in the figure, it is easier to use this simulated image in the prefecture.

During simulation operation, on the image display device 4, only the movements of the robot 1 that moved normally are retrieved and edited, assembling a series of movements, and after confirming these movements, the robot 1 is operated. Information communication device 7
It is sent to robot 2o in outer space via.

In the robot 20, this operation information is received by the communication device 12, and the robot arm 14 is operated by the controller 1 and the roller 13 based on the operation information. Theoretically, this motion of the robot arm 14 should be the same as the simulated motion of the robot arm 14 performed by the simulator 3. This motion of the robot arm I4 is photographed by a television camera I5 and sent to the ground-side device 22 by the communication device 12 as an actual image of the robot arm 14. Furthermore, if a camera 17 is provided, these captured images are also sent to the ground-side device. In this case, as mentioned above, these real images are ゛1'Wn ILili? in terms of −Oq. It is a statue.

The ground-side device 22 displays the received real image on the monitor television 6. If the camera 17 is present, its image is also displayed on another monitor television 8. Along with this, the actual image signal (NTSC signal) of the television camera l5 is sent to the overlay device lO.

An identification image is manually input to this overlay device IO via a frame scan converter 9.

This identification image is a simulated image created by a simulation and delayed by a predetermined length of time using a simulator of 31 no. TV camera 15
In the case of a captured image, the portion of image D in FIG. 5 corresponding to this image is extracted and converted by the frame scan converter 9 into an NTSC signal for one screen. The predetermined delay time is an actual image taken by the TV camera 15 of the operating state of the robot arm l4 operated by the robots 1 and 20 in response to the sending of the operation information from the robot 1 to the ground device 22. Ground 1-1 side: A17
1'. is set to the time it takes to arrive.

As a result, the actual image and the identified image from the robot 20 should theoretically make the same movement at the same timing.

Therefore, the overlay device 10 superimposes these two image signals on the same screen. As a method of combining the fields, it is possible to make one field image, for example, the identification image, a semi-transparent image, and to make the other image, for example, the real image, as it is.

The overlaid image is from the surveillance monitor TV 11-L.
will be displayed. In this case, the identified image is a continuous moving image, and the actual image is a semi-still image with intermittent motion. Therefore, by confirming that the continuously moving identification image matches the semi-stationary image, it is possible to identify the working state of the robot arm l4 of the robot 20 and the working state based on the simulation. You can confirm that it is working as instructed.

In this way, by overlaying the identification image generated by the simulator 3 with the actual image from the robot 20 and displaying it on one monitor/device Ill, the work of the robot arm 14 can be reduced by the imitation sound that is making the robot. Since it is possible to check whether or not the robot is doing the work correctly just by looking at the single monitor television 11, it becomes very easy to monitor the work of the robot.

[Problem to be solved by the invention] In order for the above-mentioned robot remote control system to operate normally, it is necessary to simulate the actual working environment (i.e., the relative position and posture of the robot and the work object, etc.) in outer space etc. There is a condition that the production environment inside the computer is the same. However, at the start of robot remote control,
Alternatively, when a robot fails in a task, the position and appearance of the work object in the real environment and the simulation environment may not necessarily be the same. Particularly in outer space, where there is swallow gravity, even if a robot fails to grasp a workpiece with its arm, the position and orientation of the workpiece may change significantly as a result.

If there is a discrepancy between the real environment and the simulated large IQ like this, Simiko. It is necessary to correct the position and orientation data of the work object in the ration calculator, but in the past there was no way to easily correct this difference in position and orientation of the environment model.

As a correction method, a method has been proposed in which the position and orientation of the workpiece is measured by some means and correction is made to the model data in the computer, but this method requires a large-scale device and is not practical.

In this way, when the robot fails in its work or the actual position and orientation of the object to be worked on differs from the environmental model data in the computer, it takes a lot of effort to correct the position and orientation. There is a problem in that it requires time and effort.

Therefore, an object of the present invention is to easily correct the position and orientation data of the simulation environment model to match that of the real environment when the position and orientation of the simulation environment model differ from those of the real environment. The purpose is to do so.

[Means to solve the problem] FIG. 1 is a diagram explaining the principle of the present invention.

1 8 The remote control device for a robot according to the present invention is a remote control device for a robot that remotely controls a robot 35 placed at a remote location together with a photographing stage 36 for photographing a robot work environment, and is a device for simulating a robot work environment. A simulator 31 that simulates robot operation while displaying images in real time, and a communication stage that sends robot operation information generated by the simulator 31 to the robot side and receives real image information of the robot work environment from the robot side. 32, the actual image of the robot work environment received by the communication 1000 steps 32 and the I'J generated by the simulator 31.
A 1000-stage display 33 that displays the IiJ image as an overlay;
The simulator 34 is provided with a correction f-stage 34 for correcting the position/orientation data regarding the environment model of the simulator 31 so that the real image of the display stage 33-L and the simulated working environment image match.

[Operation] The real environment image photographed by the work monitor photographing means 36 located at a remote location and the simulated image generated by the simulator 31 are displayed in a superimposed manner on the display 33. In this overlay image, if the real environment image and the simulated image overlap in the same way, it can be determined that there is no difference in the position and orientation of the environment. On the other hand, if these do not overlap, it can be determined that there is a discrepancy between the actual environment and the position/orientation data in the simulator 31.

Therefore, the operator uses the correction step 34 to indicate an object whose position and orientation are different in the simulated image generated by the simulator 31, and shifts the position of the object so that the model of the simulated image overlaps with the real environment image. Operate to. If they overlap completely, the simulator 31 is notified of this, and the position and orientation of the model data inside the simulator 31 are thereby corrected.

[Example] Hereinafter, an example of the robot remote control device according to the present invention will be described. The hardware 20 configuration of this embodiment device is almost the same as the remote control device described in FIG. Positive information manpower means are provided.

This positive information manual means include tablets, mice,
A control stick or the like can be used, and these tools are used to start up the simulation generation environment model to be corrected, rotate it, move it, etc., and correct it to the desired position and orientation.

The position/orientation correction operation of the environmental model by this embodiment device will be explained below.

When starting remote robot operation or when robot work fails, the relative position and position of the robot 20 and work object 21 in space may deviate from the environment model generated by the simulator 3. do. The actual environmental conditions on the space robot side at this time are photographed by television cameras 15 and 17 and transmitted to the ground 21 side device 22 via the information transmission devices 12 and 7.

The ground-side device 22 transfers this real environment image to the overlay device 1.
0 is used to display on the monitoring monitor TVII. At the same time, a simulated image of the space environment generated by the simulator 3 is also displayed on the monitor TVII using the overlay device 10, superimposed on the real environment image.

These real environment images and simulations! ii(f1
If they overlap the same on the screen of monitor TVII, there is no difference in position and orientation between the real environment on the space side and the simulated environment on the ground side, and there is no need for correction. On the other hand, if the two images do not overlap, it can be seen that there is a difference between the position and orientation data of the real environment and the simulation environment model inside the computer.

If there is a discrepancy, the operator uses the correction information input means described above while looking at the screen of the monitor TVI 1 to indicate an object (for example, an object to be supported) that has a different position or orientation in the image generated by the simulator 3, An instruction is given to shift the position of the object by rotating, moving, etc. 22 so that it overlaps with the real environment image.

When both images overlap completely, the simulator computer 3 is notified of this. Then, the simulator 3 corrects the model data it holds so that it has the position and orientation of the model data at the time of matching. After this, the remote control of the robot 1 based on the simulation described above will be resumed.

[Effects of the Invention] According to the present invention, the position and
If your posture differs from that in the real environment,
The position and reward data of the simulation environment model can be easily modified to match that of the real environment.

[Brief explanation of drawings]

FIG. 1 is a diagram explaining the principle of the present invention, FIG. 2 is a block diagram showing a remote control device for a robot based on simulation, and FIG. 3 is an external view showing an example of a robot and a work object 2 3 in outer space. Figure 4 is a diagram showing an example of the split display method of the image display device of the ground side device, Figure 5 is a diagram showing a specific example of the robot image displayed by the split display method of Figure 4, and Figure 6 is the diagram showing the example of the split display method of the image display device of the ground side device. FIG. 7 is an external view showing a specific example of a robot control device of the side device, and a block diagram showing a conventional remote control device for a robot. In the figure, ]...Robot control device 2...Robot control device controller 3...Space robot simulator 4...Image display device (graphic display) 5...Robot data, aisle 6...Space Environmental data files 7, 12... Information communication device 8... Monitor television 9... Frame scan converter 2 4 10 ・ ] 1 ・ l 3 ・ l 4 ・ 1 5, 1 6 ・ 2 0 ・ 2 1 ・2 2 ・...Overlay device...TV for monitoring robot work environment...Robot controller...Robot arm 17...TV camera...Lighting device...Space robot...Space building...Ground-side equipment 2 5 Space External drawing of the robot 3 Drawing A Elephant display Changxia display 4 11 4 Fig. 101 Robot building shrinking Yooki concrete 11 6 Fig. 19 IL Come ita 11 Fig. 7

Claims (1)

[Claims] A remote control device for a robot that remotely controls a robot (35) placed at a remote location together with a photographing means (36) for photographing a robot working environment, the device comprising: A simulator (31) that simulates the operation of the robot (35) while displaying the information, and sends the robot operation information generated by the simulator (31) to the robot side, and receives actual image information of the robot work environment from the robot side. a display means (33) for overlaying and displaying an actual image of the robot working environment received by the communication means (32) and a simulated image generated by the simulator (31); correction means (3) for correcting the position/orientation data regarding the environment model of the simulator (31) so that the working environment images of the real image and the simulated image of the display means (33) match;
4) A remote control device for a robot comprising: