JPH08257948A - Remote control device for robot - Google Patents

Abstract

PURPOSE: To enable a switch to be operated without taking hands off a master arm even when temporarily stopping the control of a robot arm by displaying a control panel in computer graphics, and showing the panel in a head-mounted display. CONSTITUTION: An operator 1 controls a robot arm 9 by controlling a master arm 4 while watching on an HMD 2 an image captured by a slave-side monitor camera 8, and varies the direction of the monitor camera 8 following the movement of the head of the operator 1 which is detected by a magnetic sensor 3. To change the mode of a slave robot, a foot switch 5 is pressed to switch the image on the HMD 2 to an image created in advance on a control panel 12 by a CG originating computer. In this case, a control switch is controlled with a hand shown in computer graphics, so that the operational mode of the robot can be changed at a slave control device 10.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a remote control device for a master-slave robot.

[0002]

2. Description of the Related Art A configuration diagram of a conventional example in which an operator remotely operates a robot arm by using a master arm or a joystick while watching an image sent from a slave robot in an unstructured environment, etc. Shown in FIGS. 3 and 4. In FIG. 3, the operator 1 is a TV
When the operation is performed while looking at the monitor 16, the arrows in the figure show the flow of signals. The operator normally operates the robot arm 9 with the master arm 4, but the operation of the robot arm 9 is temporarily interrupted and the operation coordinate system and operation speed of the robot arm 9 or switching operation such as manual / automatic operation or monitoring is performed. When changing the viewpoint of the camera 8, the operator 1 once releases his / her hand from the master arm 4 that was operating the robot arm 9, and operates the switches on the operation panel 17 arranged in front of the operator 1. Further, when the operator 1 cannot release the hand from the master arm 4, the assistant 1 operates the operation panel 17
Perform the operation. Further, FIG. 4 shows a case where the operator 1 wears the head-mounted display (= HMD) 2 and operates the robot arm 9 while stereoscopically viewing the image transmitted from the surveillance camera 8 of the slave robot. The arrows inside indicate the flow of signals. In this case, by presenting the image from the surveillance camera 8 to the HMD 2 in a three-dimensional manner, it is possible to obtain a realistic sensation as if the user were on the slave robot side. , And the operation panel 17 in front of the operator 1 cannot be seen if the operator leaves the HMD.
I am operating with 2 removed from my head. Further, even in an HMD having a see-through function (a function in which the operator can see the front of the operator's eyes through the optical system without removing the HMD from the head), the operation panel 17 must be operated once the hand is released from the master arm 4. . The above example is applicable when operating a two-arm robot, but when operating a one-arm robot, if the operator's left arm remains, the switch operation may be performed with the left hand. In the example of 4, the HMD 2 is removed from the head and the switch operation is performed.

[0003]

However, in the prior art, each time the switches on the operation panel are operated, it is necessary to remove the hand from the operation device such as the master arm or the joystick or remove the HMD from the head. HMD that has a certain size and does not have a see-through function
The operability of the robot remarkably deteriorated when both the and the operation panel were used together. In addition, since the HMD with the see-through function also passes through the optical system, the visibility in front of the operator is poor,
This also deteriorated the operability. Further, in the case of operating a one-arm robot, even if the left arm of the operator 1 remains, in the example of FIG. 3, the operator shifts the line of sight on the operation panel to operate the switches on the operation panel with the left hand, The annoyance of tilting forward and operating the switch remains. Further, in the example of FIG. 4, the trouble of removing the HMD from the head, shifting the line of sight to the operation panel, and tilting the upper body forward and operating the switch remains. Therefore, the present invention has been made to solve the above problems, and when the operator performs a switch operation or the like, without releasing the hand from the operating device or removing the HMD from the head,
It is another object of the present invention to provide an apparatus that can be operated without significantly shifting the line of sight or tilting the upper body to reduce the burden on the operator and improve the work efficiency of the master-slave robot.

[0004]

SUMMARY OF THE INVENTION In order to solve the above problems, the present invention provides an image pickup apparatus mounted on or near a slave robot for remotely controlling a slave robot located at a remote place. In a remote control device of a robot equipped with a head-mounted display for presenting an actual image captured by the operator to a operator, a computer graphics generation device for generating an operation panel image to be operated by the operator, and the actual image capturing Means for selecting or dividing one of the image of the device and the operation panel image generated by the computer graphics generation device and displaying the divided image on the head-mounted display. To do.

[0005]

Operation: An image showing an operator's hand is generated by the computer graphics generation device, and the hand is moved on the computer graphics by a remote control device to operate the displayed operation panel, and the robot arm is operated. When the switch operation is temporarily interrupted, the switch operation is performed without releasing the hand from the master arm or the joystick or removing the HMD from the head.

[0006]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing a configuration of an embodiment of the present invention, in which a master side transmits an actual image transmitted from a slave robot and computer graphics ( = CG) Head-mounted display (= HMD) 2, which presents a stereoscopic image to the operator 1,
A magnetic sensor 3 for detecting the movement of the operator's head, a master arm 4 for detecting the movement of the arm, a foot switch 5 for detecting the movement of the foot, a computer 6 for CG generation, and a master controller 7 for controlling these. The slave side is composed of a surveillance camera 8, a robot arm 9, and a slave control device 10. The arrows in the figure show the flow of signals. Further, the slave robot may have a traveling function. In addition, the computer 6 for CG generation has an operation coordinate system of the robot arm 9, an operation speed, switching between manual / automatic operation, and switching of traveling start / stop when the slave robot has a moving traveling function. An operation panel with switches for switching the operation mode of the slave robot, a monitor for displaying the state of the slave robot, or the like is generated in advance by computer graphics. FIG. 2 shows a display example of computer graphics on the operation panel presented on the HMD 2. Here, 11 is a display unit frame of the HMD 2, and is composed of an operation panel 12, an operation switch 13, a state monitor 14 for displaying the state of the robot, and a hand 15 for operating the operation switch 13. . Note that both are on computer graphics.

Next, the operation of this apparatus will be described. The operator 1 HMDs the image captured by the surveillance camera 8 on the slave side.
While looking at 2, the master arm 4 is operated to operate the robot arm 9. The surveillance camera 8 follows the movement of the head of the operator by detecting the movement of the head of the operator by the magnetic sensor 3. Here, when the operator changes the mode of the slave robot, he depresses the foot switch 5 to switch the image of the HMD 2 to the image of the operation panel 12 generated in advance by the CG generation computer 6. It should be noted that, without switching, the upper half may be combined and displayed as the actual image and the lower half as the operation panel image. Foot switch 5 when changing modes
Instead of using, a switch arranged at the tip of the master arm 3 may be used, and the computer graphics may be superimposed on the image from the surveillance camera 8 on the slave side.

When operating the switch, the master arm 4
Is switched from the robot arm operation to the switch operation, and the hand 15 moves in the computer graphics in synchronization with the movement of the master arm 4. Then, the operation panel 12 is operated by the hand 15 shown by computer graphics.
When the upper operation switch 13 is operated, a command corresponding to the operation is fetched by the master control device 7 and transmitted to the slave control device 10. Then, the operation mode of the robot is switched according to the command. Here, a joystick may be used instead of the master arm 4,
A magnetic sensor or the like may be attached to the glove that detects the movement of the finger. The operation panel 12 drawn by computer graphics is arranged at a position where the operator 1 can easily operate, the right eye and the left eye are separately displayed on the HMD 2 with an appropriate parallax, and the signal of the magnetic sensor 3 is further displayed on the CG.
If the operation panel 12 is taken in the generation computer 6 so that the line-of-sight direction to the operation panel 12 can be changed in association with the movement of the head of the operator 1, it is as if the operation panel 12 exists in front of one's eyes. You can get a sense of realism.

[0009]

As described above, according to the present invention, the operation panel is a computer graphic, and the computer graphic is a computer graphic.
By presenting it to the MD, even when the operation of the robot arm is temporarily interrupted and the switch is operated, the operator can perform the operation without removing the hand from the master arm or removing the HMD from the head, and the operation panel of the computer graphics is the operator. Since it is located in front of your eyes, you can operate the switch in its posture without shifting your line of sight or tilting your upper body forward, which reduces the operator's burden and improves work efficiency. Has the effect of improving.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of an embodiment of the present invention.

FIG. 2 is a diagram showing a display example of an operation panel in the embodiment of the present invention.

FIG. 3 is a diagram showing a configuration of a conventional example.

FIG. 4 is a diagram showing a configuration of a conventional example.

[Explanation of symbols]

 1 Operator 2 Head Mounted Display (= HMD) 3 Magnetic Sensor 4 Master Arm 5 Foot Switch 6 Computer Graphics Generation Computer 7 Master Control Device 8 Surveillance Camera 9 Robot Arm 10 Slave Control Device 11 HMD Display Frame 12 Operation Panel (Computer graphics) 13 Operation switch (〃) 14 Status display monitor (〃) 15 Hand (〃) 16 TV monitor 17 Operation panel

Claims (4)

[Claims] 1. A head-mounted device for presenting to an operator a real image captured by an imaging device attached to or installed in the vicinity of the slave robot for remote control of the slave robot at a remote place. In a remote control device of a robot having a model display, a computer graphics generation device for generating an operation panel image to be operated by the operator, an image of the real image pickup device, and an image generated by the computer graphics generation device. A remote control device for a robot, comprising means for selecting any one of an operation panel image and displaying it on the head-mounted display. 2. A head mounted for presenting to an operator a real image captured by an imaging device attached to or installed in the vicinity of the slave robot in order to remotely operate the slave robot at a remote place. In a remote control device of a robot having a model display, a computer graphics generation device for generating an operation panel image to be operated by the operator, an image of the real image pickup device, and an image generated by the computer graphics generation device. A remote control device for a robot, comprising: a means for dividing an operation panel image and displaying it on the head-mounted display. 3. An image showing the hand of the operator is generated by the computer graphics generation device, and the operation panel displayed is operated by moving the hand on the computer graphics by a remote control device. A remote control device for a robot according to claim 1 or 2. 4. A detection device for detecting the position / orientation of the head of the operator, wherein the position / position of the head of the operator is detected.
The remote control device for a robot according to claim 1, wherein the operation panel image is changed according to a posture.