JP3834606B2 - Remote control device for mobile manipulator - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a remote control device for a mobile manipulator that operates a manipulator attached to a mobile body from a remote location, and in particular, the operation state of the manipulator is photographed by a video camera mounted on the mobile body, and a user captures a camera image of a monitor. The present invention relates to a remote control device for a mobile manipulator that operates a manipulator while viewing the screen.
[0002]
[Prior art]
For example, it is difficult for humans to work directly, such as working in a nuclear power plant where it is undesirable for humans to approach, various work in places where volcanic activity is active, work in water, and work in space. In a place, it is preferable that a robot manipulator is used and a person who performs an operation performs a remote work while watching the movement of the manipulator. In particular, fixing this manipulator to a moving body and operating the manipulator while maneuvering the moving body expands the work range and also expands the range of use of the manipulator, so research and development to improve its usability is expected. Yes.
[0003]
In general, when operating a manipulator as a robot arm remotely as described above, the operator confirms that the work is being performed as intended or corrects an unexpected situation. For example, if this manipulator is remote, or if there is a partition that cannot be removed between the place where the manipulator is operated and the place where the manipulator is operated, for example, As shown in FIG. 3, the work situation of the manipulator 11 is photographed by the video camera 12, and the photographed camera image is projected on the monitor 13 and the operator 14 is observed while operating the remote controller 15 such as a joystick to operate the manipulator. 11 objects are gripped and moved.
[0004]
On the other hand, when the manipulator 11 is mounted on the moving body 16 as shown in FIG. 3, the operator further controls the moving body, moves the moving body to the object to be handled, and the manipulator as described above. In addition, the mobile body is moved while holding the object as necessary, and the object is transported to a predetermined place or assembled. When the manipulator is attached to the moving body as described above, the moving range is particularly large, so the video camera is fixed to the moving body, and when the moving body is moved, the vicinity of the moving body is photographed, and the manipulator is operated. At this time, it is necessary to perform difficult operations such as zooming up the video camera to perform an accurate operation and performing various operations depending on these images of the video camera while operating the video camera.
[0005]
[Problems to be solved by the invention]
When a manipulator and a video camera are mounted on the moving body as described above, and the operator performs operations and operations while observing the video camera image on the monitor, especially when performing fine operations with the grip at the tip of the manipulator Then, the video camera is zoomed up in a limited range centered on the gripping portion, and the resolution on the monitor is increased, and the attention is concentrated on this portion to perform operations such as gripping.
[0006]
As a result, if the work object is spreading to the full field of view during work, or if the background has no features and is uniform, it can be obtained from the on-board camera obtained when operating the work object with the manipulator. The video information of is only the relative movement between the work object and the moving manipulator, and the operator cannot recognize the movement sensation as to whether the work object is moving or the moving manipulator has moved. There is a problem.
[0007]
For example, when a mobile manipulator is operated to transport a load from a certain place, the mobile manipulator has a manipulator having a sufficient degree of freedom for performing work on the moving base and a video camera for observing the work situation. The pilot performs necessary operations with the joystick while looking at the camera image of the area around the hand. At this time, if you try to pull the load and the load is fixed or heavier than you might expect, a slip will occur between the base movement mechanism (wheels, etc.) and the ground, Instead of attracting, the robot may be attracted towards the luggage. The same applies to pushing a load.
[0008]
Since the camera mounted on the mobile robot has a limited angle of view, the operator can only obtain a limited view of the manipulator's hand and its surroundings. At this time, there is no difference in the relative movement between when the work object is moved and when the manipulator base slips and moves. If the background has no features and is uniform, they appear to move almost the same on the screen that the operator is viewing. Actually, even if the base moves due to slipping, the operator may have an illusion that the operation object has moved as intended, making it difficult to perform an accurate operation.
[0009]
This becomes a particularly important problem when the degree of freedom of the base of the manipulator increases, such as an underwater exploration robot and a space manipulator, and the moving body easily moves. For example, when an underwater exploration robot stretches a manipulator to pick up rocks on the seabed and tries to lift rocks, the hull may be drawn to the seabed, depending on the weight of the rocks and how they stick to the seabed. In such a case, if it can be detected that the hull is being pulled and this can be presented to the pilot, the pilot can adjust the hull attitude to increase the fluid resistance or adjust the screw output. And so on.
[0010]
For such a problem, conventionally, for example, the following measures have been taken as a method for presenting the status of the mobile manipulator at a remote location to the operator. One method is to use the zoom function of a video camera. That is, this method uses the zoom function of the on-board camera, and is a method of presenting to the operator while repeating a zoomed-in narrow viewing angle image and a zoomed-out wide viewing angle image. According to this method, it is necessary to operate the zoom function simultaneously with the operation of the manipulator, and there is a disadvantage that the recognition of the relative motion between the operation target and the mobile robot is lowered because the sense of distance changes during the operation of the zoom function. .
[0011]
As a second method, there is a method of simultaneously displaying a wide viewing angle image. This method is a method in which a plurality of cameras are mounted on a moving body and a zoomed-out wide viewing angle image is presented to a pilot simultaneously with a work narrow viewing angle image. According to this method, since the person who operates cannot concentrate on two or more images at the same time, the two displayed images must be switched as necessary. There is a drawback in that workability is reduced due to a change in the line of sight.
[0012]
As a third method, there is a method of displaying some message on the monitor. In this method, the movement of the moving mechanism is detected by a separately provided sensor, and sensor information such as the moving state is displayed on the camera image screen of the monitor that is continuously observed during work. This method has the advantage that the movement information of the moving mechanism can be transmitted by a simple method of overwriting the message on the camera image displayed on the monitor, but the person who performed the operation saw such a message Alone has the disadvantage that it is difficult to intuitively convey information on which direction and how much movement is occurring.
[0013]
As a fourth method, there is a method of outputting a warning sound or a voice guide. In this method, the movement of the moving mechanism is detected by a sensor, and the movement of the moving mechanism is presented to the operator by a warning sound or synthesized voice. This method has the advantage of being able to notify the occurrence of movement of the moving mechanism with the addition of a simple device, but has the disadvantage that it is difficult to convey detailed information such as in which direction the movement is occurring. .
[0014]
As a fifth method, there is a method using a motion platform. This method detects the motion of the moving mechanism with a sensor, and simulates the motion of the moving mechanism by using a motion platform used in an aircraft flight simulator, an automobile drive simulator, etc., and presents it to the operator. Is the method. This method has the advantage that it is possible to present a natural feeling as if the operator is riding on the moving mechanism and operating the manipulator, but it requires a large and complex control device and is expensive. However, it is difficult to spread widely economically.
[0015]
As described above, various methods have been studied in the past, but they have not been widely used because they all have major drawbacks.
[0016]
Therefore, according to the present invention, when a manipulator mounted on a moving body is operated from a remote location, the periphery of the manipulator is photographed by a video camera provided on the moving body, and this is displayed on a monitor installed on the driver's side. It is an object of the present invention to provide a mobile manipulator remote control device that allows a driver to grasp the state of a manipulator and the state of a moving body at a glance by operating simple means.
[0017]
[Means for Solving the Problems]
In order to solve the above-mentioned problems, the invention according to claim 1 shoots a manipulator attached to a moving body with a video camera mounted on the moving body, and displays the shot image on a remote monitor to display the manipulator. In a remote control device for a mobile manipulator for manipulating an object, a synthesis is provided based on a fixed absolute coordinate system that includes a movement detection means for detecting a slip of a moving body and indicates a state in which no slip occurs according to a signal of the movement detection means The present invention provides a remote control device for a mobile manipulator characterized in that an image is displayed superimposed on the captured video.
[0019]
The invention according to claim 2 is the remote control device for a mobile manipulator according to claim 1, wherein the composite image based on the absolute coordinate system is a grid.
[0021]
The invention according to claim 3 is characterized in that the image based on the absolute coordinate system is an image showing a combined image of a manipulator based on an operation command at a hand position where a slip of the moving body should be present. The mobile manipulator remote control device according to claim 1.
[0022]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described with reference to the drawings. FIG. 1 shows a first aspect of an embodiment of the present invention. In FIG. 1, when a driver operates a moving manipulator with a remote controller, the vicinity of the tip of the manipulator is photographed with a video camera mounted on the moving manipulator. The example of the monitor screen which zooms up and displays this part is shown. A monitor screen 2 in FIG. 1 shows a photographed image 1 of a manipulator zooming in on the vicinity of the tip of the manipulator at the approximate center of the monitor screen 2, while a grid 3 of an absolute coordinate system is displayed on the screen as a computer graphic. Are synthesized and displayed by the method of The grid 3 is displayed by perspective as shown in the figure. When the moving body 26 moves relative to the floor surface, this is detected by a separately provided sensor, and the grid moves in accordance with the movement. A grid as shown in the figure is drawn on the floor surface photographed by the camera mounted on the moving body, and image processing is performed and displayed as if it were photographed.
[0023]
By performing the display as described above on the monitor screen 2, when the operator moves the moving body with the remote control, this grid also moves, so that the moving body can be moved in the outer space, the sea floor, or in a long pipe. Recognize the speed, distance, etc. of moving objects clearly even when the surroundings are monotonous, flat, or curved, such as when the moving state is not obvious from the video camera image. Can do.
[0024]
In addition, after the moving body stops at a predetermined position, for example, when an object is grasped and moved by a manipulator, the grid indicating the absolute coordinate system does not move, and only the grasped object is moved. It is possible to know that the gripping object is gripped. On the other hand, unlike the pilot's imagination, if the object is heavy, or the object is moving like trying to move the rock when part of a large rock on the seabed is appearing on the seabed When the object cannot be moved, the moving object slips and moves on the floor surface by the force of moving the object with the manipulator. At that time, it is detected by the sensor provided on the moving body that the moving body is moving, and the grid 3 displayed on the screen is moved in accordance with the movement.
[0025]
As a result, the pilot knows that the object is unexpectedly heavy or solid, by seeing the grid move toward you as shown by the white arrow on the screen in FIG. Can do. At this time, when a large force is applied to the moving body and the moving body slips in the lateral direction, the grid displayed on the screen also moves in the lateral direction, and the operator can know the state. Further, the operator can accurately know the moving speed, moving distance, and moving direction by looking at the moving speed, moving amount, and direction of the grid shown on the screen at that time.
[0026]
When the pilot knows that the moving body is moving in this way, the pilot stops the operation or moves the object so that the moving body does not move by increasing the propulsion force of the submarine work boat, for example. Can be performed. The grid display can always be displayed on the monitor screen, but it may be displayed from the moment when it is detected that the moving body slips.
[0027]
FIG. 2 shows a second aspect of the embodiment of the present invention. In the same manner as shown in FIG. 1, the vicinity of the tip of the manipulator is photographed with a video camera mounted on the moving manipulator. An example of a monitor screen 2 displaying a zoomed-in portion is shown. In the example of the screen displayed here, the captured image 1 of the manipulator displayed at the center in the figure is an actual image. Show.
[0028]
Also, the image of the manipulator displayed on the right side of the figure in the broken line shows that when the operator tried to grip and carry the object with the manipulator, the object was unexpectedly heavy or fixed as described above FIG. 5 shows an image of a manipulator displayed in an absolute coordinate system by computer graphics when it is assumed that the manipulator exists at a position immediately before the moving body slips when the moving body slips and moves.
[0029]
Since the manipulator showing the position immediately before the moving body slips is displayed on the screen displaying the image centering on the tip of the manipulator in this way, the person who is operating the slip of the moving body It can be grasped sensuously and appropriate measures can be taken. At this time, it is preferable to adopt a means for making it easy to see, such as making a computer graphics image a color having a large contrast with respect to an actual image so that both images can be easily identified. As the means for detecting the slip, slip detecting means widely used in vehicle control devices can be used.
[0030]
In the illustrated embodiment, it is assumed that the image of the manipulator represented by the broken line indicating the position where the manipulator should be without slipping has gripped the object at the position where the moving body is actually present. It is a computer graphics image showing the state of time, and by showing such an image, the current state can be shown more clearly to the pilot.
[0031]
The computer graphics image as described above is recorded by updating the image that is always taken by the video camera for a predetermined short time, and when the moving object is detected to have slipped, the image immediately before that is detected by the computer. Output to the graphics image processing unit to form a computer graphics image at that position, and easily form and output an image viewed from the video camera at the point where the moving body slipped and moved Can do.
[0032]
【The invention's effect】
Since the present invention is configured as described above, when the manipulator mounted on the moving body is operated from a remote point, the periphery of the manipulator is photographed by a video camera provided on the moving body, and this is installed on the driver's side. When the manipulator is operated by displaying on the screen, the operator can clearly grasp the state of the manipulator and the state of the moving body at a glance by simple means.
[Brief description of the drawings]
FIG. 1 is a diagram showing an example of a monitor screen displayed in a first embodiment of the present invention.
FIG. 2 is a diagram showing an example of a monitor screen displayed in the second embodiment of the present invention.
FIG. 3 is a schematic diagram showing an overall outline of a state in which a remote control device for a mobile manipulator according to the present invention is applied.
[Explanation of symbols]
1 Manipulator video 2 Monitor screen 3 Grid

Claims (3)

The manipulator attached to the moving body taken with equipped with video cameras to the mobile, the remote control device in a mobile manipulator to maneuver the manipulator display this photographic image on the monitor of the remote to detect a slip of the moving body moving A moving manipulator comprising: a detecting unit, wherein a composite image based on a fixed absolute coordinate system representing a state in which no slip occurs is displayed on the captured image in response to a signal from the moving detecting unit . Remote control device. The remote operation device for a mobile manipulator according to claim 1, wherein the synthesized image based on the absolute coordinate system is a grid. The remote manipulator remote device according to claim 1, wherein the image based on the absolute coordinate system is an image showing a synthesized image of a manipulator based on an operation command at a hand position where a slip of the moving body should be present. Operating device.

Images