JP4558682B2 - Manipulator remote control method for mobile robot system - Google Patents

Description

  The present invention relates to a manipulator remote operation method for remotely operating a mobile robot system having a manipulator having a camera and a distance measurement sensor from a control system, and in particular, remotely operating the manipulator so that an obstacle is displayed in a camera image. The present invention relates to a manipulator remote control method in a mobile robot system that measures a three-dimensional shape of the obstacle using a distance measurement sensor.

  A manipulator is a machine having a multi-joint that simulates a human arm. By storing the operation content in advance in a manipulator or a computer connected to the manipulator, a predetermined operation can be performed. Because it can repeat precise work at high speed, it is used in production lines such as factories. In the medical field, there is also a utilization method in which a doctor remotely operates a manipulator to perform an operation or the like.

  There has been proposed a usage method in which a robot is moved to a place where a person cannot approach and a manipulator is remotely controlled to avoid dangers inherent to humans.

  When performing a remote operation, the operator is operating at a position where the manipulator can be seen or while viewing an image or video projected from the camera of the robot. For example, when a sensor attached to the robot detects an object that becomes an obstacle during traveling, the operator looks at the camera image and determines an obstacle. At this time, it is possible to instruct the robot to take an action such as removing or avoiding an obstacle based on the camera image.

  However, the operator may not be able to accurately confirm the obstacle from the camera image. Since the camera image is a two-dimensional display to the last, it is difficult to understand the sense of distance, and it is difficult to grasp the three-dimensional shape. Therefore, a method of estimating the size of an obstacle by estimating the size of another object from the size of the reference object displayed in the camera image or mounting a distance measuring device such as a laser radar sensor on the robot Is used.

  Japanese Patent Application Laid-Open No. 2004-257927 describes an apparatus that can measure the three-dimensional shape of an object by remote control. In this method, a three-dimensional shape measuring device is provided at the tip of the manipulator, and the three-dimensional shape of the measurement object can be measured from the desired position by moving the measuring device to the desired position.

  In Patent Document 2 (Japanese Patent Laid-Open No. 2002-84531), a mobile robot system capable of remote control is realized by attaching a camera to a robot. In this method, the operator can acquire information on a remote location through a camera image.

  In Patent Document 3 (Japanese Patent Application Laid-Open No. 2003-256025), a geometric element such as a trajectory is defined in advance by a parametric modeling method in a form including spatial constraints between geometric elements determined from the requirements of the target work, and is made into a library. In addition, a robot motion teaching technique is described in which various constraint conditions can be set with fewer steps in the actual teaching stage by creating a menu.

JP 2004-257927 A JP 2002-84531 A Japanese Patent Laid-Open No. 2003-256025

  In a conventional robot system, a camera is simply provided to acquire an image, or a camera and a three-dimensional shape measuring device are individually provided in a manipulator of a non-moving robot. For this reason, it is difficult to acquire a camera image after moving the robot body at a remote point and simultaneously grasp the three-dimensional shape of the obstacle. It is also difficult to visually grasp the positional relationship between the robot's own position and the obstacle after grasping the three-dimensional shape.

  In order to solve the above problems, an object of the present invention is to enable a remote robot to make it possible for an operator to visually recognize the three-dimensional shape and size of an obstacle using a distance measuring device installed in a manipulator in a mobile robot system. An object is to provide a manipulator remote control method in a mobile robot system that can perform obstacle removal and avoidance behavior by operation.

  In order to achieve the above-mentioned object, the present invention enables communication between a mobile robot system having a manipulator having a camera and a distance measuring device and a control system having an input device and a display device by communication. Configured to remotely control the manipulator of the mobile robot system from the control system using the communication, and information on a camera image (planar grayscale image or color image) captured by the camera. A first display step of transmitting the mobile robot system to the control system and displaying the camera image on the display device, and at least while viewing the camera image displayed on the display device in the first display step An operation instruction input to the input device is forwarded from the steering system so that an obstacle is projected in the camera image. An operation process that is transmitted to the mobile robot system by communication to remotely operate the manipulator, and at least an obstacle displayed in the camera image displayed on the display device by the remote operation of the manipulator in the operation process And a measuring process for measuring the three-dimensional shape of the distance by the distance measuring device.

  Further, the present invention further transmits three-dimensional shape information about at least an obstacle measured in the measurement process from the mobile robot system to the control system, thereby obtaining a three-dimensional shape (stereoscopic image) about the obstacle. It has the 2nd display process which displays on the said display apparatus, It is characterized by the above-mentioned.

  Further, the present invention provides a camera that displays a three-dimensional shape (stereoscopic image) about the obstacle displayed on the display device in the second display process on the display device in the first display process. An image (planar gray image or color image) is superimposed and displayed.

  Further, the present invention is characterized in that, in the second display step, the three-dimensional shape (stereoscopic image) of the mobile robot body including the manipulator in the mobile robot system is displayed in an overlapping manner.

  That is, according to the present invention, when an object that becomes an obstacle of a mobile robot is found based on the camera image, the manipulator is moved by an operation instruction to change the angle of the camera, and the obstacle is detected by the camera image acquired at a desired angle. An object is confirmed, and the distance to the confirmed obstacle is measured by a distance measuring device. At this time, the measurement range is programmed in the distance measurement device so that an object projected by the camera image can be measured. Furthermore, since each joint axis rotation angle of the manipulator is known by the sensor (encoder), the linear distance L from the distance measuring device to the obstacle, the horizontal direction angle α, and the depth direction angle β can be measured. Then, by using a trigonometric function for the measured distance and angle, the distance in the horizontal direction and the depth direction between the distance measuring device and the object can be obtained. That is, the positional relationship between the three-dimensional distance measuring device and the obstacle is obtained, and the shape of the obstacle can be expressed in three dimensions. Furthermore, since the manipulator can move, the three-dimensional shape can be grasped from a plurality of angles with respect to the obstacle.

  In addition, the present invention can simultaneously grasp the surface pattern of an obstacle and its three-dimensional shape by superimposing the obtained three-dimensional shape (three-dimensional image) and a camera image (planar grayscale image or color image). it can.

  Further, according to the present invention, by adding the three-dimensional shape (stereoscopic image) of the main body of the mobile robot system, the operator can clearly grasp the positional relationship with the obstacle.

  According to the present invention, a manipulator of a mobile robot system that can be remotely operated is operated by a steering system, and a surface image of an obstacle or an object to be investigated and a three-dimensional shape by a distance measuring device are viewed while viewing a camera image at a desired angle. Since it can be grasped, it is possible to safely collect detailed three-dimensional information on the site while avoiding danger to human beings by remotely operating a robot even in a dangerous zone where people cannot approach.

  Embodiments of a mobile robot system and a manipulator remote control method in the mobile robot system according to the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram showing an embodiment of a mobile robot system according to the present invention. The mobile robot system according to the present invention comprises a control system 20 capable of operating a remote robot and a robot body system 1. The The control system 20 is configured to be able to operate the mobile (running) robot body 5 using communication from a remote location. The manipulator 9, the camera 13, the distance measuring device 10, and the like provided in the mobile robot body 5 are Control is possible via the communication devices 27 and 8. The manipulator 9 includes, for example, a shoulder portion 9d, an upper arm 9a, a forearm 9b, a wrist 9c, and the like. The operator 25 controls the mobile robot body 5 from a remote location through the control system 20 and receives information from the mobile robot body 5.

  In the control system 20, the operator 25 moves the mobile robot body 5 using the input device 22 on the basis of the camera image displayed on the image display device 23 from a remote place, and each joint axis constituting the manipulator 9. By giving an angle, the position of the manipulator 9 on which the distance measuring device 10 and the camera 13 are mounted is moved, or an operation instruction regarding the robot body 5 including the specific manipulator 9 is given. The instruction processing device 21 receives the operation information 28 of the operator 25 input by the input device 22 and converts it into communication operation data, and the converted operation data is transmitted from the communication device 27 to the communication device 8 of the robot body system 1. Will be sent to.

  The robot body system 1 includes a communication device 8 that communicates with the maneuvering system side, and operation data transmitted to the robot side, the manipulator 9 including the travel of the mobile robot body 5 and the manipulator 9. For example, an obstacle in the coordinate system (camera coordinate system) of the manipulator 9 based on the distance data measured by the distance measuring device 10 while generating drive control data for operating the mounted distance measuring device 10 and the camera 13. A signal processing device 11 that calculates 30 three-dimensional solid shapes, and an image processing device 12 that acquires camera image data captured from the camera 13 and converts the acquired camera image data into communication camera image data. Configured. The signal processing device 11 also receives partial feedback information from various devices (for example, an encoder) of the mobile robot body 5 and generates the drive control data. Then, the communication camera image data converted by the image processing device 12 and the operation information of the robot body generated by the signal processing device 11 are transmitted to the communication device 27 on the steering system side via the communication device 8, and the instruction processing device. 21 is displayed on the image display device 23 as image information 29 and is transmitted to the operator 25. As a result, by inputting the operation information 28 from the remote place using the input device 22, external information (camera image) of the obstacle 30 that becomes an obstacle as the mobile robot body 5 is displayed on the image display device 23. As can be seen, the angles of the manipulator 9 and the camera 13 can be manipulated.

  That is, the present invention relates to a mobile robot body 5 having a manipulator 9 equipped with a distance measuring device 10 capable of measuring a distance to a target object such as an obstacle 30 from a camera 13 attached to the manipulator 9. The manipulator 9 is moved based on the camera image (planar grayscale image or color image), and the distance to the obstacle 30 displayed in the camera image is measured using the distance measuring device 10 installed in the manipulator 9. This is to acquire at least a three-dimensional solid shape such as the obstacle 30.

  By the way, the distance measuring device 10 measures the three-dimensional surface shape of a target object such as an obstacle located on the front side thereof with its own coordinate system (x, y, z) using a laser beam according to a triangulation method. At the same time, information representing the three-dimensional shape of the measured target object is output and transmitted to the signal processing device 11. That is, the distance measuring device 10 assumes a virtual plane substantially perpendicular to the traveling direction of laser light emitted from a laser light source (not shown) toward the target object, and is orthogonal to each other on the virtual plane. A large number of minute areas divided along the x-axis direction and the y-axis direction are assumed. Then, the distance measuring device 10 sequentially irradiates the plurality of minute areas with laser light, and sequentially detects the distance to the surface of the target object that defines the minute area by reflected light from the target object as the z-axis direction distance. The surface shape of the target object facing the distance measuring device is obtained by obtaining information (data) relating to the x, y, z coordinates representing the divided area positions obtained by dividing the surface of the target object into minute areas. Accordingly, the distance measuring device 10 includes an x-axis direction scanner (deflector) (not shown) that changes the direction of the outgoing laser light in the x-axis direction, and a y-axis that changes the direction of the outgoing laser light in the y-axis direction. A direction scanner (deflector) (not shown) and a distance detector (not shown) that receives a reflected laser beam reflected from the surface of the target object and detects a distance to the surface of the target object. Configured. As a distance detector, an imaging lens that rotates following the optical path of the outgoing laser beam and collects the reflected laser beam reflected on the surface of the target object, and a CCD that receives the collected laser beam, etc. It consists of a line sensor in which a plurality of light receiving elements are arranged in a line, and is configured such that the distance to the surface of the target object can be detected by the light receiving position of the reflected light by the line sensor.

  Next, the processing flow of the entire robot system according to the present invention will be described with reference to the flowchart shown in FIG. First, the operator 25 instructs the mobile robot body 5 of the robot body system 1 to acquire a camera image from the remote control system 20 using the input device 22 (S1). Since the camera 13 continues to acquire the latest image, it is not necessary to give an instruction to acquire an image. When the operator 25 instructs image acquisition using the input device 22, the instruction is transmitted as operation information 28 to the instruction processing device 21. The instruction processing device 21 converts the operation information 28 into communication-type operation data. Usually, in order to reduce the amount of communication information, the instruction processing device 21 performs compression conversion of information. After compressing and converting the data, the data is transmitted to the communication device 27 included in the instruction processing device 21. The communication device 27 of the control system 20 transmits information to the communication device 8 provided in the mobile robot body 5 at a remote point. A typical communication method at this time is a wireless LAN or the like, but is not limited thereto.

  After receiving the data, the communication device 8 of the mobile robot body 5 sends the data to the connected signal processing device 11. After decompressing the compressed data, the signal processing device 11 converts the data into data that allows the mobile robot body 5 to be designated to operate, and transmits the data to the mobile robot body. For example, when an image acquisition instruction is given to the camera 13, an acquisition instruction is issued to the camera 13, and the camera 13 executes image acquisition. The acquired image is transmitted to the image processing apparatus 12, and conversion such as compression is performed. In general, in the case of image data, the file size is larger than the operation information. Further, if the image acquisition frequency is set to increase per time, the communication amount becomes enormous. Therefore, by using the image processing device 12 separately from the signal processing device 11, the load on the signal processing device 11 is reduced. However, this is not the case when the image file size is very small and the acquisition frequency is low.

  The image data acquired by the camera 13 is compressed by the image processing device 12, transmitted to the communication device 8 of the mobile robot body 5, and transferred to the communication device 27 on the steering system side. The instruction processing device 21 decompresses the compressed image data and displays it as image information 29 on the image display device 23 (S2).

  From the above processing flow, the operator 25 can observe the camera image by the image display device 23. At this time, in addition to the camera image, if the camera status information, for example, each numerical value such as pan / tilt / zoom of the camera 13 is transmitted from the mobile robot body 5, the operator can also confirm this information. A procedure in which the operator operates the robot body and each of the devices provided and feeds back the acquired information so that the operator can confirm is the same processing flow. Hereinafter, the same data flow is omitted.

  Next, after displaying the camera image, the distance measuring device 10 is calibrated (S3). FIG. 3 shows a flow from the start of calibration (S31). In the calibration, the distance measurement device 10 performs basic settings when measuring the three-dimensional information of the obstacle 30, for example. The horizontal viewing angle of the camera 13 is known, and the horizontal three-dimensional shape measurement range in the distance measuring device 10 is determined by assigning this viewing angle to the maximum horizontal angle of the distance measuring device 10 ( S32). Next, similarly, the height-depth direction (elevation direction) viewing angle of the camera 13 is assigned to the maximum value of the height direction angle of the distance measuring device 10 (S33). After determining the range of distance measurement, the number of measurement coordinates (small area) is set. Here, m is the number of horizontal coordinates, and n is the number of height direction coordinates, and the number of measurement coordinates is n × m points (S34). Although n and m may depend on the distance measuring device to be used, a more detailed three-dimensional shape can be measured as the number of measurement coordinates increases. The operator specifies the number of coordinates or reads a preset value.

  After the calibration, the operator 25 changes the angles of the manipulator 9 and the camera 13 with the input device 22 while viewing the image displayed on the image display device 23 (S4). The manipulator 9 has a plurality of joint axes, and if an angle is designated with respect to the joint axes, the manipulator 9 moves to that angle. If a camera having a pan / tilt / zoom function is used as the camera 13, the operator can change the angle of the camera 13 from the input device 22. However, when the angle of the camera 13 is changed, it is necessary to perform calibration again. In that case, the angle change of the camera 13 may be added to the measurement range of the distance measuring device 10.

  Thereafter, when the distance measurement device 10 is instructed to measure distance (S5), measurement of a three-dimensional shape is started (S51). The distance measurement flow is shown in FIG. First, the distance measuring device 10 calculates the angle α in the horizontal-depth direction from the distance L to the coordinates of the surface of the obstacle 30 and the angle of each joint axis detected from the encoder of the manipulator 9 within the range of the camera image. Measure (S6). The two values L and α to be measured are obtained simultaneously. Next, the surface coordinates (x, y, z) of the obstacle 30 are calculated based on these values L, α and the height-depth angle β obtained from the encoder of the manipulator 9 (S7).

  The coordinate calculation flow is shown in FIG. In FIG. 5, when the coordinate calculation is started (S71), as shown in FIG. 7, the linear distance L (F5) to the obstacle 30 measured by the distance measuring device 10 and the angle α (from the encoder of the manipulator 9) Using F3), the horizontal coordinate x = −Lcosα is calculated by a trigonometric function (S72). The value of the coordinate x can be obtained from the measurement result of the distance measuring device 10. Next, using the angle β (F4) obtained from the encoder of the manipulator 9, the height coordinate y = −Lcos (90 ° −α) cosβ is obtained (S73). Further, the coordinate z = Lcos (90 ° −α) sinβ in the depth direction is obtained from the straight line distance L and the angle β (S74). β is known from the angle of each joint axis of the manipulator 9. For example, the signal processing device 11 stores (x, y, z), which are the coordinates of the surface of the obstacle 30, as a result of these calculations in a storage device (not shown) provided in the robot body 5 (S75). From the above, the robot body 5 of the robot body system 1 can obtain the coordinates of the obstacle 30 and store them in the storage device (not shown). An image at this time is shown in FIG. Reference numeral 9 denotes a manipulator having a distance measuring device 10 thereon. An angle α formed by the horizontal axis and L on the xL plane is defined as F3, and an angle formed by L and the height direction axis on the yz plane. When β is F4, the linear distance is F5, the obstacle is 30, the obstacle surface coordinates (x, y, z) are F7, the camera is 13, and the distance L to the obstacle 30 is projected onto the yz plane. Is indicated by A (F9).

  As shown in FIG. 4, after step S <b> 7, for example, the signal processing device 11 of the robot body 5 sequentially calculates coordinates to obtain the shape of the entire surface of the obstacle 30. For example, an x-axis direction scanner (deflector) is used to move the distance measurement target coordinates with respect to the obstacle 30 (S76). Specifically, for example, the irradiation angle of the measurement laser light emitted from the distance measuring device 10 is changed using an x-axis direction scanner (deflector). After measuring the distance to the end in the horizontal direction, for example, the y-axis direction scanner (deflector) is used to move the distance measurement target coordinate in the height direction with respect to the obstacle 30 (S77). This can also be realized by moving the angle of the manipulator 9 in the height direction. After that, the same measurement is repeated until the end in the height direction (S78).

  As described above, the set of the surface coordinates (x, y, z) of the obstacle 30 obtained by, for example, the signal processing device 11 of the robot main body 5 is processed from the robot main body system 1 through the communication devices 8 and 27 to the instruction processing of the control system 20. By transmitting to the device 21, the three-dimensional shape of the obstacle 30 can be displayed on the image display device 23 (S8). For the shape display, the visual effect can be enhanced by using a surface model that is a modeling technique for capturing the obtained three-dimensional coordinate points as a surface.

  Further, the operability of the operator is improved by displaying the relative relationship between the obstacle and the mobile robot body in three dimensions on the image display device 23 (S9). For the three-dimensional image 82 of the mobile robot body 5 as shown in FIG. 8C, the shape of the mobile robot body 5 is created in advance using modeling software or the like and provided in the robot body system 1. The data is stored in a storage device (not shown) or a storage device (not shown) provided in the control system 20. Further, the relative position between the coordinates of the manipulator 9 and the obstacle 30 as shown in FIG. 8C is determined by the relative positional relationship between the manipulator 9 and the distance measuring device 10, so the distance measuring device 10 is used. It is possible to ascertain based on the distance L measured in this manner, the angle α in the horizontal-depth direction detected from the encoder of the manipulator 9, and the angle β in the height-depth direction. In addition, since each joint axis of the manipulator 9 is known by a sensor (encoder), the coordinates including the joint axis of the mobile robot main body 5 and the joint axis of the manipulator 9 are included in the signal processing device 11 or the instruction It can be calculated by the CPU in the processing device 21.

  Next, the instruction processing device 21 of the control system 20 at a remote location determines whether or not to superimpose the camera image (S10), and in the case of superimposing, the image is captured by the camera 13 shown in FIG. On the camera image, the robot body system 1 shown in FIG. 8B transmits to the control system 20 and the image display device 23 displays the three-dimensional shape of the obstacle 30, the mobile robot body 5, etc., as shown in FIG. ) And displayed on the image display device 23 (S11). Thereafter, the operator 25 determines whether or not to acquire the outline information of the obstacle from another viewpoint while viewing the superimposed image displayed on the image display device 23 (S12), and the mobile robot body 5 changes the viewpoint. When the obstacle outline information is obtained by changing (obstructing the obstacle shape measurement), the operator 25 uses the input device 22 to instruct the mobile robot body 5 to change the viewpoint, and the mobile robot. The main body 5 changes the angle of the manipulator 9 in accordance with the instruction, measures the three-dimensional shape of the obstacle 30, acquires the outer shape, and stores it in a storage device (not shown). Then, by transmitting from the robot body system 1 to the control system 20 using communication, the instruction processing device 21 of the control system 20 sends the obstacle information (three-dimensional shape) of the obstacle from another viewpoint to the image display device 23. Can be displayed.

  As described above, in the image display device 23 of the control system 20 in a remote place, the image acquired from the camera 13 of the mobile robot body 5 is displayed on the three-dimensional obstacle shape image so as to obstruct the obstacle. You can grasp the surface and surrounding information in detail. That is, the superimposed image shown in FIG. 8C is obtained by superimposing the camera image layer shown in FIG. 8A on the obstacle layer shown in FIG. Can be understood in detail.

  Next, the flow of creating a superimposed image will be described with reference to FIG. That is, after superimposition is started (S111), in order to associate the camera image with the three-dimensional shape image, for example, the signal processing device 11 that performs signal processing of the distance measurement device 10 and the image processing device 12 that performs processing of the camera image captured by the camera 13. N × m measurement coordinates calculated by calibration in the distance measuring device 10 are equally allocated to the size of the camera image captured by the camera 13 (S112). Thereafter, for example, the signal processing device 11 plots the three-dimensional coordinates measured by, for example, the distance measuring device 10 in a three-dimensional space having the same size as the camera image processed by the image processing device 12 (S113). Next, the camera image is superimposed on the signal processing device 11 and the image processing device 12 as a texture of a three-dimensional plot image and superimposed (S114). By transmitting the superimposed image superimposed in this way from the mobile robot body 5 to the instruction processing device 21 of the control system 20 using communication, the image display device 23 displays the superimposed image shown in FIG. Will be able to. Thereby, the operator 25 can grasp the relative positional relationship between the mobile robot body and the obstacle, and can give an instruction for manipulator operation using the input device 22 while viewing the whole image.

  As described above, according to the present embodiment, the manipulator of the mobile robot that can be remotely operated is operated while viewing the camera image at a remote location, so that the surface image of the obstacle or the object to be investigated can be three-dimensionally displayed. Because it can grasp the shape, it is possible to safely collect detailed three-dimensional information of the site safely while avoiding danger to humans by operating the robot remotely even in a dangerous zone where people cannot approach Become.

1 is a schematic configuration diagram showing an embodiment of a remotely operable robot system according to the present invention. It is a figure which shows one Example of the flow at the time of the three-dimensional shape measurement which concerns on this invention. It is a figure which shows one Example of the flow of the initial setting at the time of the three-dimensional shape measurement which concerns on this invention. It is a figure which shows one Example of the flow at the time of the distance measurement which concerns on this invention. It is a figure which shows one Example of the coordinate calculation flow at the time of the three-dimensional shape measurement which concerns on this invention. It is a figure which shows one Example of the flow of the image superimposition which concerns on this invention. It is a figure which shows the positional relationship of the manipulator at the time of the three-dimensional shape measurement which concerns on this invention, and an obstruction. It is a figure which shows the image of the image superimposition which concerns on this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Robot main body system, 5 ... Mobile robot main body, 8 ... Communication apparatus, 9 ... Manipulator, 10 ... Distance measuring device, 11 ... Signal processing apparatus, 12 ... Image processing apparatus, 13 ... Camera, 20 ... Steering system, 21 ... instruction processing device, 22 ... input device, 23 ... image display device, 25 ... operator, 27 ... communication device, 30 ... obstacle.

Claims (4)

A mobile robot system having a camera and a manipulator having a distance measuring device and a maneuvering system having an input device and a display device are configured to be able to transmit and receive by communication, and the manipulator of the mobile robot system Using the communication to remotely control the steering system,
A first display process of transmitting information of a camera image captured by the camera from the mobile robot system to the control system and displaying the camera image on the display device;
The horizontal measuring range and height of the obstacle of the distance measuring device set based on the camera image of the obstacle displayed on the display device in the first display process and inputted to the input device An operation process of remotely operating the manipulator by transmitting an operation instruction including a measurement range in the depth direction and the number of minute areas to be measured from the steering system to the mobile robot system by the communication;
The three-dimensional shape of at least the obstacle displayed in the camera image displayed on the display device by the remote operation of the manipulator during the operation process is horizontal from the angle of each joint axis detected from the encoder of the manipulator. A step of calculating the obstacle surface coordinates corresponding to the entire minute area based on the angle in the depth direction and the height-angle in the depth direction, and measuring the distance with the distance measuring device for each of the coordinates. A manipulator remote control method in a mobile robot system characterized by the above.   Further, a second information for transmitting at least three-dimensional shape information about the obstacle measured in the measurement process from the mobile robot system to the control system and displaying the three-dimensional shape about the obstacle on the display device. 2. The manipulator remote control method for a mobile robot system according to claim 1, further comprising a display process.   In the second display process, the camera image displayed on the display device in the first display process is displayed in a superimposed manner on the three-dimensional shape of the obstacle displayed on the display device. The manipulator remote control method in the mobile robot system according to claim 2.   4. The manipulator in the mobile robot system according to claim 3, wherein in the second display step, the three-dimensional shape of the mobile robot body including the manipulator in the mobile robot system is further displayed in an overlapping manner. Remote operation method.

Images