CN108422435A - Remote monitoring and control system based on augmented reality - Google Patents

Abstract

The invention relates to a remote monitoring and control system based on augmented reality, including an interactive simulation control mode and a monitoring mode. In the interactive simulation mode, a server receives depth images taken by an RGBD camera on a mobile robot, and the depth images taken at different angles The image synthesizes the 3D point cloud model of the scene; then combines the 3D point cloud model of the scene, operates the manipulator simulation model, plans the path, uses the path to drive the movement of the manipulator simulation model, and finally superimposes the dynamic manipulator simulation model on the color image captured by the RGBD camera In the image, the simulation display and path planning of the combination of virtual and real are realized; in the monitoring mode, the monitoring data is collected through the remote data acquisition system, the monitoring image of the monitoring object is obtained through the mobile robot inspection, the monitoring data is obtained through the server, and the monitoring data is collected Superimposed display with monitoring data, the display interface is more intuitive.

Description

A remote monitoring and control system based on augmented reality

technical field

The invention relates to a remote monitoring and control system based on augmented reality, belonging to the fields of intelligent manufacturing and computer measurement and control.

Background technique

At present, remote monitoring and control have been widely used in production. However, in terms of monitoring, the current monitoring system displays the monitoring video and monitoring data separately. The monitoring personnel need to link the video content and monitoring data by themselves, and the interface is not intuitive enough. In addition, in terms of control, the controller can only estimate the three-dimensional information of the remote environment through the color image, plan the path of the manipulator, and control the remote manipulator to complete the operation in this way, which will result in inaccurate robot control.

The invention patent "CN106863303A" whose publication number is CN106863303A, its specific technical scheme is: the angular displacement of each joint is collected by the encoder installed at each joint of the manipulator, and the collected data is sent to the computer for processing, and the computer plans according to the angular displacement data of each joint Out of the path of the manipulator, and control the manipulator to complete the welding of the weld along this path. This technical solution is an on-site path planning method for equipment, which cannot meet the current needs of remote path planning for enterprises.

Contents of the invention

In order to solve the above technical problems, the present invention provides a remote monitoring and control system based on augmented reality, which realizes the fusion display of video monitoring and data monitoring of remote inspection applications, and generates local path planning and simulation of robot remote operation on the server side. .

Technical scheme of the present invention is as follows:

A remote monitoring and control system based on augmented reality, including an interactive simulation control mode, which uses a server and a mobile robot located in the remote device and environment to complete the local path planning and simulation of the remote operation of the manipulator in a virtual and real way ,specifically:

Described mobile robot comprises robot body, and the manipulator and RGBD camera that are located on robot body;

The server includes a remote environment reconstruction module, a manipulator operation simulation module, and an augmented reality display module; the remote environment reconstruction module receives the depth images captured by the RGBD camera, and synthesizes the depth images captured at different angles into three-dimensional points of the scene Cloud model; the manipulator operation simulation module establishes a manipulator simulation model, then operates the manipulator simulation model in conjunction with the scene three-dimensional point cloud model, plans a path, and finally utilizes the path to drive the movement of the manipulator simulation model; the augmented reality display module will The dynamic manipulator simulation model is superimposed on the color image captured by the RGBD camera to realize the simulation display combining virtual and real.

More preferably, the collision detection is also completed in the interactive simulation control mode, specifically: the manipulator operation simulation module uses the path to drive the movement of the manipulator simulation model in the scene three-dimensional point cloud model, and checks whether the manipulator simulation model is consistent with the scene three-dimensional point cloud model. The point cloud model interferes to realize virtual-real collision detection; if there is interference, the path is optimized, and then the optimized path is sent to the mobile robot; if there is no interference, the path is sent to the mobile robot, and the mobile robot is controlled according to the path The movement of the manipulator to complete the operation task.

More preferably, the mobile robot also includes an image acquisition controller, a robot body controller, and a manipulator controller; the image acquisition device controls the movement of the camera pan/tilt, and collects color images and depth images on the RGBD camera, The RGBD camera is fixed on the camera platform; the robot body controller controls the movement of the robot body, and feeds back the orientation and attitude of the robot body; the manipulator controller controls the movement of the manipulator;

The server is also provided with a pan-tilt teleoperation module, a robot body teleoperation module and a manipulator teleoperation module; the pan-tilt teleoperation module sends an instruction to the image acquisition controller to control the camera pan-tilt to move according to the instruction, thereby changing the RGBD The shooting angle of the camera; the robot body teleoperation module sends instructions to the robot body controller to control the robot body to move according to the instructions; the manipulator teleoperation module sends instructions to the manipulator controller to control the manipulator to move according to the instructions;

When the remote environment reconstruction module synthesizes the three-dimensional point cloud model of the scene, firstly, the robot body teleoperation module and the manipulator teleoperation module send control instructions to the corresponding robot body controller and the manipulator controller respectively, so that the robot body and the manipulator The manipulator remains fixed, and then sends instructions to the image acquisition controller through the pan/tilt remote operation module to control the camera pan/tilt to rotate at multiple angles according to the instructions, and shoot depth images at various angles, thereby obtaining depth images from different angles and synthesizing the scene 3D point cloud model.

More preferably, the manipulator simulation model established by the manipulator operation simulation module includes a three-dimensional virtual model of the manipulator and an inverse kinematics model, plans the end point trajectory and attitude of the manipulator in the interactive simulation environment, and obtains the end point trajectory and attitude sequence to The trajectory of the end point and the attitude sequence are used as the input of the inverse kinematics model to obtain the angle sequence of each joint of the manipulator, and then read the angle sequence of each joint of the manipulator in order to drive the movement of each joint of the three-dimensional virtual model of the manipulator.

More preferably, the augmented reality display module superimposes the dynamic three-dimensional virtual model of the manipulator on the color image taken by the RGBD camera according to the current position and attitude of the robot body, and the orientation information of the camera pan/tilt, and then displays it to realize the visual fusion of virtual and real .

More preferably, the current position and attitude of the robot body are sent to the robot body controller through the robot body remote operation module to obtain the information; Send commands to obtain the orientation information of the camera gimbal;

More preferably, the remote monitoring and control system also includes a monitoring mode, which uses the remote data acquisition system located in the remote equipment and the environment, the server and the mobile robot to complete the monitoring data and monitoring images in a virtual and real way. The fusion shows, specifically:

The remote data collection system collects the monitoring data of each monitoring object, and then sends it to the server; a label is set at each monitoring object;

The mobile robot receives control instructions from the server, controls the movement of the robot body and the camera pan/tilt, moves the robot body to the monitoring object, and controls the rotation of the camera pan/tilt, so that the RGBD camera captures a monitoring image containing tag information, and sent to the server;

The server also includes a device identification module and a storage and fault diagnosis module;

The device identification module analyzes the monitoring image taken by the RGBD camera, segment and identify the tag information on the monitoring image, and sends a query request containing the tag information;

The storage and fault diagnosis module receives and saves the monitoring data sent by the remote data acquisition system, and diagnoses whether there is a fault in the monitoring object, and if there is a fault, sends an alarm message; when the storage and fault diagnosis module receives the monitoring data from the equipment identification module When querying a request, query the monitoring data corresponding to the tag information according to the tag information, and then send it to the augmented reality display module;

The augmented reality display module superimposes the monitoring data on the monitoring image captured by the RGBD camera to display in the vicinity of the corresponding label, so as to realize the superimposed display of the monitoring data and the monitoring data, and the display interface is more intuitive.

More preferably, the remote data acquisition system includes an industrial computer and sensors, each of the sensors is connected to the industrial computer, each monitoring object is provided with the sensor, and each sensor is correspondingly provided with a label;

More preferably, the storage and fault diagnosis module receives and saves the monitoring data sent by the remote data acquisition system according to the corresponding relationship between the label, the monitoring object and the monitoring data, and according to the preset alarm threshold, combines the monitoring data with the Compare the corresponding alarm thresholds to diagnose whether there is a fault in the monitoring object, and if there is a fault, send an alarm message; when the storage and fault diagnosis module receives a query request from the device identification module, it queries the monitoring corresponding to the tag information according to the tag information The data is then sent to the augmented reality display module.

More preferably, the monitoring images include pictures and videos.

The present invention has following beneficial effects:

1. The present invention is a remote monitoring and control system based on augmented reality. The depth image of the environment is obtained by a mobile robot located in the remote device and the environment, and then the three-dimensional point cloud model of the remote device and the environment is calculated and synthesized on the server side. Furthermore, on the server side, the local path planning and simulation of the remote operation of the mobile robot is realized through the virtual simulated manipulator model. It is a remote control system combining virtual and real, which improves the control accuracy of the mobile robot;

2. A remote monitoring and control system based on augmented reality of the present invention collects monitoring data through a remote data acquisition system, then obtains monitoring images of monitoring objects through a mobile robot, and identifies label information on the server side to obtain the current status of the monitoring objects. State information, and then superimpose the monitoring information on the monitoring image, realizing the fusion display of data monitoring and on-site video monitoring or data monitoring and image monitoring for remote inspection applications.

Description of drawings

Fig. 1 is a system block diagram of the present invention;

Fig. 2 is the flowchart of interactive simulation control mode of the present invention;

Fig. 3 is a flow chart of the monitoring mode of the present invention.

The reference signs in the figure represent:

1. Robot body; 2. Manipulator; 3. RGBD camera; 4. Camera pan/tilt; 5. Robot body controller; 6. Manipulator controller; 7. Image acquisition controller; 10. Robot body teleoperation module; 20. Manipulator remote operation module; 30. PTZ remote operation module; 40. Remote environment reconstruction module; 50. Manipulator operation simulation module; 60. Augmented reality display module; 70. Equipment identification module; 80. Storage and fault diagnosis module; 100 , industrial computer; 200, sensor; 300, label.

Detailed ways

The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

Embodiment one:

Please refer to Figure 1 and Figure 2, a remote monitoring and control system based on augmented reality, including an interactive simulation control mode, which uses a server and a mobile robot located in the remote device and environment to complete the manipulator 2 in a virtual and real way. Local path planning and simulation for remote operations, specifically:

The mobile robot comprises a robot body 1, and a manipulator 2 and an RGBD camera 3 arranged on the robot body 1;

The server includes a remote environment reconstruction module 40, a manipulator operation simulation module 50, and an augmented reality display module 60; the remote environment reconstruction module 40 receives the depth images taken by the RGBD camera 3, and the depth images taken from different angles Image synthesis scene three-dimensional point cloud model, the manipulator operation simulation module 50 establishes the manipulator simulation model, then combines the scene three-dimensional point cloud model, operates the manipulator simulation model, plans the path, and finally utilizes the path to drive the movement of the manipulator simulation model; The augmented reality display module 60 superimposes the dynamic manipulator simulation model on the color image captured by the RGBD camera 3 to realize the simulation display combining virtual and real. The manipulator 2 includes a manipulator with multiple degrees of freedom, such as a manipulator with six degrees of freedom, a manipulator with three degrees of freedom or a manipulator with two degrees of freedom. The more degrees of freedom, the greater the flexibility of the manipulator.

Collision detection is also completed under the interactive simulation control mode, specifically: the manipulator operation simulation module 50 uses the path to drive the movement of the manipulator simulation model in the three-dimensional point cloud model of the scene, and checks whether the manipulator simulation model is consistent with the scene The three-dimensional point cloud model interferes to realize virtual-real collision detection; if there is interference, the path is optimized, and then the optimized path is sent to the mobile robot, thereby controlling the movement of the manipulator 2 on the mobile robot and completing the operation task of the manipulator 2; The optimized path can also be simulated and displayed through the augmented reality display module 60 .

The mobile robot also includes an image acquisition controller 7, a robot body controller 5 and a manipulator controller 6; the image acquisition device controls the movement of the camera platform 4, and collects color images and depth images on the RGBD camera 3 , the RGBD camera 3 is fixed on the camera platform 4; the robot body controller 5 controls the movement of the robot body 1, and feeds back the orientation and attitude of the robot body 1; the manipulator controller 6 controls the motion of the manipulator 2;

The server is also provided with a pan-tilt teleoperation module 30, a robot body teleoperation module 10 and a manipulator teleoperation module 20; the pan-tilt teleoperation module 30 sends instructions to the image acquisition controller 7 to control the camera pan-tilt 4 according to command movement, thereby changing the shooting angle of the RGBD camera 3; the robot body teleoperation module 10 sends instructions to the robot body controller 5, and controls the robot body 1 to move according to the instructions; the manipulator teleoperation module 20 sends instructions to the manipulator controller 6 command, to control the manipulator 2 to move according to the command;

When the remote environment reconstruction module 40 synthesizes the three-dimensional point cloud model of the scene, it first sends control instructions to the corresponding robot body controller 5 and manipulator controller 6 through the robot body teleoperation module 10 and the manipulator teleoperation module 20 respectively. , keep the robot body 1 and the manipulator 2 fixed, then send instructions to the image acquisition controller 7 through the pan/tilt remote operation module 30, control the camera pan/tilt 4 to rotate at multiple angles according to the instructions, and take depth images at various angles, so that Obtain depth images from different angles and synthesize a 3D point cloud model of the scene.

The manipulator simulation model established by the manipulator operation simulation module 50 includes a three-dimensional virtual model of the manipulator and an inverse kinematics model. In the interactive simulation environment (that is, in combination with the three-dimensional point cloud model of the scene), the trajectory and posture of the end point of the manipulator 2 are planned, and the end point is obtained. The point trajectory and attitude sequence, using the terminal point trajectory and attitude sequence as the input of the inverse kinematics model, obtain the angle sequence of each joint of the manipulator 2, read the angle sequence of each joint of the manipulator 2 in order, and drive the three-dimensional virtual model of the manipulator movement of each joint.

The augmented reality display module 60 superimposes the dynamic three-dimensional virtual model of the manipulator on the color image taken by the RGBD camera 3 according to the current position and attitude of the robot body 1 and the orientation information of the camera pan/tilt 4, and then displays it to realize the visual fusion of virtual and real . The current position and attitude of the robot body 1 is sent to the robot body controller 5 through the robot body teleoperation module 10 to obtain the information. The orientation information of the camera pan/tilt 4 is sent to the image acquisition controller 7 through the pan/tilt remote operation module 30 to obtain the orientation information of the camera pan/tilt 4 . The augmented reality display module 60 obtains the camera platform 4 in the world coordinate system according to the current position and posture of the robot body 1 and the orientation information of the camera platform 4 (the world coordinate system is the absolute coordinate system of the system, and the user coordinate system is not established). The coordinates of all points on the previous screen are based on the origin of the coordinate system to determine their respective positions), so as to realize augmented reality registration, and then superimpose the 3D virtual model of the manipulator on the RGBD camera 3 according to the coordinate information of the 3D virtual model of the manipulator. in the color image.

Specifically, collision detection is also completed in the interactive simulation control mode, and the manipulator operation simulation module 50 uses the path to drive the three-dimensional virtual model of the manipulator to move in the three-dimensional point cloud model of the scene, and checks whether the three-dimensional virtual model of the manipulator is consistent with The 3D point cloud model of the scene interferes to realize virtual-real collision detection; if there is interference, the path is optimized, and then the optimized path is sent to the mobile robot, and the manipulator controller 6 of the mobile robot receives and controls the movement of the manipulator 2 on the mobile robot , to complete the operation tasks of the mobile robot.

The operation process of the interactive simulation control mode is as follows:

Step 1, manipulator operation simulation module 50 establishes the manipulator simulation model of manipulator 2, comprises manipulator three-dimensional virtual model and inverse kinematics model;

Step 2, keeping the robot body 1 and the manipulator 2 fixed, the remote environment reconstruction module 40 receives the depth images taken by the RGBD camera 3, and synthesizes the depth images taken at different angles into a three-dimensional point cloud model of the scene;

Step 3, the end point trajectory and attitude of the three-dimensional virtual model of the manipulator interactively planned in the interactive simulation environment, and the end point trajectory and attitude sequence are obtained;

Step 4, using the end point trajectory and the attitude sequence as the input of the inverse kinematics model of the manipulator 2, to obtain the angle sequence of each joint of the manipulator 2;

Step 5. Read the angle sequences of the joints of the manipulator 2 in order, and drive the joints of the 3D virtual model of the manipulator to move; check whether the 3D virtual model of the manipulator interferes with the 3D point cloud model of the scene during the movement, and realize virtual-real collision detection. If there is a collision , then optimize the path, and then send the optimized path to the manipulator controller 6 of the mobile robot, thereby controlling the manipulator 2 to complete the operation task;

At the same time, the dynamic three-dimensional virtual model of the manipulator can also be superimposed on the color image according to the current position and attitude of the robot body 1 and the orientation information of the camera pan/tilt 4, and the virtual and real simulation display can be realized through the augmented reality display module 60.

The remote monitoring and control system based on augmented reality in the present invention can calculate and synthesize the three-dimensional point cloud model of the remote equipment and the environment on the server side, and then realize the local path planning of the remote operation of the mobile robot through the virtual simulated manipulator model on the server side And simulation, is a combination of virtual and real remote control system, which improves the control accuracy of mobile robots.

Embodiment two,

Please refer to FIG. 1 and FIG. 3 , the difference between this embodiment and the first embodiment is that the second embodiment can execute the monitoring mode while executing the interactive simulation control mode, and the interactive simulation control mode will not be repeated here.

The monitoring mode uses the remote data acquisition system located in the remote device and the environment, the server and the mobile robot, and completes the fusion display of monitoring data and monitoring images in a combination of virtual and real. The monitoring images can be pictures and/or or video. specifically:

The remote data acquisition system includes an industrial computer 100 and a sensor 200, each of the sensors 200 is connected to the industrial computer 100, each monitoring object is provided with the sensor 200, and each sensor 200 is correspondingly provided with a tag 300; The label 300 includes labels 300 in asymmetric forms such as two-dimensional codes and barcodes;

The mobile robot receives control instructions from the server, controls the robot body 1 and the RGBD camera 3, moves the robot body 1 to the monitoring object, and makes the RGBD camera 3 capture a monitoring image containing label information (the monitoring image is a color image) and send it to the server; another embodiment is that the RGBD camera 3 is fixed through the camera pan/tilt 4, and the rotation of the camera pan/tilt 4 is controlled by the control command of the server, so that the RGBD camera 3 captures images containing tag information. Monitor images. The mobile robot can also receive control instructions from the server to control the motion of the manipulator 2; specifically, the robot body teleoperation module 10 and the cloud platform teleoperation module 30 of the server send the robot body controller 5 and the image acquisition module of the mobile robot respectively. The controller 7 sends a control command, the robot body controller 5 controls the robot body 1 to move to the monitoring object, and the image acquisition controller 7 controls the rotation of the camera pan/tilt 4, so that the RGBD camera 3 captures a monitoring image containing label information, realizing Remotely control the mobile robot to obtain monitoring images containing label information and the current operating status of the monitored object; the manipulator teleoperation module 20 of the server can also send control instructions to the manipulator controller 6 of the mobile robot, which is controlled by the manipulator controller 6 The movement of the manipulator 2 realizes the remote control of the manipulator 2;

The server also includes a device identification module 70 and a storage and fault diagnosis module 80;

The device identification module 70 analyzes the monitoring image taken by the RGBD camera 3, segment and identify the tag information on the monitoring image, and sends a query request including the tag information;

The storage and fault diagnosis module 80 receives and saves the monitoring data sent by the remote data acquisition system according to the corresponding relationship between the tag 300, the monitoring object and the monitoring data, and compares the monitoring data with the corresponding The alarm threshold is compared to diagnose whether there is a fault in the monitoring object, and if there is a fault, an alarm message is sent; when the storage and fault diagnosis module 80 receives a query request from the device identification module 70, it queries the monitoring device corresponding to the tag information according to the tag information. data, and then sent to the augmented reality display module 60;

The augmented reality display module 60 superimposes the monitoring data on the monitoring image captured by the RGBD camera 3 and displays the vicinity of the label 300, so as to realize the superimposed display of the monitoring data and the monitoring image (that is, the color image of the monitoring object), and the display interface is more intuitive .

The operation process of the interactive simulation monitoring mode is as follows:

Step 10, the operator operates the computer input device on the server side, and sends the robot controller, the robot body controller 5, and the image acquisition controller 7 respectively through the manipulator teleoperation module 20, the robot body teleoperation module 10, and the pan/tilt teleoperation module 30. Send control instructions to control the movement of the manipulator 2, the robot body 1 and the camera pan/tilt 4 located in the remote device and the environment;

Step 20, the device identification module receives the monitoring image taken by the RGBD camera 3, and identifies the tag information on the monitoring image by segmentation;

Step 30, according to the identified tag information, query the current information of the monitoring object corresponding to the tag 300 from the storage and fault diagnosis module 80;

Step 40: Send the queried information to the augmented reality display module 60, and superimpose it on the monitoring image captured by the RGBD camera 3 for display in the vicinity of the corresponding tag 300, realizing superimposed display of the monitoring information and the monitoring image.

The remote monitoring and control system based on augmented reality of the present invention can control the remote mobile robot locally on the server, obtain the monitoring image of the monitoring object, identify the label information on the server side, obtain the current state of the monitoring object, and then send the monitoring information to Superimposed on the monitoring image, the fusion display of data monitoring and on-site video monitoring or data monitoring and image monitoring for remote inspection applications is realized.

The above is only an embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the description of the present invention and the contents of the accompanying drawings, or directly or indirectly used in other related technologies fields, all of which are equally included in the scope of patent protection of the present invention.

Claims (10)

1. a kind of long-range monitoring and control system based on augmented reality, it is characterised in that：It, should including interactive simulation control model Pattern completes manipulator by server and the mobile robot in remote equipment and environment in such a way that actual situation combines (2) the local path planning and simulation of far-end operation, specifically：

The mobile robot includes robot body (1), and manipulator (2) and RGBD on robot body (1) Camera (3)；

The server includes that distal environment reconstruction module (40), manipulator operating simulation module (50) and augmented reality are aobvious Show module (60)；The distal environment rebuilds the depth image that module (40) receives RGBD cameras (3) shooting, and will be different The range image integration scene three-dimensional point cloud model of angle shot；It is imitative that the manipulator operating simulation module (50) establishes manipulator True mode, then in conjunction with scene three-dimensional point cloud model, manipulator's simulation model, planning path finally utilizes the path The manipulator simulation model is driven to move；Dynamic manipulator simulation model is superimposed by the augmented reality display module (60) In the coloured image that RGBD cameras (3) are shot, simulative display and path planning that actual situation combines are realized.

2. a kind of long-range monitoring and control system based on augmented reality according to claim 1, it is characterised in that：It is described Collision detection is also completed under interactive simulation control model, specifically：The manipulator operating simulation module (50) is in scene three-dimensional It is moved using the path drives manipulator simulation model in point cloud model, checks whether manipulator simulation model is three-dimensional with scene Point cloud model interferes, and realizes actual situation collision detection；If there is interference, then path optimizing is sent out the path after optimization Mobile robot is given, if there is no interference, the path is sent to mobile robot, mobile robot is controlled according to path Operation task is completed in the movement of manipulator (2) processed.

3. a kind of long-range monitoring and control system based on augmented reality according to claim 1, it is characterised in that：It is described Further include image capture controller (7), robot body controller (5) and Manipulator Controller (6) in mobile robot；Institute It states image acquisition device control camera head (4) to move, and acquires the coloured image and depth image on the RGBD cameras (3), The RGBD cameras (3) are fixed on camera head (4)；Robot body controller (5) the control robot body (1) is transported It is dynamic, and feed back the orientation and posture of robot body (1)；The movement of Manipulator Controller (6) control machinery hand (2)；

It is distant that holder remote operating module (30), robot body remote operating module (10) and manipulator are additionally provided on the server Operation module (20)；The holder remote operating module (30) sends to image capture controller (7) and instructs, and controls camera head (4) it is moved according to instruction, to change the shooting angle of RGBD cameras (3)；The robot body remote operating module (10) to Robot body controller (5) sends instruction, and control robot body (1) is moved according to instruction；The manipulator remote operating mould Block (20) sends to Manipulator Controller (6) and instructs, and control machinery hand (2) is moved according to instruction；

The distal environment rebuilds module (40) when synthesizing scene three-dimensional point cloud model, passes through robot body remote operating first Module (10) and manipulator remote operating module (20) are respectively to corresponding robot body controller (5) and manipulator control Device (6) sends control instruction, so that robot body (1) and manipulator (2) is remained stationary, then passes through holder remote operating mould Block (30) sends to image capture controller (7) and instructs, and controls camera head (4) according to instruction multi-angle rotation, and at each angle Degree shooting depth image, to obtain the depth image of different angle, and synthesizes scene three-dimensional point cloud model.

4. a kind of long-range monitoring and control system based on augmented reality according to claim 3, it is characterised in that：It is described The manipulator simulation model that manipulator operating simulation module (50) is established, including manipulator three dimensional virtual models and inverse kinematics mould Type plans the end locus of points and posture of manipulator (2) in interactive simulation environment, obtains the end locus of points and posture sequence, Using the end locus of points and posture sequence as the input of the inverse kinematics model, the angle sequence in manipulator (2) each joint is sought Then row read the angle sequence in manipulator (2) each joint, each joint motions of driving manipulator three dimensional virtual models in order.

5. a kind of long-range monitoring and control system based on augmented reality according to claim 3, it is characterised in that：It is described Augmented reality display module (60) will according to the current position of robot body (1), posture and camera head (4) azimuth information Dynamic manipulator three dimensional virtual models are superimposed upon in the coloured image of RGBD cameras (3) shooting, are then shown, are realized empty Real vision fusion.

6. a kind of long-range monitoring and control system based on augmented reality according to claim 5, it is characterised in that：The machine The current position of device human body (1), posture send control instruction by robot body remote operating module (10) and give robot sheet Body controller (5), obtains the information；Camera head (4) azimuth information is adopted by holder remote operating module (30) to image Collect controller (7) and send instruction, obtains the azimuth information of camera head (4).

7. a kind of long-range monitoring and control system based on augmented reality according to claim 1, it is characterised in that：Further include Monitoring pattern passes through remote data acquisition system in remote equipment and environment and the server and mobile machine People, and complete the fusion of monitoring data and monitoring image in such a way that actual situation combines and show, specifically：

The remote data acquisition system acquires the monitoring data of each monitoring object, is then forwarded to server；Each monitoring object Place is respectively provided with a label (300)；

When the mobile robot inspection, the control instruction from server, control robot body (1) and RGBD cameras are received (3), so that robot body (1) is moved at monitoring object, RGBD cameras (3) is made to take the monitoring figure comprising label information Picture, and it is sent to server；

The server further includes equipment identification module (70) and storage and fault diagnosis module (80)；

The equipment identification module (70) analyzes the monitoring image of RGBD cameras (3) shooting, and segmentation identifies the monitoring image On label information, and send and include the inquiry request of label information；

The storage and fault diagnosis module (80) receive and preserve the monitoring data that the remote data acquisition system is sent, and If diagnostic monitoring object sends warning message with the presence or absence of failure there are failure；When storage and fault diagnosis module (80) connect When receiving the inquiry request from equipment identification module (70), the corresponding monitoring data of the label information are inquired according to label information, It is then forwarded to the augmented reality display module (60)；

Monitoring data are superimposed upon corresponding in the monitoring image of RGBD cameras (3) shooting mark by the augmented reality display module (60) Region is shown near label (300), realizes that the Overlapping display of monitoring data and monitoring data, display interface are more intuitive.

8. a kind of long-range monitoring and control system based on augmented reality according to claim 7, it is characterised in that：It is described remote Journey data collecting system includes industrial personal computer (100) and sensor (200), and each sensor (200) accesses the industrial personal computer (100), the sensor (200) is equipped at each monitoring object, each sensor (200) is correspondingly arranged a label (300) 。

9. a kind of long-range monitoring and control system based on augmented reality according to claim 7, it is characterised in that：It is described to deposit Storage and fault diagnosis module (80) are received and are protected according to the correspondence between label (300), monitoring object and monitoring data Deposit the monitoring data that the remote data acquisition system is sent, and according to preset alarm threshold value, by monitoring data with it is corresponding Alarm threshold value is compared, and if diagnostic monitoring object sends warning message with the presence or absence of failure there are failure；When storage and When fault diagnosis module (80) receives the inquiry request from equipment identification module (70), label letter is inquired according to label information Corresponding monitoring data are ceased, the augmented reality display module (60) is then forwarded to.

10. a kind of long-range monitoring and control system based on augmented reality according to claim 7, it is characterised in that：It is described Monitoring image includes picture and video.