CN113001569A - Live-line work mechanical arm remote operation system based on VR technology and man-machine interaction method - Google Patents

Abstract

The invention discloses a hot-line work mechanical arm remote operation system based on VR technology, which comprises an embedded control system of a mechanical arm work platform and an augmented reality monitoring and operating system, wherein the embedded control system of the mechanical arm work platform comprises an embedded control unit, a sensor input unit, an actuator control unit, a power supply management unit and a wireless communication unit; the augmented reality monitoring and operating system comprises a high-performance operation host, a man-machine interaction device and a wireless communication device, wherein the man-machine interaction device comprises a head-mounted display, a left mechanical arm joint detection device, a right mechanical gripper body sensing glove, a left mechanical gripper body sensing glove, a right mechanical gripper body sensing glove and a keyboard and mouse input device.

Description

Live-line work mechanical arm remote operation system based on VR technology and man-machine interaction method

Technical Field

The invention relates to a hot-line work mechanical arm remote control system and a man-machine interaction method based on VR technology, which are used in the field of electric power construction operation.

Background

The power industry is an important industry related to the national civilization, and if a distribution line is damaged in the operation process, the problem of large-area power supply interruption can be caused, and the personal life of residents is greatly influenced. Live working is a special engineering technology for maintaining, replacing and testing components on high-voltage electrical equipment without power failure. In order to reduce the power failure time and improve the power supply reliability and service level, live working has become the most direct and effective important means, and the importance thereof is increasingly highlighted.

At present, manual live working is a always adopted live working mode in China. In order to solve the problems that an operator works in an environment with high altitude, open air, high voltage and high electromagnetic radiation, falls, high-voltage electric shock and other casualty accidents occur, a live working robot is concerned by many experts at home and abroad, the research and the start of the live working robot are earlier in Japan and other countries, and a third-generation live working robot has the capability of three-dimensional identification on the environment and decision and control. The research and starting of the live working robot in China are relatively late, at present, a manual control system of a main hand operating rod and a hand-held terminal is adopted to realize two modes of a mechanical arm, and when the distance between the mechanical arm and a working object is relatively long, master-slave control is adopted; autonomous control is employed when the distance is close. Through hydraulic drive, take multistage insulation safeguard measure.

Because the main operation mode of the existing domestic high-voltage live working robot is that an operator and a working mechanical arm are positioned in an aerial work platform together, the operator controls a slave hand to operate through a master hand of the working mechanical arm, although the operation efficiency is high, the danger caused when the operator is completely separated from electromagnetic radiation and aerial work is avoided, and therefore, an operation room with the operator positioned on the ground or a mobile automobile is developed, and the working mode that the working mechanical arm is remotely controlled through data provided by a vision system of the robot to complete the operation is realized. This type of work enables the operator to perform work in a safer manner, but since the operator cannot directly observe the work object at a short distance, the work operation and the spatial feeling of the operating machine are likely to vary, which affects the work efficiency.

Disclosure of Invention

The invention aims to overcome the defects of the prior art, and provides a hot-line work mechanical arm remote operation system and a man-machine interaction method based on VR technology, which can enable an operator to intuitively perform remote somatosensory operation on power construction tools.

One technical scheme for achieving the above purpose is as follows: a hot-line work mechanical arm remote operation system based on VR technology comprises an embedded control system of a mechanical arm work platform and an augmented reality monitoring and operating system;

the embedded control system of the mechanical arm operation platform comprises an embedded control unit, a sensor input unit, an actuator control unit, a power management unit and a wireless communication unit; the embedded control unit is responsible for instruction receiving and sending, sensor acquisition processing and logic operation; the sensor input unit is externally connected with the panoramic camera, the PTZ monitoring camera, the film pressure sensor group and the attitude sensor group and is responsible for visual feedback, attitude sensing and operation force feedback; the actuator control unit is responsible for the motion drive of the joint of the industrial-grade six-degree-of-freedom mechanical arm, the joint of the mechanical gripper and the PTZ holder; the power supply management unit is responsible for charge-discharge management and regulated power supply output; the wireless communication unit is responsible for data parsing, encoding, compression and transmission.

The augmented reality monitoring and operating system comprises a high-performance operation host, a man-machine interaction device and a wireless communication device, wherein the man-machine interaction device comprises a head-mounted display, a left mechanical arm joint detection device, a right mechanical gripper joint detection device, a left mechanical gripper body sensing glove, a right mechanical gripper body sensing glove and a keyboard mouse input device, and the system can control the mechanical arm and the mechanical gripper in a man-machine interaction mode.

Furthermore, the augmented reality monitoring and operating system has the functions of data storage, video storage, safety management and network communication, can record operating data and system state in real time, and can also store video stream data of the panoramic camera and the monitoring camera in real time in the operation process for subsequent operation playback and fault review.

Furthermore, augmented reality monitoring and operating system is equipped with the safety control unit, and its motion range that limits the arm avoids between two arms and with self equipment including the camera to take place to interfere.

Further, the insulating layer is wrapped outside the front section clamping device of the mechanical arm, so that the error contact or interphase discharge between the two mechanical arms in the operation process is avoided.

Furthermore, the chassis of the mechanical arm is subjected to omnidirectional insulation coating, so that a loop is prevented from being formed between the high-voltage wire and the ground.

Furthermore, the system adopts a battery pack for power supply, the embedded control system of the mechanical arm operation platform and the two systems of the augmented reality monitoring and operating system respectively supply power through the two power supply systems, and the two systems are in signal connection by adopting wireless communication, so that electrical physical isolation is realized, and the mechanical arm is prevented from being electrically connected with the control cabin.

The man-machine interaction method of the hot-line work mechanical arm remote operation system based on the VR technology comprises the following steps:

s1, loading the stereoscopic images and the mechanical platform state information collected by the panoramic camera and the PTZ monitoring camera through the head-mounted display;

s2, rotating the head display to control the shooting angles of the panoramic camera and the PTZ monitoring camera;

s3, detecting the arm posture and the movement speed of an operator through the mechanical arm joint detection equipment, and controlling the arm type of the mechanical arm so as to achieve the effect of operating the mechanical arm in person;

and S4, detecting the hand gesture of an operator through the manipulator somatosensory glove, controlling the simulated mechanical gripper gripping tool or the cable, and also realizing the purpose of holding the trimming tool to finish the trimming operation of the cable.

According to the live working mechanical arm remote operation system and the human-computer interaction method based on the VR technology, a live working site is monitored through monitoring equipment such as a panoramic camera, a stereoscopic image is built through an augmented reality technology, an operator remotely utilizes the remote operation technology according to a working task, and controls the live working mechanical arm and the mechanical arm through the human-computer interaction equipment, so that the purposes of efficient and reliable live working and electrical isolation are achieved.

Drawings

Fig. 1 is a schematic structural diagram of a live-wire work mechanical arm remote operation system based on VR technology according to the present invention;

fig. 2 is a schematic diagram of a system architecture of a hot-line work robot remote control system based on VR technology according to the present invention;

fig. 3 is a schematic layout structure diagram of the embedded control system of the robot arm operation platform of the live-wire work robot arm remote operation system based on the VR technology.

Detailed Description

In order to better understand the technical solution of the present invention, the following detailed description is made by specific examples:

referring to fig. 1, the present invention provides a hot-line work robot remote control system based on VR technology, which includes a robot work platform embedded control system 1 and an augmented reality monitoring and operating system 2.

The embedded control system 1 of arm operation platform installs in the front portion of bucket arm car davit, is equipped with two six degree of freedom arms and multichannel high definition vision feedback system, can feed back operation end high definition image in real time to the control end, in order to avoid and control having electrical connection between the cabin, the automatic operation platform of arm is equipped with the lithium cell group and supplies power, realizes the electrical physics isolation. Aiming at the distribution lines of 10KV and below, the mechanical arm can carry various electrician tools to complete common electrician operation operations such as cable cutting, switching off and the like.

Augmented reality control and operating system 2 installs in the bucket arm car moving platform control cabin, can the arm on the direct control work platform, and the arm is automatic to be communicated through low-delay high-speed wireless link between work platform and the virtual reality monitored control system, avoids the high pressure to produce the injury to operating personnel through control scheme. The augmented reality monitoring and operating system comprises a high-performance operation host, a man-machine interaction device and a wireless communication device, wherein the man-machine interaction device comprises a head-mounted display, a left mechanical arm joint detection device, a right mechanical gripper joint detection device, a left mechanical gripper body sensing glove, a right mechanical gripper body sensing glove and a keyboard mouse input device, and the system can control the mechanical arm and the mechanical gripper in a man-machine interaction mode.

Fig. 2 shows the components of the live-wire work mechanical arm remote operation system based on VR technology, which includes an embedded control unit 3 of the mechanical arm automation work platform, a sensor input unit 4, an actuator control unit 5, a power management unit 6, a wireless communication unit 7, a high-performance industrial computing host 8, and a human-computer interaction device 9. The power management unit 6 supplies power to the mechanical arm operation platform independently through the configured lithium battery, and performs electrical isolation. The sensor input unit 4 can acquire a wide-angle view of 200 degrees in front of the mechanical arm through the configured panoramic camera, the two PTZ monitoring cameras and the pressure sensor group, can avoid a view blind area, can acquire pressure change of 0-10kg detected in the mechanical arm, can transmit video and pressure information coding compression to the embedded control unit 3 of the automatic operation platform of the mechanical arm, and sends the information to the high-performance industrial computing host 8 through the wireless communication unit 7 after processing.

The high-performance industrial computing host 8 comprises an operation IO interface unit, a head attitude calculation unit, a VR image operation unit, a safety management unit and a human-computer interaction interface. The VR image operation unit decodes the data transmitted by the wireless communication unit 7, and loads the image and the mechanical platform state information into the head-mounted display to form an interactive interface consisting of a three-dimensional image and the mechanical platform state information; the operator operates the human-computer interaction device 9 according to the interaction interface and the actual work task: the head-mounted display detects the head rotation amplitude, sends the turn signals of the panoramic camera and the PTZ monitoring camera through a head posture settlement unit settlement code, detects the arm posture and the movement speed of an operator through mechanical arm joint detection equipment, and sends joint control signals of two six-degree-of-freedom mechanical arms. Detect operating personnel's hand gesture through wearable body sensing gloves, send the joint control signal of two emulation gripper. The control signal is sent to the high-performance industrial computing host 8 through the IO interface unit; the safety management module can limit the motion range of the mechanical arms, avoid interference between the two mechanical arms and self equipment such as a camera and the like, and protect the safety of the equipment.

The high-performance industrial computer 6 compresses control signal codes and transmits the control signal codes to the embedded control unit 3 of the mechanical arm automatic operation platform through the wireless communication unit 5, control information after decoding is sent to the actuator control unit 5, and the left mechanical arm motor set, the right mechanical arm motor set, the left mechanical arm motor set and the right mechanical arm are controlled through the mechanical arm motion control module and the mechanical arm motion control module of the actuator control unit 5, so that the aim of remotely operating the mechanical arm is fulfilled, objects such as a grabbing tool or a cable can be completed, and the cable trimming operation of a cable trimming tool can be completed.

Fig. 3 is a layout of the embedded control system 1 of the mechanical arm operation platform, which mainly includes an operation platform 10, a guardrail 11, a bucket arm 12, an embedded control box 13 of the mechanical arm, a left mechanical arm 14, a right mechanical arm 15, a panoramic camera 16, and a PTZ monitoring camera 17. The platform 10, the guard rail 11 and the bucket arm 12 provide protection and support; the mechanical arm embedded control box 13 receives control information and controls joint corners of the left mechanical arm 14 and the right mechanical arm 15; the panoramic camera 16 can realize a wide-angle view of 200 degrees, but because the installation position is between the mechanical arms, certain angles can be shielded by the mechanical arms to cause a view blind area, in order to eliminate the view blind area, two PTZ high-definition cameras 17 are installed on two sides of the operation platform, and the panoramic camera has the functions of pan-tilt control, focal length adjustment and the like.

The hot-line work mechanical arm remote operation system based on the VR technology has three-dimensional electrical isolation modes:

(1) the spraying of an insulating coating is considered on the mechanical arm, and an insulating layer is wrapped outside the clamping device at the front section of the mechanical arm, so that the mistaken contact or interphase discharge between the two mechanical arms in the operation process is avoided;

(2) the mechanical arm chassis adopts an insulation design, so that a loop is prevented from being formed between a high-voltage wire and the ground;

(3) in order to avoid having electrical connection between arm end and the control cabin, the automatic operation platform of arm is equipped with the lithium cell group and supplies power, adopts wireless communication between two control systems, realizes electric physics and keeps apart, avoids high pressure to produce the injury to operating personnel through control scheme.

The man-machine interaction method of the hot-line work mechanical arm remote operation system based on the VR technology comprises the following steps:

s1, loading the stereoscopic images and the mechanical platform state information collected by the panoramic camera and the PTZ monitoring camera through the head-mounted display;

s2, rotating the head display to control the shooting angles of the panoramic camera and the PTZ monitoring camera;

s3, detecting the arm posture and the movement speed of an operator through the mechanical arm joint detection equipment, and controlling the arm type of the mechanical arm so as to achieve the effect of operating the mechanical arm in person;

and S4, detecting the hand gesture of an operator through the manipulator somatosensory glove, controlling the simulated mechanical gripper gripping tool or the cable, and also realizing the purpose of holding the trimming tool to finish the trimming operation of the cable.

It should be understood by those skilled in the art that the above embodiments are only for illustrating the present invention and are not to be used as a limitation of the present invention, and that changes and modifications to the above described embodiments are within the scope of the claims of the present invention as long as they are within the spirit and scope of the present invention.

Claims (7)

1. The utility model provides a teleoperation system of live working arm based on VR technique, includes the embedded control system of arm work platform and augmented reality control and operating system, its characterized in that:

the embedded control system of the mechanical arm operation platform comprises an embedded control unit, a sensor input unit, an actuator control unit, a power management unit and a wireless communication unit; the embedded control unit is responsible for instruction receiving and sending, sensor acquisition processing and logic operation; the sensor input unit is externally connected with the panoramic camera, the PTZ monitoring camera, the film pressure sensor group and the attitude sensor group and is responsible for visual feedback, attitude sensing and operation force feedback; the actuator control unit is responsible for the motion drive of the joint of the industrial-grade six-degree-of-freedom mechanical arm, the joint of the mechanical gripper and the PTZ holder; the power supply management unit is responsible for charge-discharge management and regulated power supply output; the wireless communication unit is responsible for data analysis, coding, compression and transmission;

the augmented reality monitoring and operating system comprises a high-performance operation host, a man-machine interaction device and a wireless communication device, wherein the man-machine interaction device comprises a head-mounted display, a left mechanical arm joint detection device, a right mechanical gripper joint detection device, a left mechanical gripper body sensing glove, a right mechanical gripper body sensing glove and a keyboard mouse input device, and the system can control the mechanical arm and the mechanical gripper in a man-machine interaction mode.

2. The VR technology-based live-working mechanical arm teleoperation system of claim 1, wherein the augmented reality monitoring and operating system has functions of data storage, video storage, safety management and network communication, can record operating data and system states in real time, and can also store video stream data of a panoramic camera and a monitoring camera in real time during operation for subsequent operation playback and fault review.

3. The VR technology-based teleoperation system for live-working mechanical arms of claim 1, wherein the augmented reality monitoring and operating system is provided with a safety management unit which limits a range of motion of the mechanical arms and prevents interference between the two mechanical arms and with own equipment including a camera.

4. The VR technology-based live-wire work mechanical arm remote operation system of claim 1, wherein an insulating layer is wrapped outside a front-end clamping device of the mechanical arm, so that accidental contact or interphase discharge between the two mechanical arms in an operation process is avoided.

5. The VR technology-based teleoperation system of claim 1, wherein a chassis of the robotic arm is omni-directionally insulated to avoid a loop between a high voltage line and ground.

6. The VR technology-based live-working mechanical arm remote operation system of claim 1, wherein the system is powered by a battery pack, the two systems of the mechanical arm operation platform embedded control system and the augmented reality monitoring and operating system are respectively powered by two power supply systems, and the two systems are in signal connection by wireless communication, so that electrical physical isolation is realized, and electrical connection between the mechanical arm and the operation cabin is avoided.

7. The man-machine interaction method of the hot-line work mechanical arm remote control system based on the VR technology is characterized by comprising the following steps:

s1, loading the stereoscopic images and the mechanical platform state information collected by the panoramic camera and the PTZ monitoring camera through the head-mounted display;

s2, rotating the head display to control the shooting angles of the panoramic camera and the PTZ monitoring camera;

s3, detecting the arm posture and the movement speed of an operator through the mechanical arm joint detection equipment, and controlling the arm type of the mechanical arm so as to achieve the effect of operating the mechanical arm in person;

and S4, detecting the hand gesture of an operator through the manipulator somatosensory glove, controlling the simulated mechanical gripper gripping tool or the cable, and also realizing the purpose of holding the trimming tool to finish the trimming operation of the cable.

Images