WO2021196529A1 - Air-ground cooperative intelligent inspection robot and inspection method - Google Patents

Abstract

Provided is an air-ground cooperative intelligent inspection robot. The robot comprises a robot platform (1) and an unmanned aerial vehicle (2), wherein the robot platform (1) comprises a vehicle body (10), vehicle wheels (11), a driving assembly, a mechanical arm (12), an environmental perception assembly, a communicator (14), a robot controller (16) and a power supply assembly (17); and the unmanned aerial vehicle (2) is in communication connection with a base station by means of the communicator (14). The inspection method comprises: carrying out air-ground cooperative multi-robot positioning and mapping, which comprises: perceptual positioning calculation, map creation, and multi-information fusion positioning; and air-ground cooperative tracking and control, which comprises: unmanned aerial vehicle flight control design, robot platform trajectory tracking control, and unmanned aerial vehicle autonomous landing control. The robot can execute given navigation and inspection tasks in an all-round and all-weather mode. By means of the application of the techniques of the Internet of Things, artificial intelligence, etc., and integration with environmental perception, dynamic decision making, behavior control and an alarm device, the robot has the capabilities of autonomous perception, walking, protection, interactive communication, etc., and can complete basic, repeated and dangerous safety work, thereby reducing the cost of safety operations.

Description

Open-ground collaborative intelligent inspection robot and inspection method Technical field

The invention belongs to the technical field of robots, and specifically relates to an air-ground cooperative intelligent inspection robot and an inspection method.

Background technique

In recent years, major explosion accidents in the chemical industry are mainly caused by the leakage of hazardous chemicals, and the main reason for leakage accidents is insufficient inspections. In order to reduce the incidence of safety accidents in the chemical industry, inspection work has become one of the indispensable tasks in the chemical industry. At present, maintenance and inspection operations are mainly completed manually. Due to the uneven quality of personnel, there are usually problems such as poor safety awareness, incomplete safety responsibilities, and inadequate safety supervision work. It is difficult to ensure the reliability and accuracy of inspection work. In addition, the chemical industry usually occupies a large area, and contains a large number of pressure vessels and nearly 100 kilometers of pressure transmission pipelines. The working environment for inspection operations is very complicated, and it is difficult to complete all inspection tasks only by relying on manual labor.

With the advancement of intelligent operation and maintenance, the demand for robot inspections is increasing. The chemical industry has gradually used robots to replace manual inspection tasks. This can not only greatly reduce labor costs, but also ensure the efficiency and reliability of inspections. Although there are many advantages in applying robots to safety inspections in the chemical industry, inspection robots at this stage are generally inspection equipment of a single robot type, and their intelligence is not high, including unmanned vehicles and unmanned aerial vehicles.

At this stage, inspection robots include unmanned vehicles and drones, which mainly have the following shortcomings:

1. Lack of multi-robot collaboration technology for unmanned vehicles and UAVs:

Single unmanned vehicle inspection equipment is restricted by its own working principle, and requires too much road surface smoothness, and cannot perform inspection work on rugged roads, stair climbs, and high-altitude environments; single drone inspection equipment is limited by its cruise capability , The flight time and load capacity are very limited, it is impossible to carry a variety of sensors, and can not fly long distances.

2. Lack of autonomous mobile platforms that can carry drones:

At this stage, the inspection drone still needs operators to take it to the designated place by means of transportation when performing its tasks. This inspection method still has inefficiencies such as low efficiency and waste of human and material resources.

3. Lack of autonomy:

At this stage, when drones perform inspection tasks, operators still need to remotely control drones on-site to collect images of the chemical environment and equipment at close range. This inspection method lacks autonomy and is inefficient in inspections.

4. Lack of explosion-proof performance:

At this stage, unmanned vehicles and drones generally do not have explosion-proof performance. High-speed brushless motors and high-energy batteries will generate ultra-high currents. Once they come into contact with combustible gas, they will explode. Potential fires, explosions and other hidden dangers are already there. It is hard to imagine the consequences.

5. Lack of high-intelligence environmental perception and understanding:

At this stage, UAVs have not fully realized the intelligence of environmental image acquisition and recognition. When the one-key return operation is initiated in an emergency, it is easy to collide or entangle with obstacles on the return path, which will be harmful to chemical industry. The safety of raw materials, chemical products and inspection drones brings huge hidden dangers.

6. Lack of synchronous mapping and positioning in a dynamic environment:

At this stage, most of the robot's SLAM navigation technology is only for indoor environments, and can only be tested and operated in a relatively controllable environment. Outdoor patrol robots usually operate in a more complex environment, and require extremely high positioning and navigation technology for mobile robots. Especially the land-air cooperation system equipped with drones is subject to changes and uncertainties in the outdoor environment. SLAM technology has never had a more mature solution.

7. Lack of high-speed and high-precision path planning technology:

The difficulty of path optimization is to ensure the search speed while ensuring that the path is as optimal as possible. The current path planning technology usually does not incorporate the kinematic constraints of the robot into the global path planning, and it is impossible to ensure real-time performance while the robot is actually operating; Air-ground cooperative path planning is one of the difficulties to ensure mutual cooperation and constraints under the premise that the respective paths are feasible; the technical difficulty of local path planning is to track the global path as much as possible while completing the obstacle avoidance of the robot based on real-time sensor data.

8. Lack of accident early warning capability:

At this stage, there is no mature accident early warning technology, so it is not possible to predict the occurrence of accidents in advance. In the algorithm of system fault diagnosis and accident prediction, how to predict accidents based on the fusion of heterogeneous data based on discrete event system theory is one of the difficult points.

Summary of the invention

An object of the present invention is to provide an air-ground collaborative intelligent inspection robot, which is applied to the safety inspection work in the chemical industry.

In order to achieve the above objective, the technical solution adopted by the present invention is:

An air-ground collaborative intelligent inspection robot includes a robot platform and an unmanned aerial vehicle. The robot platform includes a vehicle body, wheels and drive components arranged on the bottom of the vehicle body, and The robot arm, the environment sensing component, the communicator, the robot controller and the power supply component, the communicator realizes the communication connection between the unmanned aerial vehicle and the base station.

Preferably, the mechanical arm includes a base provided on the vehicle body, a joint assembly rotatably connected to the base, a mechanical claw rotatably connected to the joint assembly, and a driving mechanism. The rotation driving part for the rotation of each component, the joint assembly includes one or more connecting joints that are rotatably connected head to tail, and the robot arm can be adjusted for the landing position of the drone to facilitate the The drone performs wireless charging; the robotic arm can also replace various sensors for the drone; the robotic arm can also perform operations such as grasping, rotating, and pressing in dangerous environments, such as , Perform valve switching operations.

Further preferably, the joint assembly includes a first connection joint rotatably connected with the base, a second connection joint rotatably connected with the first connection joint, and a second connection joint with the second connection joint. A third connecting joint rotatably connected with the joint, a fourth connecting joint rotatably connected with the third connecting joint, and a fifth connecting joint rotatably connected with the fourth connecting joint, the The mechanical pawl is rotatably connected with the fifth connecting joint, and the rotating drive part adopts a motor, and 6 rotating joints are driven by 6 motors to realize the movement of 6 degrees of freedom in the robot arm, and drive The mechanical claw performs a grasping operation.

Further preferably, the mechanical claw includes a claw body rotatably connected to the joint assembly, a pair of claw hands connected to the claw body, and a mechanism that drives the claw hand to perform a grasping action. Claw hand drive assembly, the claw hand drive assembly includes a worm provided on the claw body, a turbine connected to one end of the claw hand and matched with the worm, and a claw that drives the worm to rotate The hand driving part, the claw hand driving part can adopt the mechanical claw driven by the motor to realize the grasping function.

Preferably, the robot platform further includes a platform main body arranged on the vehicle body for taking off and landing and charging the drone, and the platform main body is connected to the power supply assembly. The drone provides wireless charging.

Preferably, the environmental sensing component includes a lidar, a sensor component, and a camera and camera component.

Further preferably, the sensor assembly includes a gas concentration sensor and a humidity temperature sensor. The gas concentration sensor can detect the gas composition and concentration in the air to determine whether there is a leakage of hazardous gases, such as toxic gases (carbon monoxide, The concentration of vinyl chloride, hydrogen sulfide, etc.) and flammable gas (hydrogen, methane, ethane, etc.), once the ambient temperature and humidity or harmful, combustible gas concentration is found to exceed the safety threshold, immediately report to the staff for processing, the humidity The temperature sensor can obtain the temperature and humidity of the actual inspection area environment.

Further preferably, the camera and camera assembly includes a visible light high-definition camera, an infrared camera, and a monocular camera. The infrared camera is mainly used for night patrols and shooting night video images. In order to visually perceive and understand the chemical production environment, the monocular camera can acquire and process image information of the working environment.

Preferably, the robot platform further includes a touch screen display arranged on the vehicle body, and the touch screen display provides a human-computer interaction interface for the user to facilitate the user to modify control parameters and collect relevant monitoring data.

Further preferably, a microphone and a speaker are integrated in the touch screen display, so that the robot platform has functions of sound data collection and audio playback.

Preferably, inertial navigation equipment and GPS equipment are integrated in the communicator.

Preferably, the unmanned aerial vehicle includes a fuselage, an unmanned aerial vehicle control component, a landing gear, a propeller set, an unmanned aerial vehicle driving and power supply assembly, and a camera and sensing assembly arranged on the fuselage.

Further preferably, the drone control assembly includes a flight control module, a data transmission and a video transmission module, and the flight control module includes a flight control sealing box arranged on the fuselage, and The flight controller, power manager and parameter adjustment interface in the flight control sealed box are respectively connected with the power manager and parameter adjustment interface; the data transmission and image transmission modules include settings The data transmission and image transmission sealed box on the fuselage, the data transmission and image transmission drone terminal arranged in the data transmission and image transmission sealed box, and the data transmission and image transmission are not compatible with the data transmission and image transmission sealed box. The data transmission and image transmission ground terminal connected with the human-machine terminal, and the data transmission and image transmission UAV terminal is connected with the flight controller.

Further preferably, the unmanned aerial vehicle drive and power supply assembly includes a power source explosion-proof box, a battery set in the power source explosion-proof box, an electronic speed controller and a hub, a motor seat, and a power source provided on the motor seat Motor, the battery is connected to the power manager, the battery is connected to the electronic speed governor through the hub, and the input end of the electronic speed governor is connected to the The parameter adjustment interface is connected, and the output end of the electronic speed governor is connected with the motor.

Further preferably, the camera and sensor assembly includes a camera, a sensor sealing box, a sensor control board arranged in the sensor sealing box, and a gas sensor connected to the sensor control board.

Further preferably, the propeller group is provided with four groups, each group of the propeller group is provided with two propellers, and the two propellers are arranged up and down, and the double-propeller structure can provide stronger power for the drone. The flight dynamics is so that it can carry more sensing components.

An object of the present invention is to provide an air-ground collaborative intelligent inspection method.

In order to achieve the above objective, the technical solution adopted by the present invention is:

An air-ground collaborative intelligent inspection method, including:

1) Air-ground coordinated multi-robot positioning and mapping: including perception positioning calculation, map creation, and multi-information fusion positioning, including:

Perceptual positioning calculation uses sensor components to collect surrounding environment information, and after processing the collected data, it analyzes and processes effective environment perception and detection data;

Map creation includes modeling and scanning the environment, collecting data from sensor components and creating local 3D point cloud maps in their respective reference coordinate systems, initial alignment of the motion trajectories of the drone and robot platform, and extracting two local maps Optimize the map alignment of the two local maps for the ground plane part, and perform global optimization and adjustment on the motion trajectories and local maps obtained by the drone and robot platform;

Multi-information fusion positioning includes fusion calculation of absolute position information such as relative position information and GPS global coordinates, a priori map and current perception data registration, to obtain robot position and attitude information,

2) Air-ground coordinated tracking and control: including UAV flight control system design, robot platform trajectory tracking control, and UAV self-service landing control, including:

The design of the UAV flight control system includes the establishment of dynamic model equations for the six degrees of freedom of the UAV, analysis of the UAV actuator model composed of the motor model and the propeller aerodynamic model, and the use of the actual measured tension and torque Curve calculation of UAV-related aerodynamic parameters; according to the kinematics and dynamics model of UAV, the model is divided into two parts: attitude and position. The motion control method is divided into two parts: position control and attitude control. The attitude of the drone is called a target point. The path control of the drone is a collection of many target points in space. The drone needs to arrive at the planned target point in the order of the target points;

The trajectory tracking control of the robot platform uses the DDPG algorithm to control the robot platform to track the planned path according to the status information of the robot platform and the information feedback from the environment;

The autonomous landing control of the UAV includes flying over the unmanned vehicle through the planned route, searching for the visual sign through the visual guidance system, detecting the visual sign, and starting the automatic guided landing procedure to realize the autonomous landing of the UAV.

Preferably, the method further includes the detection and early warning of accidents, including the establishment of a mapping relationship from faults to sensor events and a mapping relationship from accidents to sensor event sequences based on the faults and accidents that may occur in the actual system, according to the established mapping relationship , Construct a system diagnostic device based on the state tree. The diagnostic device conducts fault detection and accident prediction online in real time by observing system events. When a system failure is detected, the diagnostic device issues a system warning; the diagnostic device calculates the probability of an accident in real time. If the probability exceeds the threshold set by the system, a warning is issued.

Due to the application of the above technical solutions, the present invention has the following advantages compared with the prior art:

The invention builds an all-weather autonomous navigation multi-type robot platform to ensure that the robot can perform given navigation and inspection tasks in all directions and all-weather. It comprehensively uses the Internet of Things, artificial intelligence, cloud computing, big data and other technologies to integrate environmental perception and dynamics. Decision-making, behavior control and alarm devices, with autonomous perception, autonomous walking, autonomous protection, interactive communication and other capabilities, can help humans complete basic, repetitive, and dangerous security tasks, promote security service upgrades, and reduce security operating costs. Functional integrated intelligent equipment.

Description of the drawings

Figure 1 is a schematic structural diagram of this embodiment;

Figure 2 is a schematic diagram of the structure of the robot platform in this embodiment (the main body of the blanking platform);

Figure 3 is a schematic diagram of the structure of the robotic arm in this embodiment;

Figure 4 is a schematic diagram of the structure of the mechanical claw in this embodiment;

Figure 5 is a schematic diagram of the structure of the drone in this embodiment;

Fig. 6 is a schematic block diagram of the structure of this embodiment;

Figure 7 is a schematic diagram of the relationship between the inspection system in this embodiment;

Fig. 8 is a work flow chart for realizing image perception;

Fig. 9 is a schematic block diagram of the positioning and mapping of multi-robots cooperatively in the hollow ground according to this embodiment;

FIG. 10 is a schematic block diagram of the sensing and positioning calculation in this embodiment;

Fig. 11 is a schematic block diagram of map creation in this embodiment;

Fig. 12 is a schematic block diagram of multi-information fusion positioning in this embodiment;

Figures 13a and 13b are the dynamic model of the UAV rotor wing in this embodiment;

Figure 14 is a schematic block diagram of the UAV tracking control system in this embodiment;

Figure 15 is a schematic diagram of the state machine of the drone controller in this embodiment;

Figure 16 is a schematic block diagram of the design of the DDPG trajectory tracking controller in this embodiment;

Figure 17 is a schematic diagram of the tracking error design in this embodiment: P is a point at a designated distance L in front of the trolley, q is a target point on the trajectory, and pq is perpendicular to L;

Figure 18 is a schematic diagram of the UAV/unmanned vehicle collaborative inspection process in this embodiment;

Fig. 19 is a schematic diagram of extracting event information from sensor data in this embodiment;

Fig. 20 is a schematic block diagram of an accident prediction related model and architecture in this embodiment;

FIG. 21 is a schematic block diagram of video recognition of a person's action behavior in this embodiment.

In the above drawings: 1. Robot platform; 10, car body; 11, wheels; 12, mechanical arm; 120, base; 121, mechanical claw; 1210, claw body; 1211, claw hand; 1212, worm; 1213, turbine 122. The first connection joint; 123. The second connection joint; 124. The third connection joint; 125. The fourth connection joint; 126. The fifth connection joint; 130. Lidar; 131. Gas concentration sensor; 132. Humidity. Temperature sensor; 133, visible light high-definition camera; 134, infrared camera; 135, rotatable structure; 14, communicator; 15, touch screen display; 16, robot controller; 17, power supply components; 18, platform main body; 2, UAV; 20. Fuselage; 21. Landing gear; 22. Propeller; 23. Camera.

Detailed ways

The technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. The indicated orientation or positional relationship is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the pointed device or element must have a specific orientation or a specific orientation. The structure and operation cannot therefore be understood as a limitation of the present invention. In addition, the terms "first", "second", and "third" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance.

In the description of the present invention, it should be noted that the terms "installed", "connected", and "connected" should be understood in a broad sense unless otherwise clearly specified and limited. For example, they can be fixed or detachable. Connected or integrally connected; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal communication between two components. For those of ordinary skill in the art, the specific meanings of the above-mentioned terms in the present invention can be understood in specific situations.

An air-ground collaborative intelligent inspection robot as shown in Figures 1 and 2 includes a robot platform 1 and an unmanned aerial vehicle 2. The robot platform 1 and UAV 2 will be described in detail below.

The robot platform 1 includes a car body 10, wheels and driving components arranged on the bottom of the car body 10, and a robot arm 12, an environment sensing component, a communicator 14, a touch screen display 15, a robot controller 16 and Power supply assembly 17. in:

The wheels and driving components are driven by four motors with a reducer to drive the wheels 11 to rotate, and the robot controller 16 is used to control the four wheels 11 respectively, and the steering is realized through the differential speed.

As shown in Figure 3, the robotic arm 12 includes a base 120 arranged on the vehicle body 10, a joint assembly rotatably connected to the base 120, a mechanical pawl 121 rotatably connected to the joint assembly, and a rotation that drives the rotation of various components. Drive. The joint component includes one or more connecting joints that are rotatably connected head to tail in turn. In this embodiment: the joint component includes a first connection joint 122 rotatably connected with the base 120, a second connection joint 123 rotatably connected with the first connection joint 122, and a second connection joint 123 rotatably connected The third connecting joint 124 is rotatably connected to the third connecting joint 124, the fifth connecting joint 126 is rotatably connected to the fourth connecting joint 125, and the mechanical pawl 121 is connected to the fifth connecting joint 126. Rotatable connection. The rotating driving part is a motor, and 6 rotating joints are driven by 6 motors to realize the movement of 6 degrees of freedom in the mechanical arm 12, and drive the mechanical pawl 121 to perform a grasping operation.

The robotic arm 12 can adjust the landing position of the drone 2 to facilitate wireless charging of the drone 2; the robotic arm 12 can also replace various sensors for the drone 2; the robotic arm 12 can also perform in dangerous environments Operation tasks such as grasping, rotating, pressing, etc., for example, performing valve switching operations.

As shown in Figure 4, the mechanical pawl 121 includes a pawl body 1210 rotatably connected to the fifth connecting joint 126 in the joint assembly, a pair of pawl hands 1211 connected to the pawl body 1210, and a driving pawl hand 1211 for grasping action The claw hand drive assembly. In this embodiment, the claw hand drive assembly includes a worm 1212 arranged on the claw body 1210, a turbine 1213 connected to one end of the claw hand 1210 and matched with the worm 1212, and a claw hand driving member that drives the worm 1212 to rotate. The claw hand 1211 is provided with serrations to help ensure the reliability of grasping. The gripper driver can use, for example, a motor to drive the mechanical gripper 121 to achieve a gripping function.

Environmental sensing components include lidar 130, sensor components, and camera and camera components. in:

The lidar 130 is a radar system that emits a laser beam to detect characteristic quantities such as the position and speed of a target. The robot platform 1 uses the lidar 130 to detect the dynamic environment, and collect parameters such as target distance, azimuth, height, speed, posture, and shape. The lidar 130 is used to detect obstacle information around the robot platform 1, and after the raw data of the lidar 130 is obtained, data preprocessing is required. Since the initial laser data is relatively messy, including some abnormal data obtained from measurement errors and accidental errors, it needs to be filtered. Usually, low-pass filtering or Gaussian filtering can be used. The amount of raw data is determined by the resolution of the sensor, which is usually huge. In order to facilitate the application in practice, the data needs to be down-sampled. The lidar data after data filtering and down-sampling will be used in obstacle recognition. , Object detection and other related perception functions.

The sensor assembly includes a gas concentration sensor 131 and a humidity temperature sensor 132. The gas concentration sensor 131 can detect the gas composition and concentration in the air to determine whether there is a leak of hazardous gases, such as toxic gases (carbon monoxide, vinyl chloride, hydrogen sulfide, etc.) and flammable gases (hydrogen, methane, ethane, etc.) Once it is found that the ambient temperature and humidity or the concentration of harmful or combustible gas exceeds the safety threshold, it should be reported to the staff immediately for processing. The humidity temperature sensor 132 can obtain the temperature and humidity conditions of the actual inspection area environment.

The camera and camera components include a visible light high-definition camera 133, an infrared camera 134, and a monocular camera. The infrared camera 134 is mainly used for night patrols and shooting night video images. In order to be able to visually perceive and understand the chemical production environment, a monocular camera can acquire and process image information of the working environment. The inspection requirements of the chemical environment usually include the following categories: whether the protective equipment of the staff is fully worn; whether the appearance of the production equipment is intact; whether the key indicator is normal, etc.

To solve these problems, the existing methods usually adopt template matching method for recognition, but this method has poor generalization. If the matching target in the original image rotates or changes in size, the effect of this method is poor. This embodiment uses a deep learning method to recognize images. First, collect image samples of objects in the work scene and set corresponding labels. Then, build a deep neural network and train the samples. Through training, the network can perform image processing. Recognition, classification and other functions, as shown in Figure 8.

A rotatable structure 135 is provided on the vehicle body 10. For example, the laser radar 130, the visible light high-definition camera 133, and the infrared camera 134 can all be set on the rotatable structure 135.

The communicator 14 is a wireless communicator, and the communicator 14 is mainly responsible for information transmission with the UAV 2 and the base station to ensure smooth information transmission. Inertial navigation equipment and GPS equipment are integrated in the communicator 14.

The touch screen display 15 provides a human-computer interaction interface for the user to facilitate the user to modify control parameters and collect relevant monitoring data. Through the human-computer interaction interface and the high-intelligence robot system, the system operation process is simplified to reduce labor costs. The touch screen display 15 is integrated with a microphone and a speaker, so that the robot platform 1 has the functions of sound data collection and audio playback.

The power supply assembly 17 includes an explosion-proof box, a battery and a motor arranged in the explosion-proof box. For the chemical industry, explosion-proof capability is essential. The ultra-high current of the robot is prone to electric sparks during the working process. Once it comes into contact with combustible gas, it will cause explosion hazards. Such hidden dangers will be unimaginable consequences. During the inspection process, if there is a flammable gas leakage, it is very likely to ignite the flammable gas. The explosion-proof performance is the most important technical feature of the system. The explosion-proof box structure is adopted to ensure its explosion-proof performance.

In addition, the robot platform 1 also includes a platform main body 18 arranged on the vehicle body 10 for taking off, landing and charging the drone 1, and the platform main body 18 is connected to the power supply assembly 17. The platform main body 18 provides a loading and landing platform for the UAV 2. The platform main body 18 is circular, and the circular platform main body 18 is more suitable for the random error of the landing position of the UAV 2. At the same time, the platform main body 18 also provides the wireless charging function for the UAV 2, which must be charged in time after the UAV 2 has performed an inspection mission. Therefore, the platform main body 18 uses wireless charging after the UAV 2 returns to the take-off and landing platform. Function to charge it. The high-power wireless power supply technology provided by the platform main body 18 does not require any physical connection. The charging operation is completed through non-radiative wireless energy transmission, which can completely eliminate the manual operation of the UAV 2 during charging, which greatly improves the unmanned operation. The automation level of machine inspection.

As shown in Figures 5 and 6: UAV 2 includes a fuselage 20, UAV control components, landing gear 21, a propeller set, UAV drive and power components, and cameras and sensors arranged on the fuselage 20. Components.

The drone control component includes a flight control module, a data transmission and a video transmission module. The flight control module includes a flight control sealed box arranged on the fuselage 20, a flight controller, a power manager and a regulator arranged in the flight control sealed box. The flight controller is connected to the power manager and the parameter adjustment interface respectively. The flight controller contains a barometer, an inertial navigation system and an attitude stabilization system. The data transmission and image transmission module includes the data transmission and image transmission sealed box set on the fuselage 20, the data transmission and image transmission drone terminal set in the data transmission and image transmission sealed box, and the data transmission and image transmission without The data transmission and image transmission ground terminal connected to the human-machine terminal, and the data transmission and image transmission UAV terminal are connected to the flight controller.

The drone drive and power supply components include the power source explosion-proof box, the battery set in the power source explosion-proof box, the electronic speed controller and the hub, the motor base, the motor set on the motor base, the battery is connected to the power manager, and the battery passes through the hub It is connected with the electronic speed governor, the input end of the electronic speed governor is connected with the parameter adjustment interface, and the output end of the electronic speed governor is connected with the motor.

The camera and sensor assembly includes a camera 23, a sensor sealing box, a sensor control board arranged in the sensor sealing box, and a gas sensor connected to the sensor control board.

The propeller group is provided with four groups, each group of propeller group is provided with two propellers 22, and the two propellers 22 are arranged up and down. The design of the double-propeller structure can provide stronger flight power for the UAV, so as to carry more sensor components.

In addition, a robot platform can carry multiple drones, and the style of the robot platform is not limited, such as sensor components, the installation position of the manipulator, the increase/decrease of wheels, and the similar explosion-proof design. These changes also belong to the original The scope of protection applied for.

The following specifically describes the method of cooperative intelligent inspection of the lower space and ground:

1: Cooperative multi-robot positioning and mapping in the open space:

It mainly includes high-precision and low-latency perception and positioning calculation, open-ground collaborative multi-robot map creation in a chemical environment, and multi-information fusion positioning in a high-dynamic chemical environment. Specifically:

As shown in Figure 10: Perceptual positioning calculation is realized by three parts, including sensor unit, clock synchronization device and computer unit.

The sensor unit uses industrial cameras (color and grayscale), three-dimensional lidar, inertial navigation unit, GPS and other equipment to collect information about the surrounding environment of the robot; the raw data generated by each sensor unit is synchronized by the clock and sent to the computer unit; the computer unit After the sensor data is collected, the data is preprocessed. The processing content includes point cloud noise filtering, normal vector analysis, feature point extraction, feature description calculation, etc. The computer unit is installed on the drone and the robot platform, and the effective environment is The sensing and chemical detection data are transmitted back to the ground workstation for comprehensive analysis and processing.

The open-ground collaborative multi-robot map creation in the chemical environment mainly solves the problems of map holes and lack of perspective caused by the limited perspective of a single robot, and is the fundamental guarantee for the construction of high-precision full-coverage environmental maps. In this embodiment, the three-dimensional geometric model of the environment of the chemical plant where the robot is located is reconstructed based on the perception data sent by the computer unit on the drone and the robot platform.

As shown in Figure 11: The map creation steps include:

1. UAVs and robot platforms perform modeling and scanning of the chemical plant area environment in the remote control mode, and the motion trajectories of the two are in the following state, collect the sensor data of their respective computer units and perform local three-dimensional point clouds in their respective reference coordinate systems The creation of the map;

2. Use the GPS location information on the drone and the robot platform to initially align the trajectories of the two;

3. Extract the ground plane part of the two local maps, and then optimize the map alignment of the two local maps based on the surface-to-surface closest point iteration algorithm;

4. Based on the optimization theory, global optimization and adjustment of the motion trajectory and local map obtained by the drone and robot platform are performed to obtain a more accurate chemical plant environment model.

As shown in Figure 12: Multi-information fusion positioning introduces an extended Kalman filter framework, which can perform absolute registration of relative position information such as inertial navigation integral, wheel odometer, laser odometer, GPS global coordinates, prior map and current perception data. The position information is effectively fused and calculated to obtain high-precision and low-latency robot position and posture information.

2. Air-ground coordinated tracking and control:

It mainly includes UAV flight control system design, robot platform trajectory tracking control, and UAV self-service landing control. Specifically:

UAV flight control system design:

Establish dynamic model equations for the six degrees of freedom of the UAV, analyze the UAV actuator model combined by the motor model and the propeller aerodynamic model, and use the actual measured tension and torque curves to calculate the UAV related air Power parameters. The model can calculate the pulling force and torque received by each actuator at this time according to the input airflow velocity and the speed and angular velocity of the four sets of UAV propeller sets, so as to realize the simulation verification of the model, as shown in Figures 13a and 13b. .

According to the kinematics and dynamics model of the drone, the model can be divided into two parts: attitude and position. The same motion control method is divided into two parts: position control and attitude control. A position and a drone attitude are called a target. Point, the path control of the UAV is a collection of many target points in space. The UAV needs to arrive at the planned target points in the order of the target points, as shown in Figure 14.

In the attitude control of the UAV, the active disturbance rejection controller is used to realize the control of the altitude, yaw, pitch, and roll motion parameters of the UAV. In the position control, the position controller based on the anti-stepping method is designed to make The UAV can complete the tracking of the target trajectory. In order to enable the UAV to achieve flexible maneuverability in a complex environment, it is necessary to design a state machine controller for the controller. The controller can automatically adjust the flying attitude and position of the robot according to the different environmental conditions of the UAV. As shown in Figure 15.

Tracking control of robot platform:

Use reinforcement learning for trajectory tracking control. Reinforcement learning is a method of learning the controller without knowing the control and mechanical knowledge. Reinforcement learning emphasizes the interaction with the environment and is a dynamic learning process. The Deep Deterministic Strategy Gradient Algorithm (DDPG) is a deep reinforcement learning algorithm that inherits the characteristics of the strategy gradient algorithm and the actor-critic algorithm. It uses the DDPG algorithm to control the wheeled robot based on the wheeled robot status information and environmental feedback information. The robot tracks the planned path.

First, the actual position of the robot and the target point on the planned trajectory are used to calculate the driving error through the error function, and the error information is passed to the DDPG network. The DDPG network perceives the environment through the state description, and makes the optimal decision based on the current environmental state information, and passes Set the reward function to guide self-learning, and finally realize the high-precision tracking of the planned path, as shown in Figure 16. The error function can be designed using the lateral error of the robot. The robot obtains the current position through the sensing system, and the path planning system obtains the preset trajectory position. The distance between the hypothetical point p in front of the robot and the target q is calculated as the value of the error function. Lateral errors continue to guide the robot to walk along the trajectory.

Before using the DDPG algorithm, pre-training of the network is required, and simulation training of the real environment is usually performed on the simulator. In the traditional DDPG algorithm, too few training samples will make the training efficiency very low, and the network cannot converge quickly. By improving the strategy of returning the samples to the training experience pool in the algorithm, when the samples are small, no network training is performed. Let the robot continue to explore and fill in the number of samples to achieve the purpose of accelerating training; at the same time, in order to solve the complex environment, the robot exploration cost is too large, and the large amount of trial and error process in the early stage consumes a lot of useless work. This embodiment uses the migration learning method and pre-training first. The DDPG network in a simple environment puts the trained network in a complex environment, and gradually increases the complexity of the environment, so that the network has the ability to generate sports strategies in a complex environment.

Autonomous landing control of UAV:

The robot platform (unmanned vehicle UGV) is used as the carrier of the unmanned aerial vehicle (UAV), which can realize the automatic take-off and automatic vision guided landing of the unmanned aerial vehicle. In normal inspection tasks, unmanned vehicles perform inspection work according to preset inspection routes. In some scenarios, unmanned vehicles cannot directly reach inspection points. At this time, drones can be used to reach these inspection points. , Using the drone as the extra eye of the unmanned vehicle to transmit inspection information to the robot controller of the unmanned vehicle; after the drone has performed the information collection task, it can automatically land to the On the main body of the unmanned vehicle platform, and complete the subsequent inspection tasks. The autonomous landing of the UAV needs to control the height and attitude adjustment of the UAV, which requires the design of visual signs with directionality and certain specifications. The UAV first flies over the unmanned vehicle through the planned path of the planning system, and then searches for the visual sign through the visual guidance system. Once the visual sign is detected, the automatic guided landing procedure is started to realize the autonomous landing of the UAV. The controller that controls the drone's landing process can use a fuzzy controller to increase the smoothness of the landing process, as shown in Figure 18.

Three: Detection and early warning of accidents:

In this embodiment, a discrete event system model will be established for the operating dynamics of the chemical plant, and the sensor data will be analyzed based on the theory of the discrete event system. For data with different structures in the system, such as personnel's behavior, temperature, gas concentration, etc., build a sensor data dictionary based on the data structure and data format, establish a sensor data packet analysis method, integrate logic judgments, deep learning and other methods to extract all the data in the data. The required feature information is combined with the knowledge of information theory to establish a strong mapping relationship between feature information and event information, as shown in Figure 19.

According to the faults and accidents that may occur in the actual system, the mapping relationship between faults and sensor events and the mapping relationship between accidents and sensor event sequences are established, that is, system faults are modeled as fault events in the system, and system accidents are modeled as a series of event sequences. Combination, according to the established mapping relationship, build a system diagnostic device based on the state tree, the diagnostic device by observing system events, online and real-time fault detection and accident prediction. When a system failure is detected, the diagnostic device will issue a system warning; for accident prediction, the diagnostic device will calculate the probability of an accident in real time, and when the probability exceeds the threshold set by the system, a warning will be issued, as shown in Figure 20.

Aiming at personnel safety issues in the production process of chemical companies, video surveillance technology is used to realize the detection and tracking of personnel and the recognition of specific behaviors. When performing personnel detection and tracking, comprehensive consideration of the speed and accuracy of human behavior recognition and analysis in the microenvironment Pedestrian detection and tracking algorithms that combine deep features and artificial features can be used to meet the real-time requirements of the system while achieving the best personnel detection accuracy. At the same time, combined with the item detection algorithm based on deep learning, the Whether to wear a safety helmet, protective eyepieces, whether to wear long-sleeved overalls, etc. for automatic identification, when performing identification and analysis of a person’s specific behavior, a neural network-based face recognition method is used to identify the tracking target, and the tracking information is classified and performed multiple times. Fusion, analyze the relationship between individual behavior and movement trajectory, the relationship between individual behavior and group behavior, establish an automata model of individual and group behavior, and use the relevant knowledge of the discrete event system to identify behavior, as shown in Figure 21.

In addition, similar transformations to the accident detection and early warning methods in this embodiment also belong to the protection scope of this application.

The present invention can achieve the following beneficial effects:

1. Collaboration between unmanned vehicles and drones:

The present invention designs a wheeled robot platform equipped with drones, which can meet the requirements for carrying, take-off and landing of inspection drones, get rid of the restriction that workers need to carry drones to the scene, and greatly improve the The autonomous control capability of humans and machines; the wheeled robot platform can carry drones, and can carry large high-energy batteries to charge the drone when the power is insufficient; the wheeled robot platform can also carry different types of sensor modules, including Visible light cameras, infrared cameras, laser radars, satellite navigation system receivers and other devices to make up for the lack of drone load capacity; drones can select the required sensor modules on the platform, and use robotic arms to replace sensors for drones The module greatly improves the inspection capability and efficiency of the UAV.

2. Good explosion-proof performance:

The invention uses the explosion-proof box isolation method to isolate the instantaneous high-voltage current from the outside, so that the inspection robot has explosion-proof performance.

3. Highly intelligent environmental perception and understanding:

The invention integrates a variety of sensors to perceive and understand the chemical production environment, and proposes to train a deep neural network by collecting picture samples on site to realize the recognition and detection of people or equipment in the chemical production environment. Compared with traditional recognition Method, the detection result is more reliable, and the generalization of the model is better.

4. Synchronous mapping and positioning of dynamic environment:

Facing the complex and changeable chemical environment, the present invention introduces the air-ground collaborative multi-robot SLAM method, breaks through the full coverage environment modeling technology of large-scale chemical plant areas, overcomes the multi-sensor fusion positioning technology, and realizes the inspection robot in the mixed chemical environment. High-precision modeling and safe and reliable positioning.

5. High-speed and high-precision path planning technology:

The invention applies the hybrid algorithm to the solution of path optimization, makes the algorithm more efficient, integrates kinematics constraints into the global planning, makes the trajectory more reasonable, easy to track, can complete the air-ground cooperative path planning, and ensures that the respective paths are feasible At the same time cooperate with each other in motion.

6. Accident early warning capability:

Aiming at the accidents that may occur in the production process of chemical enterprises, the present invention designs a set of fusion heterogeneous data accident prediction algorithm based on discrete event system theory, which can operate stably on the air-ground collaborative intelligent inspection robot platform, and realize the inspection robot in Complete safety inspections in the chemical operating environment, oversee the safety of accidents in the production process of chemical companies, greatly reduce safety accidents, and ensure the safety of chemical companies' production properties and personnel.

The above-mentioned embodiments are only to illustrate the technical concept and characteristics of the present invention, and their purpose is to enable those familiar with the technology to understand the content of the present invention and implement them accordingly, and should not limit the protection scope of the present invention. All equivalent changes or modifications made according to the spirit of the present invention should be covered by the protection scope of the present invention.

Claims (18)

An air-ground collaborative intelligent inspection robot, which is characterized in that it includes a robot platform and an unmanned aerial vehicle. The robot platform includes a vehicle body, wheels and drive components arranged on the bottom of the vehicle body, and The robot arm, the environment sensing component, the communicator, the robot controller and the power supply component on the vehicle body, the communicator realizes the communication connection between the unmanned aerial vehicle and the base station. The air-ground cooperative intelligent inspection robot according to claim 1, wherein the mechanical arm includes a base provided on the vehicle body, a joint assembly rotatably connected to the base, A mechanical claw rotatably connected to the joint assembly and a rotation driving member for driving each component to rotate, and the joint assembly includes one or more connecting joints that are rotatably connected end to end in turn. The air-ground cooperative intelligent inspection robot according to claim 2, wherein the mechanical claw comprises a claw rotatably connected to the joint assembly, and a claw connected to the claw body. The claw hand drive assembly that drives the claw hand to perform a grasping action. The claw hand drive assembly includes a worm provided on the claw body, connected to one end of the claw hand and connected to the claw hand. The worm gear is matched with the worm gear, and the claw hand driver that drives the worm gear to rotate. The air-ground collaborative intelligent inspection robot according to claim 2, wherein the joint assembly includes a first connecting joint rotatably connected with the base, and a first connecting joint rotatably connected with the first connecting joint. The second connection joint connected to the ground, the third connection joint rotatably connected to the second connection joint, the fourth connection joint rotatably connected to the third connection joint, and the fourth connection joint. The fifth connecting joint is rotatably connected with the connecting joint, and the mechanical pawl is rotatably connected with the fifth connecting joint. The air-ground collaborative intelligent inspection robot according to claim 1, wherein the robot platform further comprises a platform body arranged on the vehicle body for the drone to take off and land and charge, so The platform main body is connected with the power supply assembly. The air-ground collaborative intelligent inspection robot according to claim 1, wherein the environment sensing component includes a lidar, a sensor component, and a camera and camera component. The air-ground collaborative intelligent inspection robot according to claim 6, wherein the sensor component includes a gas concentration sensor and a humidity temperature sensor. The air-ground collaborative intelligent inspection robot according to claim 6, wherein the camera and camera components include a visible light high-definition camera, an infrared camera, and a monocular camera. The air-ground collaborative intelligent inspection robot according to claim 1, wherein the robot platform further comprises a touch screen display arranged on the vehicle body. The air-ground collaborative intelligent inspection robot according to claim 9, wherein a microphone and a speaker are integrated in the touch screen display. The air-ground cooperative intelligent inspection robot according to claim 1, characterized in that: the drone includes a fuselage, a drone control component, a landing gear, a propeller set arranged on the fuselage, UAV drive and power components, as well as camera and sensor components. The air-ground collaborative intelligent inspection robot according to claim 11, characterized in that: the drone control component includes a flight control module, a data transmission and a video transmission module, and the flight control module includes a flight control module set in the The flight control sealed box on the fuselage, the flight controller, the power manager and the parameter adjustment interface arranged in the flight control sealed box, the flight controller is respectively connected with the power manager and the parameter adjustment interface. The interface is connected; the data transmission and image transmission module includes a data transmission and image transmission sealed box arranged on the fuselage, and a data transmission and image transmission sealed box arranged in the data transmission and image transmission sealed box UAV terminal, data transmission and video transmission ground terminal connected to said data transmission and video transmission UAV terminal, said data transmission and video transmission UAV terminal is connected to said flight controller . The air-ground cooperative intelligent inspection robot according to claim 11, characterized in that: the unmanned aerial vehicle drive and power supply assembly includes a power source explosion-proof box, a battery arranged in the power source explosion-proof box, and an electronic speed controller And a hub, a motor seat, and a motor arranged on the motor seat, the battery is connected to the power manager, and the battery is connected to the electronic speed governor through the hub The input end of the electronic speed governor is connected with the parameter adjustment interface, and the output end of the electronic speed governor is connected with the motor. The open-ground collaborative intelligent inspection robot according to claim 11, wherein the camera and sensor components include a camera, a sensor sealed box, a sensor control board arranged in the sensor sealed box, and The gas sensor connected to the sensor control board. The air-ground cooperative intelligent inspection robot according to claim 11, wherein the propeller group is provided with four groups, each of the propeller groups is provided with two propellers, and the two propellers are arranged up and down . An air-ground collaborative intelligent inspection robot, which is characterized in that it includes a robot platform and an unmanned aerial vehicle, in which: The robot platform includes a car body, wheels and drive components arranged on the bottom of the car body, a mechanical arm arranged on the car body, an environment sensing component, a communicator, a robot controller, and a power supply component, and a power supply unit. The main body of the platform for take-off, landing and charging of the drone, a touch screen display integrated with a microphone and a speaker, the communicator realizes a communication connection between the drone and the base station, and the platform main body is connected to the base station. The power supply components are connected, where: The mechanical arm includes a base provided on the vehicle body, a joint assembly rotatably connected to the base, a mechanical claw rotatably connected to the joint assembly, and driving each component to rotate The rotating drive member, the joint assembly includes one or more connecting joints rotatably connected head to tail, and the mechanical pawl includes a paw body rotatably connected to the joint assembly, and A pair of claws on the claw body and a claw drive assembly that drives the claws to perform a grasping action. The claw drive assembly includes a worm provided on the claw body and connected to the claws. One end of the hand is matched with the worm gear, and the claw hand driving member that drives the worm to rotate. The joint assembly includes a first connecting joint rotatably connected to the base, and A second connecting joint rotatably connected with a connecting joint, a third connecting joint rotatably connected with the second connecting joint, a fourth connecting joint rotatably connected with the third connecting joint, and The fourth connection joint is rotatably connected to the fifth connection joint, and the mechanical pawl is rotatably connected to the fifth connection joint; the environment sensing component includes a laser radar, a sensor component, and a camera and Camera assembly, the sensor assembly includes a gas concentration sensor and a humidity temperature sensor, and the camera and camera assembly includes a visible light high-definition camera, an infrared camera, and a monocular camera; The UAV includes a fuselage, a UAV control component, a landing gear, a propeller set, a UAV driving and power supply component, and a camera and sensing component arranged on the fuselage, wherein: The drone control assembly includes a flight control module, a data transmission and a video transmission module, and the flight control module includes a flight control sealing box set on the fuselage, and a flight control sealing box set on the airframe. The flight controller, the power manager and the parameter adjustment interface are respectively connected to the power manager and the parameter adjustment interface; the data transmission and image transmission module includes the The data transmission and image transmission sealed box on the fuselage, the data transmission and image transmission drone end set in the data transmission and image transmission sealed box, and the data transmission and image transmission drone end are the same The connected data transmission and image transmission ground end, the data transmission and image transmission UAV end is connected with the flight controller; the UAV drive and power supply assembly includes a power source explosion-proof box, set in the The battery in the explosion-proof box of the power supply, the electronic speed controller and the hub, the motor seat, the motor arranged on the motor seat, the battery is connected to the power manager, and the battery passes through the The hub is connected to the electronic speed governor, the input end of the electronic speed governor is connected to the parameter adjustment interface, and the output end of the electronic speed governor is connected to the motor. Connection; The camera and sensor assembly includes a camera, a sensor sealing box, a sensor control board arranged in the sensor sealing box, and a gas sensor connected to the sensor control board. An air-ground collaborative intelligent inspection method, characterized in that it adopts the air-ground collaborative intelligent inspection robot according to any one of the above claims, and includes: 1) Air-ground coordinated multi-robot positioning and mapping: including perception positioning calculation, map creation, and multi-information fusion positioning. Among them: perception positioning calculation uses sensor components to collect information about the surrounding environment, and process the collected data to determine an effective environment Analyze and process sensing and detection data; Map creation includes modeling and scanning the environment, collecting data from sensor components and creating local 3D point cloud maps in their respective reference coordinate systems, initial alignment of the motion trajectories of the drone and robot platform, and extracting two local maps Optimize the map alignment of the two local maps for the ground plane part, and perform global optimization and adjustment on the motion trajectories and local maps obtained by the drone and robot platform; Multi-information fusion positioning includes fusion calculation of relative position information and GPS global coordinates, registration of a priori map and current perception data and other absolute position information to obtain robot position and posture information. 2) Air-ground coordinated tracking and control: including UAV flight control system design, robot platform trajectory tracking control, and UAV self-service landing control, including: The design of the UAV flight control system includes the establishment of dynamic model equations for the six degrees of freedom of the UAV, analysis of the UAV actuator model composed of the motor model and the propeller aerodynamic model, and the use of the actual measured tension and torque Curve calculation of UAV-related aerodynamic parameters; according to the kinematics and dynamics model of UAV, the model is divided into two parts: attitude and position. The motion control method is divided into two parts: position control and attitude control. The attitude of the drone is called a target point. The path control of the drone is a collection of many target points in space. The drone needs to arrive at the planned target point in the order of the target points; The trajectory tracking control of the robot platform uses the DDPG algorithm to control the robot platform to track the planned path according to the status information of the robot platform and the information feedback from the environment; The autonomous landing control of the UAV includes flying over the unmanned vehicle through the planned route, searching for the visual sign through the visual guidance system, detecting the visual sign, and starting the automatic guided landing procedure to realize the autonomous landing of the UAV. The air-ground collaborative intelligent inspection method according to claim 17, characterized in that: the method further includes detection and early warning of accidents, including establishing a mapping from faults to sensor events based on the faults and accidents that may occur in the actual system The relationship and the mapping relationship between the accident and the sensor event sequence. According to the established mapping relationship, a system diagnostic device based on the state tree is constructed. The diagnostic device conducts fault detection and accident prediction online and in real time by observing system events. When a system failure is detected, The diagnostic device issues a system warning; the diagnostic device calculates the probability of an accident in real time, and issues a warning when the probability exceeds the threshold set by the system.

Images