CN114537666A - Unmanned aerial vehicle and unmanned ship cooperative water surface floating garbage removing equipment and operation method - Google Patents

Unmanned aerial vehicle and unmanned ship cooperative water surface floating garbage removing equipment and operation method Download PDF

Info

Publication number
CN114537666A
CN114537666A CN202210246232.XA CN202210246232A CN114537666A CN 114537666 A CN114537666 A CN 114537666A CN 202210246232 A CN202210246232 A CN 202210246232A CN 114537666 A CN114537666 A CN 114537666A
Authority
CN
China
Prior art keywords
aerial vehicle
garbage
unmanned
unmanned ship
unmanned aerial
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210246232.XA
Other languages
Chinese (zh)
Other versions
CN114537666B (en
Inventor
杜轩
吕金迟
李宝万
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hubei Tianyi Machinery Co ltd
Original Assignee
China Three Gorges University CTGU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Three Gorges University CTGU filed Critical China Three Gorges University CTGU
Priority to CN202210246232.XA priority Critical patent/CN114537666B/en
Publication of CN114537666A publication Critical patent/CN114537666A/en
Application granted granted Critical
Publication of CN114537666B publication Critical patent/CN114537666B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B64AIRCRAFT; AVIATION; COSMONAUTICS
    • B64CAEROPLANES; HELICOPTERS
    • B64C39/00Aircraft not otherwise provided for
    • B64C39/02Aircraft not otherwise provided for characterised by special use
    • B64C39/024Aircraft not otherwise provided for characterised by special use of the remote controlled vehicle type, i.e. RPV
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B63SHIPS OR OTHER WATERBORNE VESSELS; RELATED EQUIPMENT
    • B63BSHIPS OR OTHER WATERBORNE VESSELS; EQUIPMENT FOR SHIPPING 
    • B63B35/00Vessels or similar floating structures specially adapted for specific purposes and not otherwise provided for
    • B63B35/32Vessels or similar floating structures specially adapted for specific purposes and not otherwise provided for for collecting pollution from open water
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B64AIRCRAFT; AVIATION; COSMONAUTICS
    • B64DEQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
    • B64D47/00Equipment not otherwise provided for
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B64AIRCRAFT; AVIATION; COSMONAUTICS
    • B64DEQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
    • B64D47/00Equipment not otherwise provided for
    • B64D47/08Arrangements of cameras
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B63SHIPS OR OTHER WATERBORNE VESSELS; RELATED EQUIPMENT
    • B63BSHIPS OR OTHER WATERBORNE VESSELS; EQUIPMENT FOR SHIPPING 
    • B63B35/00Vessels or similar floating structures specially adapted for specific purposes and not otherwise provided for
    • B63B2035/006Unmanned surface vessels, e.g. remotely controlled
    • B63B2035/008Unmanned surface vessels, e.g. remotely controlled remotely controlled
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B64AIRCRAFT; AVIATION; COSMONAUTICS
    • B64UUNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2101/00UAVs specially adapted for particular uses or applications
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02TCLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T90/00Enabling technologies or technologies with a potential or indirect contribution to GHG emissions mitigation
    • Y02T90/10Technologies relating to charging of electric vehicles
    • Y02T90/14Plug-in electric vehicles

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Biophysics (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Environmental & Geological Engineering (AREA)
  • Public Health (AREA)
  • Chemical & Material Sciences (AREA)
  • Combustion & Propulsion (AREA)
  • Mechanical Engineering (AREA)
  • Ocean & Marine Engineering (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)

Abstract

The invention provides a water surface floating garbage removing device with cooperation of an unmanned aerial vehicle and an unmanned ship and an operation method, wherein the water surface floating garbage removing device comprises the unmanned aerial vehicle and the unmanned ship; the unmanned aerial vehicle builds a map of a target water area and identifies water surface garbage, the ground workstation transmits garbage coordinate information to each unmanned ship, the unmanned ship plans a global path of a water area in charge of the unmanned ship to reach a garbage point, and the garbage is picked up through a mechanical arm; the device also comprises a control system, wherein the control system is divided into three layers: a management layer, a communication layer and a control execution layer; the management layer is a ground workstation and is used for controlling, visualizing and operating the integral structure of the unmanned aerial vehicle and the integral structure of the unmanned ship; the communication layer transmits the running state of the unmanned aerial vehicle unmanned ship to the ground workstation mainly through WIFI data transmission; the control layer mainly acquires environment data and receives a control instruction of the management layer.

Description

Unmanned aerial vehicle and unmanned ship cooperative water surface floating garbage removing equipment and operation method
Technical Field
The invention belongs to the field of cleaning of floating garbage in rivers and lakes in cities, and particularly relates to automatic garbage cleaning equipment for cleaning floating garbage on water surfaces by using unmanned aerial vehicles and unmanned ships.
Background
The water areas of urban rivers and lakes are large, and garbage, blue-green algae and the like floating on the water surface have large influence on the ecological environment and are difficult to remove. Therefore, the method becomes a big link of environmental management aiming at the cleaning of the water surface garbage. A large amount of manpower is spent for special cleaning every year, the working efficiency is low, the problems of safety and the like need to be considered in manual operation on the water surface, and the method is not the optimal choice for cleaning river channel garbage. Compare manual work, unmanned ship on water surface can realize all-weather, independent operation, can carry out the operation in some artifical danger areas that are difficult to reach simultaneously, can promote clean efficiency effectively. Along with the rapid development of mobile internet and big data, the unmanned aerial vehicle is more and more widely applied in the aspects of aerial surveying and mapping, agricultural plant protection, forest fire control and the like. The unmanned ship on the water surface is deficient due to the advantages of wide visual field, strong maneuverability and the like. The unmanned aerial vehicle and the unmanned ship are combined, so that the search capability of the garbage in a large-scale water area can be greatly improved, and the garbage cleaning efficiency is improved.
Disclosure of Invention
The invention mainly solves the problems of low efficiency, high cost, unsafe and the like of manually cleaning water surface garbage. In order to improve the cleaning capacity of the water surface garbage, the invention provides a water surface floating garbage cleaning device with cooperation of an unmanned aerial vehicle and an unmanned ship and a use method thereof, the frame is constructed based on an open-source robot operating system ROS, and the main technical scheme is as follows: the unmanned aerial vehicle is used for drawing a target water area and identifying water surface garbage, the ground workstation transmits garbage coordinate information to each unmanned ship, the unmanned ship carries out global path planning on the water area in charge of the unmanned ship to reach a garbage point, and the garbage is picked up through the mechanical arm.
In order to achieve the technical features, the invention is realized as follows: a water surface floating garbage removing device with cooperation of an unmanned aerial vehicle and an unmanned ship comprises the unmanned aerial vehicle and the unmanned ship;
the unmanned aerial vehicle builds a map of a target water area and identifies water surface garbage, the ground workstation transmits garbage coordinate information to each unmanned ship, the unmanned ship carries out global path planning on a water area in charge of the unmanned ship to reach a garbage point, and the garbage is picked up through a mechanical arm;
the device also comprises a control system, wherein the control system is divided into three layers: a management layer, a communication layer and a control execution layer; the management layer is a ground workstation and is used for controlling, visualizing and operating the integral structure of the unmanned aerial vehicle and the integral structure of the unmanned ship; the communication layer transmits the running state of the unmanned aerial vehicle unmanned ship to the ground workstation mainly through WIFI data transmission; the control layer mainly acquires environment data and receives a control instruction of the management layer.
The unmanned aerial vehicle is provided with an on-board computer, a first depth camera, a first laser radar, a GPS module, a PX4 open source flight control device, a WIFI data transmission device and a charging and discharging device;
the onboard computer receives first depth camera and first laser radar information, and a control instruction; the method comprises the steps that a first depth camera obtains information of floating garbage on the water surface, wherein the information comprises garbage category and position information; the method comprises the following steps that a first laser radar obtains surrounding environment information and is matched with an SLAM algorithm to carry out positioning and mapping; the GPS module realizes the positioning of the unmanned aerial vehicle; WIFI data transmission is used for communication between unmanned aerial vehicle overall structure and unmanned ship overall structure.
The unmanned ship is provided with a mechanical arm, a garbage storage device, a driving device, a WIFI data transmission module, a high-precision inertial navigation module, a second laser radar, a GPS module, an onboard computer and a second depth camera;
the mechanical arm picks up the garbage; the high-precision inertial navigation module and the GPS module perform position and speed fusion in an extended Kalman filtering loose coupling mode, wherein the high-precision inertial navigation module comprises a speed acceleration, an angular speed and a yaw angle, the position and the speed are obtained by resolving through an inertial system, the GPS module resolves the position and the speed through a navigation system, data fusion is performed through extended Kalman filtering, and finally, the position and the speed are output after the output result is subjected to feedback correction; the second laser radar acquires surrounding environment information and performs positioning navigation by matching with an SLAM algorithm; the GPS module provides positioning for the unmanned ship; the depth camera identifies spam, including spam category and location information.
Unmanned aerial vehicle's location is built the picture and is discerned including rubbish: the unmanned aerial vehicle carries out mapping positioning through the SLAM technology, a target water area two-dimensional grid map is established through a SLAM framework Cartogrier algorithm based on map optimization, floating garbage in a water area is identified through a deep learning YOLOV3 algorithm in the mapping process, XYZ coordinates are calculated according to point cloud information in an identification frame, the XYZ coordinates are converted into a world coordinate system, and the world coordinate system is sent to the unmanned ship through a workstation.
Converting coordinates of the unmanned aerial vehicle and the unmanned ship, converting the motion of the unmanned aerial vehicle into three-dimensional space rigid motion, converting the coordinates of a first depth camera of the unmanned aerial vehicle into a base mark of the unmanned aerial vehicle, and finally converting the coordinates into a world coordinate system; the motion of the unmanned ship in the plane can be regarded as rigid motion, the rigid motion is converted into a world coordinate system through homogeneous coordinate transformation, and the unmanned ship of the unmanned plane is located in the world coordinate system.
Unmanned aerial vehicle, unmanned ship and workstation's communication, unmanned aerial vehicle, unmanned ship and workstation constitute in this communication system, and the workstation belongs to the main control computer, and unmanned aerial vehicle and unmanned ship are as following the machine, connect through the WIFI LAN between the three, and the main control computer is responsible for controlling unmanned aerial vehicle and unmanned ship, mainly realizes: the unmanned aerial vehicle and the unmanned ship realize image transmission and interaction of environmental information.
The method comprises the steps of positioning the unmanned ship, mainly obtaining the position of the unmanned ship in a map, leading the positioning of the unmanned ship to be deviated due to the influence of wind speed waves when the unmanned ship sails on the water surface, adopting a high-precision inertial navigation module and a GPS module to carry out global positioning, adopting an extended Kalman filtering loose coupling mode to estimate the state errors of the inertial navigation module and the GPS module, correcting each navigation system through feedback through estimation of the state errors, and improving the positioning precision.
Planning the path of the unmanned ship, dividing a two-dimensional grid map and areas by a ground workstation, acquiring a world coordinate system of each unmanned ship, and obtaining a garbage coordinate relation of a water area in which the unmanned ship is responsible through coordinate transformation; the unmanned ship carries out path planning from near to far according to the received garbage coordinate information of the water area in which the unmanned ship is responsible, carries out global path planning by using an A-star algorithm in navigation, and calculates the optimal path from the unmanned ship to the target position.
And the garbage picking of the mechanical arm mainly comprises the steps of using a point cloud generated by a second depth camera after reaching the position, carrying out target detection on the floating garbage on the water surface, calculating a point cloud coordinate in the identification frame, converting the acquired garbage coordinate into a mechanical arm coordinate system, and starting motion planning and grabbing by the mechanical arm.
The operation method of the water surface floating garbage removing equipment with the cooperation of the unmanned aerial vehicle and the unmanned ship comprises the following steps:
s1 is started;
s2 shows that the unmanned vehicle reaches the target water area;
s3 is that the unmanned aerial vehicle unmanned ship stops still and establishes wireless communication with the ground workstation;
s4, drawing a SLAM of the target water area according to a planned route and identifying garbage;
s5 is returned by the unmanned aerial vehicle;
s6, dividing the target water area for the ground station according to the number of unmanned ships, wherein one ship is responsible for one water area;
s7, converting the coordinates of the identified garbage into the world coordinate system;
s8, global path planning is carried out for the garbage coordinate traversal of the water area in charge of each ship;
s9 shows that the unmanned ship arrives at a target point;
s10, if the garbage is not identified, the search target is rotated in place;
s11, grabbing by a mechanical arm;
s12 is a head to unscanned area;
s13 is the end;
the specific process of S2 is as follows:
step A1: the unmanned ship unmanned aerial vehicle reaches a target water area;
step A2: checking the surrounding environment of the target water area;
the specific process of S3 is as follows:
step B1: starting an unmanned ship, an unmanned aerial vehicle and a ground workstation;
step B2: checking the running condition of the equipment;
step B3: inputting an IP address and establishing communication among the three through WIFI data transmission;
the specific process of S4 is as follows:
step C1: vertically lifting the unmanned aerial vehicle to a proper height;
step C2: starting SLAM map building, and building a two-dimensional grid map by adopting a Cartogrph algorithm;
step C3: identifying the floating garbage on the water surface through the first depth camera by using a deep learning Yolov3 algorithm while constructing the map;
the specific process of S5 is as follows:
step D1: the unmanned aerial vehicle carries the collected garbage coordinate information and the two-dimensional map and returns to the ground workstation;
the specific process of S6 is as follows:
step E1: processing a two-dimensional map and garbage coordinate information established by the unmanned aerial vehicle;
step E2: reasonably dividing the responsible water area of each ship according to the map, the number of ships and the water area environment;
the specific process of S7 is as follows:
step F1: converting the garbage coordinate calculated by the unmanned aerial vehicle into a world coordinate system by the ground workstation;
step F2: converting the coordinate system of the unmanned ship into a world coordinate system by the ground workstation;
step F3: transmitting the garbage coordinates under each area to an unmanned ship in charge of the area;
the specific process of S8 is as follows:
step G1: receiving garbage coordinate information under a world coordinate system by the unmanned ship;
step G2: setting a navigation point from near to far according to the garbage coordinate information;
step G3: carrying out global path planning according to the grid map, and finding out an optimal path through an A-star algorithm;
the specific process of S9 is as follows:
step H1: the unmanned ship sequentially goes to the areas where the garbage are located according to the planned route obtained by S8;
the specific process of S10 is as follows:
step I1: when the unmanned ship reaches the region where the floating garbage on the water surface is located, the unmanned ship identifies the garbage by using a YOLOV3 target identification algorithm;
step I2: if the garbage is not identified, the unmanned ship rotates in situ to search for a target;
the specific process of S11:
step J1: identifying a target, processing point cloud information in an identification frame, acquiring garbage coordinate information, and calculating coordinates of garbage relative to the mechanical arm through coordinate conversion;
step J2: moveit is adopted to plan a mechanical arm movement route, and a mechanical arm clamping jaw moves to reach a garbage position for grabbing;
the specific process of S12:
step K1: after the work of the area is finished, carrying the unmanned ship unmanned plane to move to the next sub-area for work, and repeating the steps S3-S11;
step K2: if the cleaning work for all the areas is completed, S13 is performed and the work is finished.
The invention has the following beneficial effects:
1. according to the invention, the unmanned aerial vehicle and the multiple unmanned ships are used for cleaning the garbage on the water surface in a cooperative working mode, so that the defect of the sight of the unmanned ships can be overcome, and the efficiency of searching for the garbage and cleaning the garbage can be effectively improved. The operation of multiple unmanned ships can shorten the working time and reduce the time consumed by cleaning garbage.
2. The invention relates to an unmanned aerial vehicle, which establishes a two-dimensional grid map for a target water area through a laser radar and a depth camera carried by the unmanned aerial vehicle, and the method comprises the following steps: and a software framework based on ROS is used, and a laser SLAM algorithm is adopted, so that the problem of mapping and positioning of the unmanned aerial vehicle is solved. A Cartogrer algorithm is adopted in a mapping algorithm, loop detection is added to the algorithm based on a graph optimization framework to eliminate accumulated errors, and the algorithm can be used for an unmanned aerial vehicle to construct a two-dimensional grid map.
3. By adopting the YOLOV3 algorithm to identify the garbage, the accuracy and the reliability of identification are ensured.
Drawings
The invention is further illustrated by the following figures and examples.
Fig. 1 is an overall structure diagram of the unmanned aerial vehicle of the present invention.
Fig. 2 is an overall configuration diagram of the unmanned ship of the present invention.
Fig. 3 is an overall block diagram of the system of the present invention.
Fig. 4 illustrates a method of positioning an unmanned ship according to the present invention.
FIG. 5 illustrates an embodiment of the present invention.
In the figure: a GPS module 101, a battery compartment 102, an onboard computer 103, a first depth camera 104, a lidar 105;
a robotic arm 201, a waste storage bin 202, a two-dimensional lidar 203, a second depth camera 204.
Detailed Description
Embodiments of the present invention will be further described with reference to the accompanying drawings.
Example 1:
referring to fig. 1-5, a water surface floating garbage removing device with cooperation of an unmanned aerial vehicle and an unmanned ship comprises the unmanned aerial vehicle and the unmanned ship; the unmanned aerial vehicle builds a map of a target water area and identifies water surface garbage, the ground workstation transmits garbage coordinate information to each unmanned ship, the unmanned ship carries out global path planning on a water area in charge of the unmanned ship to reach a garbage point, and the garbage is picked up through a mechanical arm; the device also comprises a control system, wherein the control system is divided into three layers: a management layer, a communication layer and a control execution layer; the management layer is a ground workstation and is used for controlling, visualizing and operating the integral structure of the unmanned aerial vehicle and the integral structure of the unmanned ship; the communication layer transmits the running state of the unmanned aerial vehicle unmanned ship to the ground workstation mainly through WIFI data transmission; the control layer mainly acquires environment data and receives a control instruction of the management layer. Through foretell clear away equipment, its realization frame is based on open source robot operating system ROS and is found the picture and discern surface of water rubbish to the target waters through unmanned aerial vehicle, and ground workstation sends rubbish coordinate information to each unmanned ship, and unmanned ship carries out global path planning to oneself responsible for the waters and reachs the rubbish point, picks up rubbish through the robotic arm.
Further, the unmanned aerial vehicle is provided with an on-board computer 103, a first depth camera 104, a first laser radar 105, a GPS module 101, a PX4 open source flight control device, a WIFI data transmission device and a charging and discharging device; the on-board computer 103 receives the first depth camera 104 and first lidar 105 information, as well as control instructions; the first depth camera 104 acquires information of floating garbage on the water surface, including garbage category and position information; the first laser radar 105 acquires surrounding environment information and performs positioning and mapping by matching with an SLAM algorithm; the GPS module 101 realizes the positioning of the unmanned aerial vehicle; WIFI data transmission is used for communication between unmanned aerial vehicle overall structure and unmanned ship overall structure.
Further, the unmanned ship is provided with a mechanical arm 201, a garbage storage device, a driving device, a WIFI data transmission module, a high-precision inertial navigation module, a second laser radar 203, a GPS module, an onboard computer and a second depth camera 204; the mechanical arm 201 picks up the garbage; the high-precision inertial navigation module and the GPS module perform position and speed fusion in an extended Kalman filtering loose coupling mode, wherein the high-precision inertial navigation module comprises a speed acceleration, an angular speed and a yaw angle, the position and the speed are obtained by resolving through an inertial system, the GPS module resolves the position and the speed through a navigation system, data fusion is performed through extended Kalman filtering, and finally, the position and the speed are output after the output result is subjected to feedback correction; the second laser radar 203 acquires surrounding environment information and performs positioning navigation by matching with an SLAM algorithm; the GPS module provides positioning for the unmanned ship; the depth camera identifies spam information, including spam category and location information.
Further, unmanned aerial vehicle's location is built the picture and is discerned including rubbish: the unmanned aerial vehicle carries out mapping positioning through the SLAM technology, a target water area two-dimensional grid map is established through a SLAM framework Cartogrier algorithm based on map optimization, floating garbage in a water area is identified through a deep learning YOLOV3 algorithm in the mapping process, XYZ coordinates are calculated according to point cloud information in an identification frame, the XYZ coordinates are converted into a world coordinate system, and the world coordinate system is sent to the unmanned ship through a workstation. The Cartogrer algorithm is a set of graph optimization-based Laser SLAM algorithm, supports 2D and 3D Laser SLAMs simultaneously, can be used in a cross-platform mode, and supports various sensor configurations such as Laser, IMU, Odetry and GPS. The algorithm can realize real-time positioning and mapping. In addition, the algorithm is based on a graph optimization framework, loop detection is added to eliminate accumulated errors, and the two-dimensional grid map can be constructed by the unmanned aerial vehicle. The accuracy of garbage identification is ensured by adopting the YOLOV3 algorithm.
Further, coordinate conversion is carried out on the unmanned aerial vehicle and the unmanned ship, the motion of the unmanned aerial vehicle is rigid motion in a three-dimensional space, the coordinate of the first depth camera 104 of the unmanned aerial vehicle is converted to be under the base mark of the unmanned aerial vehicle, and finally the coordinate is converted to be under a world coordinate system; the motion of the unmanned ship in the plane can be regarded as rigid motion, the rigid motion is converted into a world coordinate system through homogeneous coordinate transformation, and the unmanned ship of the unmanned plane is located in the world coordinate system. Through the coordinate transformation, subsequent path planning is facilitated.
Further, unmanned aerial vehicle, unmanned ship and workstation's communication, unmanned aerial vehicle, unmanned ship and workstation constitute in this communication system, and the workstation belongs to the main control computer, and unmanned aerial vehicle and unmanned ship are as following the machine, connect through the WIFI LAN between the three, and the main control computer is responsible for controlling unmanned aerial vehicle and unmanned ship, mainly realizes: the unmanned aerial vehicle and the unmanned ship realize the transmission of images and the interaction of environmental information. By the communication mode, the reliability of communication among the three is ensured.
Furthermore, the unmanned ship is positioned, the position of the unmanned ship in a map is mainly obtained, the unmanned ship is influenced by wind speed waves when sailing on the water surface, so that the unmanned ship is positioned in an offset mode, the high-precision inertial navigation module and the GPS module are combined to perform global positioning, the state errors of the inertial navigation module and the GPS module are estimated in an expanded Kalman filtering loose coupling mode, and the navigation systems are corrected through feedback through estimation of the state errors, so that the positioning precision is improved. Through the positioning mode, the unmanned ship can be accurately positioned in a dynamic environment.
Further, planning a path of the unmanned ship, dividing a two-dimensional grid map and areas by the ground workstation, acquiring a world coordinate system of each unmanned ship, and obtaining a garbage coordinate relation of a water area in which the unmanned ship is responsible through coordinate transformation; the unmanned ship carries out path planning from near to far according to the received garbage coordinate information of the water area in which the unmanned ship is responsible, carries out global path planning by using an A-star algorithm in navigation, and calculates the optimal path from the unmanned ship to the target position. The A-algorithm is a most effective direct search method for solving the shortest path in the static road network, and is also an effective algorithm for solving a plurality of search problems. The closer the distance estimate is to the actual value in the algorithm, the faster the final search speed. And further, the optimal path planning of the unmanned ship can be rapidly realized.
Further, the garbage picking of the mechanical arm mainly comprises the steps of performing target detection on the floating garbage on the water surface by using point cloud generated by the second depth camera 204 after the garbage reaches the position, calculating cloud coordinates of points in the identification frame, converting the acquired garbage coordinates into a mechanical arm coordinate system, and starting motion planning and grabbing by the mechanical arm. The mechanical arm is subjected to motion planning through a Moveit function package based on the ROS frame, and garbage is picked up. The Moveit is the most advanced software aiming at the mobile operation at present. It combines the latest advances in motion planning, manipulation, three-dimensional perception, kinematics, control and navigation.
Example 2:
the operation method of the water surface floating garbage removing equipment with the cooperation of the unmanned aerial vehicle and the unmanned ship comprises the following steps:
s1 is started;
s2 is that unmanned vehicle unmanned ship arrives at the target water area:
step A1: the unmanned ship unmanned aerial vehicle reaches a target water area;
step A2: checking the surrounding environment of the target water area;
s3 is unmanned aerial vehicle unmanned ship stationary, and establishes wireless communication with ground workstation:
step B1: starting an unmanned ship, an unmanned aerial vehicle and a ground workstation;
step B2: checking the running condition of the equipment;
step B3: inputting an IP address and establishing communication among the three through WIFI data transmission;
s4, mapping the SLAM of the target water area according to a planned route, and performing garbage recognition:
step C1: vertically lifting the unmanned aerial vehicle to a proper height;
step C2: starting SLAM map building, and building a two-dimensional grid map by adopting a Cartogrph algorithm;
step C3: identifying the floating garbage on the water surface through the first depth camera by using a deep learning Yolov3 algorithm while constructing the map;
s5 returns for the unmanned aerial vehicle:
step D1: the unmanned aerial vehicle carries the collected garbage coordinate information and the two-dimensional map and returns to the ground workstation;
s6, dividing the target water area for the ground station according to the number of unmanned ships, wherein one ship is responsible for one water area:
step E1: processing a two-dimensional map and garbage coordinate information established by the unmanned aerial vehicle;
step E2: reasonably dividing the responsible water area of each ship according to the map, the number of ships and the water area environment;
s7 is a step of converting the coordinates of the identified garbage into the world coordinate system:
step F1: converting the garbage coordinate calculated by the unmanned aerial vehicle into a world coordinate system by the ground workstation;
step F2: converting the coordinate system of the unmanned ship into a world coordinate system by the ground workstation;
step F3: transmitting the garbage coordinates under each area to an unmanned ship in charge of the area;
s8, performing global path planning for garbage coordinate traversal of the responsible water area of each ship:
step G1: receiving garbage coordinate information under a world coordinate system by the unmanned ship;
step G2: setting a navigation point from near to far according to the garbage coordinate information;
step G3: carrying out global path planning according to the grid map, and finding out an optimal path through an A-star algorithm;
s9 shows that the unmanned ship arrives at a target point:
step H1: the unmanned ship sequentially goes to the areas where the garbage are located according to the planned route obtained by S8;
s10 is that if the in-place rotation search target is not identified by identifying the garbage:
step I1: when the unmanned ship reaches the region where the floating garbage on the water surface is located, the unmanned ship identifies the garbage by using a YOLOV3 target identification algorithm;
step I2: if the garbage is not identified, the unmanned ship rotates in situ to search for a target;
s11 is mechanical arm grabbing:
step J1: identifying a target, processing point cloud information in an identification frame, acquiring garbage coordinate information, and calculating coordinates of garbage relative to the mechanical arm through coordinate conversion;
step J2: moveit is adopted to plan a mechanical arm movement route, and a mechanical arm clamping jaw moves to reach a garbage position for grabbing;
s12 for heading to an unscanned area:
step K1: after the work of the area is finished, carrying the unmanned ship unmanned plane to move to the next sub-area for work, and repeating the steps S3-S11;
step K2: if the cleaning work for all the regions is completed, S13 is performed and the work is finished.
And S13 is ended.

Claims (10)

1. The utility model provides an unmanned aerial vehicle and unmanned ship collaborative surface of water floats rubbish and clears away equipment which characterized in that: the unmanned aerial vehicle comprises an unmanned aerial vehicle and an unmanned ship;
the unmanned aerial vehicle builds a map of a target water area and identifies water surface garbage, the ground workstation transmits garbage coordinate information to each unmanned ship, the unmanned ship plans a global path of a water area in charge of the unmanned ship to reach a garbage point, and the garbage is picked up through a mechanical arm;
the device also comprises a control system, wherein the control system is divided into three layers: a management layer, a communication layer and a control execution layer; the management layer is a ground workstation and is used for controlling, visualizing and operating the integral structure of the unmanned aerial vehicle and the integral structure of the unmanned ship; the communication layer transmits the running state of the unmanned aerial vehicle unmanned ship to the ground workstation mainly through WIFI data transmission; the control layer mainly acquires environment data and receives a control instruction of the management layer.
2. The unmanned aerial vehicle and unmanned ship cooperative water surface floating garbage removing equipment as claimed in claim 1, wherein: the unmanned aerial vehicle is provided with an on-board computer (103), a first depth camera (104), a first laser radar (105), a GPS module (101), a PX4 open source flight control device, a WIFI data transmission device and a charging and discharging device;
the on-board computer (103) receiving first depth camera (104) and first lidar (105) information, and control instructions; a first depth camera (104) acquires information of floating garbage on the water surface, wherein the information comprises garbage category and position information; a first laser radar (105) acquires surrounding environment information and performs positioning and mapping by matching with an SLAM algorithm; the GPS module (101) realizes the positioning of the unmanned aerial vehicle; WIFI data transmission is used for communication between unmanned aerial vehicle overall structure and unmanned ship overall structure.
3. The unmanned aerial vehicle and unmanned ship cooperative water surface floating garbage removing equipment as claimed in claim 1, wherein: the unmanned ship is provided with a mechanical arm (201), a garbage storage device, a driving device, a WIFI data transmission, a high-precision inertial navigation module, a second laser radar (203), a GPS module, an onboard computer and a second depth camera (204);
the mechanical arm (201) picks up the garbage; the high-precision inertial navigation module and the GPS module perform position and speed fusion in an extended Kalman filtering loose coupling mode, wherein the high-precision inertial navigation module comprises a speed acceleration, an angular speed and a yaw angle, the position and the speed are obtained by resolving through an inertial system, the GPS module resolves the position and the speed through a navigation system, data fusion is performed through extended Kalman filtering, and finally, the position and the speed are output after the output result is subjected to feedback correction; the second laser radar (203) acquires surrounding environment information and performs positioning navigation by matching with an SLAM algorithm; the GPS module provides positioning for the unmanned ship; the depth camera identifies spam information, including spam category and location information.
4. The unmanned aerial vehicle and unmanned ship cooperative water surface floating garbage removing equipment as claimed in claim 2, wherein: unmanned aerial vehicle's location is built the picture and is discerned including rubbish: the unmanned aerial vehicle carries out mapping positioning through the SLAM technology, a target water area two-dimensional grid map is established through a SLAM framework Cartogrier algorithm based on map optimization, floating garbage in a water area is identified through a deep learning YOLOV3 algorithm in the mapping process, XYZ coordinates are calculated according to point cloud information in an identification frame, the XYZ coordinates are converted into a world coordinate system, and the world coordinate system is sent to the unmanned ship through a workstation.
5. The unmanned aerial vehicle and unmanned ship cooperative water surface floating garbage removing equipment as claimed in claim 4, wherein: converting coordinates of the unmanned aerial vehicle and the unmanned ship, converting the motion of the unmanned aerial vehicle into three-dimensional space rigid motion, converting the coordinates of a first depth camera (104) of the unmanned aerial vehicle into a base mark of the unmanned aerial vehicle, and finally converting the coordinates into a world coordinate system; the motion of the unmanned ship in the plane can be regarded as rigid motion, the rigid motion is converted into a world coordinate system through homogeneous coordinate transformation, and the unmanned ship of the unmanned plane is located in the world coordinate system.
6. The unmanned aerial vehicle and unmanned ship cooperative water surface floating garbage removing equipment as claimed in claim 1, wherein: unmanned aerial vehicle, unmanned ship and workstation's communication, unmanned aerial vehicle, unmanned ship and workstation constitute in this communication system, and the workstation belongs to the main control computer, and unmanned aerial vehicle and unmanned ship are as following the machine, connect through the WIFI LAN between the three, and the main control computer is responsible for controlling unmanned aerial vehicle and unmanned ship, mainly realizes: the unmanned aerial vehicle and the unmanned ship realize the transmission of images and the interaction of environmental information.
7. The unmanned aerial vehicle and unmanned ship cooperative water surface floating garbage removing equipment as claimed in claim 1, wherein: the method comprises the steps of positioning the unmanned ship, mainly obtaining the position of the unmanned ship in a map, leading the positioning of the unmanned ship to be deviated due to the influence of wind speed waves when the unmanned ship sails on the water surface, adopting a high-precision inertial navigation module and a GPS module to carry out global positioning, adopting an extended Kalman filtering loose coupling mode to estimate the state errors of the inertial navigation module and the GPS module, correcting each navigation system through feedback through estimation of the state errors, and improving the positioning precision.
8. The unmanned aerial vehicle and unmanned ship cooperative water surface floating garbage removing equipment as claimed in claim 1, wherein: planning the path of the unmanned ship, dividing a two-dimensional grid map and areas by a ground workstation, acquiring a world coordinate system of each unmanned ship, and obtaining a garbage coordinate relation of a water area in which the unmanned ship is responsible through coordinate transformation; the unmanned ship carries out path planning from near to far according to the received garbage coordinate information of the water area in which the unmanned ship is responsible, carries out global path planning by using an A-star algorithm in navigation, and calculates the optimal path from the unmanned ship to the target position.
9. The unmanned aerial vehicle and unmanned ship cooperative water surface floating garbage removing equipment as claimed in claim 1, wherein: and the garbage picking of the mechanical arm mainly comprises the steps of using a point cloud generated by a second depth camera (204) after reaching the position, carrying out target detection on the floating garbage on the water surface, calculating cloud coordinates of points in a recognition frame, converting the acquired garbage coordinates into a mechanical arm coordinate system, and starting motion planning and grabbing by the mechanical arm.
10. A method of operating a surface floating refuse removal installation in combination with an unmanned aerial vehicle and an unmanned ship as claimed in any one of claims 1 to 9, comprising the steps of:
s1 is start;
s2 shows that the unmanned vehicle reaches the target water area;
s3 is that the unmanned aerial vehicle unmanned ship stops still and establishes wireless communication with the ground workstation;
s4, drawing a SLAM of the target water area according to a planned route and identifying garbage;
s5 is returned by the unmanned aerial vehicle;
s6, dividing the target water area for the ground station according to the number of unmanned ships, wherein one ship is responsible for one water area;
s7, converting the coordinates of the identified garbage into the world coordinate system;
s8, global path planning is carried out for the garbage coordinate traversal of the water area in charge of each ship;
s9, enabling the unmanned ship to arrive at a target point;
s10 is that if the garbage is identified, the in-place rotation searching target is not identified;
s11, grabbing by a mechanical arm;
s12 is heading for an unscanned area;
s13 is the end;
the specific process of S2 is as follows:
step A1: the unmanned ship unmanned aerial vehicle reaches a target water area;
step A2: checking the surrounding environment of the target water area;
the specific process of S3 is as follows:
step B1: starting an unmanned ship, an unmanned aerial vehicle and a ground workstation;
step B2: checking the running condition of the equipment;
step B3: inputting an IP address and establishing communication among the three through WIFI data transmission;
the specific process of S4 is as follows:
step C1: vertically lifting the unmanned aerial vehicle to a proper height;
step C2: starting SLAM map building, and building a two-dimensional grid map by adopting a Cartogrph algorithm;
step C3: identifying the floating garbage on the water surface through the first depth camera by using a deep learning Yolov3 algorithm while constructing the map;
the specific process of S5 is as follows:
step D1: the unmanned aerial vehicle carries the collected garbage coordinate information and the two-dimensional map and returns to the ground workstation;
the specific process of S6 is as follows:
step E1: processing a two-dimensional map and garbage coordinate information established by the unmanned aerial vehicle;
step E2: reasonably dividing the responsible water area of each ship according to the map, the number of ships and the water area environment;
the specific process of S7:
step F1: converting the garbage coordinate calculated by the unmanned aerial vehicle into a world coordinate system by the ground workstation;
step F2: converting the coordinate system of the unmanned ship into a world coordinate system by the ground workstation;
step F3: transmitting the garbage coordinates of each region to an unmanned ship in charge of the region;
the specific process of S8 is as follows:
step G1: receiving garbage coordinate information under a world coordinate system by the unmanned ship;
step G2: setting a navigation point from near to far according to the garbage coordinate information;
step G3: carrying out global path planning according to the grid map, and finding out an optimal path through an A-star algorithm;
the specific process of S9 is as follows:
step H1: the unmanned ship sequentially goes to the areas where the garbage are located according to the planned route obtained by S8;
the specific process of S10 is as follows:
step I1: when the unmanned ship reaches the region where the floating garbage on the water surface is located, the unmanned ship identifies the garbage by using a YOLOV3 target identification algorithm;
step I2: if the garbage is not identified, the unmanned ship rotates in situ to search for a target;
the specific process of S11 is as follows:
step J1: identifying a target, processing point cloud information in an identification frame, acquiring garbage coordinate information, and calculating coordinates of garbage relative to the mechanical arm through coordinate conversion;
step J2: moveit is adopted to plan a mechanical arm movement route, and a mechanical arm clamping jaw moves to reach a garbage position for grabbing;
the specific process of S12 is as follows:
step K1: after the work of the area is finished, carrying the unmanned ship unmanned plane to move to the next sub-area for work, and repeating the steps S3-S11;
step K2: if the cleaning work for all the areas is completed, S13 is performed and the work is finished.
CN202210246232.XA 2022-03-14 2022-03-14 Unmanned aerial vehicle and unmanned ship cooperative water surface floating garbage removing equipment and operation method Active CN114537666B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210246232.XA CN114537666B (en) 2022-03-14 2022-03-14 Unmanned aerial vehicle and unmanned ship cooperative water surface floating garbage removing equipment and operation method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210246232.XA CN114537666B (en) 2022-03-14 2022-03-14 Unmanned aerial vehicle and unmanned ship cooperative water surface floating garbage removing equipment and operation method

Publications (2)

Publication Number Publication Date
CN114537666A true CN114537666A (en) 2022-05-27
CN114537666B CN114537666B (en) 2023-02-21

Family

ID=81664044

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210246232.XA Active CN114537666B (en) 2022-03-14 2022-03-14 Unmanned aerial vehicle and unmanned ship cooperative water surface floating garbage removing equipment and operation method

Country Status (1)

Country Link
CN (1) CN114537666B (en)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107117268A (en) * 2017-05-12 2017-09-01 西南科技大学 The ocean rubbish recovering method and system of a kind of heterogeneous system
CN108287542A (en) * 2018-01-04 2018-07-17 浙江大学 Unmanned plane and unmanned boat cooperation control system and method based on collaboration cloud control
CN110096056A (en) * 2019-04-08 2019-08-06 三峡大学 A kind of intelligent vehicle detection system and its control method based on unmanned aerial vehicle platform
CN110758658A (en) * 2019-11-08 2020-02-07 湘潭大学 Method for cleaning water surface floating garbage by using unmanned aerial vehicle and unmanned ship in cooperation
CN112278170A (en) * 2020-10-06 2021-01-29 武汉烽火凯卓科技有限公司 Unmanned aerial vehicle and unmanned ship cooperative water area cleaning method, system, device and storage medium
CN214150420U (en) * 2020-12-08 2021-09-07 南京信息工程大学 Unmanned aerial vehicle unmanned ship is water installation in coordination
US20210405654A1 (en) * 2018-03-22 2021-12-30 Infinium Robotics Pte Ltd Autonomous taking off, positioning and landing of unmanned aerial vehicles (uav) on a mobile platform
CN215622702U (en) * 2021-08-26 2022-01-25 浙江安防职业技术学院 Intelligent cleaning system for water surface garbage
KR20220029822A (en) * 2020-08-28 2022-03-10 금오공과대학교 산학협력단 USV-UAV Collaboration for Development of Marine Environment

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107117268A (en) * 2017-05-12 2017-09-01 西南科技大学 The ocean rubbish recovering method and system of a kind of heterogeneous system
CN108287542A (en) * 2018-01-04 2018-07-17 浙江大学 Unmanned plane and unmanned boat cooperation control system and method based on collaboration cloud control
US20210405654A1 (en) * 2018-03-22 2021-12-30 Infinium Robotics Pte Ltd Autonomous taking off, positioning and landing of unmanned aerial vehicles (uav) on a mobile platform
CN110096056A (en) * 2019-04-08 2019-08-06 三峡大学 A kind of intelligent vehicle detection system and its control method based on unmanned aerial vehicle platform
CN110758658A (en) * 2019-11-08 2020-02-07 湘潭大学 Method for cleaning water surface floating garbage by using unmanned aerial vehicle and unmanned ship in cooperation
KR20220029822A (en) * 2020-08-28 2022-03-10 금오공과대학교 산학협력단 USV-UAV Collaboration for Development of Marine Environment
CN112278170A (en) * 2020-10-06 2021-01-29 武汉烽火凯卓科技有限公司 Unmanned aerial vehicle and unmanned ship cooperative water area cleaning method, system, device and storage medium
CN214150420U (en) * 2020-12-08 2021-09-07 南京信息工程大学 Unmanned aerial vehicle unmanned ship is water installation in coordination
CN215622702U (en) * 2021-08-26 2022-01-25 浙江安防职业技术学院 Intelligent cleaning system for water surface garbage

Also Published As

Publication number Publication date
CN114537666B (en) 2023-02-21

Similar Documents

Publication Publication Date Title
Wang et al. Map-based localization method for autonomous vehicles using 3D-LIDAR
CN108226938A (en) A kind of alignment system and method for AGV trolleies
CN112113573B (en) Planning method for coverage path of single unmanned measurement boat
CN114332360A (en) Collaborative three-dimensional mapping method and system
CN111912419A (en) High-precision semantic navigation map construction method and device based on laser radar
CN108645420B (en) Method for creating multipath map of automatic driving vehicle based on differential navigation
CN102998679B (en) GIS (Geographic Information System) data acquisition method applied to unmanned vehicle
CN104067145A (en) Pruning robot system
CN111982114B (en) Rescue robot for estimating three-dimensional pose by adopting IMU data fusion
CN112346463B (en) Unmanned vehicle path planning method based on speed sampling
EP3799618B1 (en) Method of navigating a vehicle and system thereof
CN110749895B (en) Laser radar point cloud data-based positioning method
CN110758658A (en) Method for cleaning water surface floating garbage by using unmanned aerial vehicle and unmanned ship in cooperation
CN110926485A (en) Mobile robot positioning method and system based on linear features
CN110389369A (en) Canopy point cloud acquisition methods based on RTK-GPS and mobile two dimensional laser scanning
CN115790571A (en) Simultaneous positioning and map construction method based on mutual observation of heterogeneous unmanned system
Liu et al. An enhanced lidar inertial localization and mapping system for unmanned ground vehicles
CN116540706A (en) System and method for providing local path planning for ground unmanned aerial vehicle by unmanned aerial vehicle
CN107255446B (en) Dwarfing close-planting fruit tree canopy three-dimensional map construction system and method
CN114353799A (en) Indoor rapid global positioning method for unmanned platform carrying multi-line laser radar
CN114537666B (en) Unmanned aerial vehicle and unmanned ship cooperative water surface floating garbage removing equipment and operation method
US20220269281A1 (en) Method and system for generating a topological graph map
CN115049825B (en) Water surface cleaning method, device, equipment and computer readable storage medium
CN115328122B (en) Unmanned ship obstacle avoidance method and system based on multi-radar sensing
CN113741425B (en) Full-coverage path planning method and navigation system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20221019

Address after: 443300 Shilipu Industrial Park, Lucheng, Yidu City, Yichang City, Hubei Province (West of the city)

Applicant after: HUBEI TIANYI MACHINERY Co.,Ltd.

Address before: 443002 No. 8, University Road, Xiling District, Yichang, Hubei

Applicant before: CHINA THREE GORGES University

GR01 Patent grant
GR01 Patent grant
PE01 Entry into force of the registration of the contract for pledge of patent right
PE01 Entry into force of the registration of the contract for pledge of patent right

Denomination of invention: A Water Surface Floating Garbage Removal Equipment and Operation Method for Cooperation between Unmanned Aerial Vehicles and Unmanned Ships

Effective date of registration: 20230830

Granted publication date: 20230221

Pledgee: Hubei Yidu Rural Commercial Bank Co.,Ltd.

Pledgor: HUBEI TIANYI MACHINERY Co.,Ltd.

Registration number: Y2023980054449