CN109029422B - Method and device for building three-dimensional survey map through cooperation of multiple unmanned aerial vehicles - Google Patents
Method and device for building three-dimensional survey map through cooperation of multiple unmanned aerial vehicles Download PDFInfo
- Publication number
- CN109029422B CN109029422B CN201810753172.4A CN201810753172A CN109029422B CN 109029422 B CN109029422 B CN 109029422B CN 201810753172 A CN201810753172 A CN 201810753172A CN 109029422 B CN109029422 B CN 109029422B
- Authority
- CN
- China
- Prior art keywords
- unmanned aerial
- information
- map
- aerial vehicle
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
- G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
- G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Automation & Control Theory (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Processing Or Creating Images (AREA)
- Instructional Devices (AREA)
Abstract
The invention provides a three-dimensional map construction method, a three-dimensional map construction device, a three-dimensional map construction system, electronic equipment and a computer readable storage medium, wherein the three-dimensional map construction method comprises the steps of acquiring at least two pieces of map data with environment survey data, which are acquired by at least two unmanned aerial vehicles in the flight process; generating a fused three-dimensional survey map with the environmental survey data based on the at least two map data. According to the invention, the enhanced three-dimensional map is constructed by sensing the tree and environment data information around the unmanned aerial vehicles by the plurality of unmanned aerial vehicles. The map is independently constructed and the investigation of the corresponding area is completed by each unmanned aerial vehicle, meanwhile, the three-dimensional map information of each unmanned aerial vehicle is fused by the background server, the fused map is shared through information interaction between the background server and each unmanned aerial vehicle, repeated detection of the same part by different unmanned aerial vehicles is avoided, and the system can be ensured to finish the construction of the three-dimensional map of the whole unknown area as soon as possible.
Description
Technical Field
The invention belongs to the field of unmanned aerial vehicle control, and particularly relates to a method and a device for building a three-dimensional survey map by cooperation of multiple unmanned aerial vehicles.
Background
With the continuous development of scientific technology, the unmanned aerial vehicle technology is becoming more and more perfect, and multiple core technologies such as artificial intelligence, autonomous flight, signal processing and the like are integrated, so that the unmanned aerial vehicle is widely applied to various industries, and besides the camera technology is used for monitoring real-time road conditions in the traffic field, and is used for detecting the fighting conditions, the interference and the destruction targets in the military field, the unmanned aerial vehicle is used for exploring the unknown field, and the unmanned aerial vehicle is becoming a new direction.
In the forest industry, the detection of forest is an important work, the collection and processing of forest data, the prediction of forest resources and the like are carried out, and the method has important significance on forest management and ecological monitoring. The method is an efficient method for collecting and processing forest data by using the unmanned aerial vehicle in the field of forest detection, the unmanned aerial vehicle is easier to deploy than a satellite, the telemetering distance is short, and the measuring precision is high. Therefore, various methods and practices for forest detection using drones are gradually emerging in the industry.
At present, most methods for detecting forest environments by using unmanned aerial vehicles need to fuse GPS information and shot environment image information to perform positioning and navigation of the unmanned aerial vehicles. However, in actual forest detection, the aircraft may need to fly under the tree crown, and at this moment, the GPS signal cannot provide position information for the unmanned aerial vehicle, and even the GPS signal can be shielded by the tree crown, so that the signal is interfered and the positioning information of the unmanned aerial vehicle cannot be acquired. Meanwhile, the GPS information can not provide barrier information; in addition, if use an unmanned aerial vehicle to carry out forest investigation, because electric power continuation of the journey is limited and because the trees barrier makes unmanned aerial vehicle can't fly fast for large tracts of land forest investigation is difficult to accomplish fast.
Disclosure of Invention
In order to solve the problems, the invention provides a method, a device, electronic equipment and a computer-readable storage medium for constructing a three-dimensional survey map by multiple unmanned aerial vehicles, and the method, the device, the electronic equipment and the computer-readable storage medium are used for completing the construction of a complete three-dimensional map of an unknown area through autonomous cooperation among the multiple unmanned aerial vehicles.
The invention provides a method for building a three-dimensional survey map by cooperation of multiple unmanned aerial vehicles, which comprises the following steps:
acquiring at least two map data with environment survey data acquired by at least two unmanned aerial vehicles in the flight process;
generating a fused three-dimensional survey map with the environmental survey data based on the at least two map data.
In some embodiments, the environmental survey data is forest environment survey data, and the three-dimensional survey map is a three-dimensional map displaying information about trees and environment in a forest area.
In some embodiments, further comprising: judging whether the map data acquired by each unmanned aerial vehicle and the generated and fused three-dimensional survey map have overlapped parts in real time based on the generated and fused three-dimensional survey map; and if the overlapped part exists, sending a command for changing the flight direction to the corresponding unmanned aerial vehicle.
In some embodiments, the determination of the overlap portion comprises: extracting a plurality of pieces of feature point information of map data acquired by each unmanned aerial vehicle, and judging that an overlapped part exists if a plurality of pieces of same feature point information exist in the map data and the generated and fused three-dimensional survey map; and/or comparing the relative position information of each unmanned aerial vehicle with the public position reference point, and if the relative position information is the same as the relative position information in the generated and fused three-dimensional survey map, judging that an overlapped part exists.
In some embodiments, the flight altitude of the drone is included above the crown and/or below the crown.
In some embodiments, the drone acquires aerial information while flying over a crown, the aerial information including at least one of tree altitude information, species information, geographic location information, crown breadth information, and crown layer point cloud density information; the unmanned aerial vehicle acquires ground information when flying under a crown, wherein the ground information comprises at least one of diameter information, roundness information, species information, density information and geographical position information under a tree.
In some embodiments, further comprising performing cross-validation based on the aerial information and ground information.
In some embodiments, the step of generating a fused three-dimensional survey map with the environmental survey data based on at least two map data comprises: and fusing the at least two map data through the determined fusion base point, wherein the fusion base point refers to a reference point capable of unifying the at least two maps to the same coordinate system.
In some embodiments, the reference point comprises an absolute coordinate point and/or a reference possessing a unique identification.
In some embodiments, said fusing the at least two map data by the determined fusion base point comprises:
and determining the direction and mileage information of each unmanned aerial vehicle flying relative to the fusion base point, and fusing the at least two map data according to the direction and mileage information.
In some embodiments, when the difference in flying heights of the at least two drones is less than a first set threshold and the distance between the at least two drones is less than a second set threshold, an instruction to change the flight direction is sent, causing at least one drone to change the flight direction.
In some embodiments, the difference in flying heights of the at least two drones is greater than a third set threshold such that the ground plane coordinates of the at least two drones at the same time are the same.
In some embodiments, the map data includes location calibration information and/or alarm information for the drone when the temperature and/or gas sensor collected temperature and/or smoke data for the drone exceeds a fourth set threshold.
The second aspect of the invention provides a device for building a three-dimensional survey map by cooperation of multiple unmanned aerial vehicles, which comprises:
the map data acquisition module is used for acquiring at least two pieces of map data with environment survey data, which are acquired by at least two unmanned aerial vehicles in the flight process;
and the three-dimensional survey map fusion generation module is used for generating a fused three-dimensional survey map with the environmental survey data based on the at least two map data.
In some embodiments, the three-dimensional survey map overlap determination module is further included, and is configured to determine, in real time, whether there is an overlap between the map data acquired by each drone and the generated and fused three-dimensional survey map, based on the generated and fused three-dimensional survey map.
In some embodiments, the unmanned aerial vehicle further comprises an instruction control module for sending an instruction of the flight direction to the unmanned aerial vehicle.
In some embodiments, the unmanned aerial vehicle further comprises an alarm module, and when the temperature and/or smoke data collected by the temperature and/or gas sensor of the unmanned aerial vehicle exceeds a fourth set threshold, the alarm module issues an alarm, wherein the alarm includes position calibration information and/or alarm information.
A third aspect of the invention provides an electronic device comprising one or more processors; storage means for storing one or more programs; when executed by the one or more processors, cause the one or more processors to implement any of the methods described above.
A fourth aspect of the invention provides a computer-readable storage medium having stored thereon computer-executable instructions operable, when executed by a computing device, to perform any of the methods described above.
In summary, the present invention provides a method, an apparatus, an electronic device, and a computer-readable storage medium for constructing a three-dimensional map based on multiple drones, where the method includes acquiring at least two pieces of map data with environmental survey data acquired by at least two drones during a flight process; generating a fused three-dimensional survey map with the environmental survey data based on the at least two map data. The invention provides a method for completing construction of a three-dimensional map of an unknown area through autonomous cooperation among a plurality of unmanned aerial vehicles. The map is independently constructed and the investigation of the corresponding area is completed by each unmanned aerial vehicle, meanwhile, the three-dimensional map information of each unmanned aerial vehicle is fused by the background server, the fused map is shared through information interaction between the background server and each unmanned aerial vehicle, repeated detection of the same part by different unmanned aerial vehicles is avoided, and the system can be ensured to finish the construction of the three-dimensional map of the whole unknown area as soon as possible.
The technical scheme of the invention has the following beneficial technical effects:
1. in the process of detecting the forest environment by the unmanned aerial vehicle, the unmanned aerial vehicle autonomously finishes obstacle shielding and flying without planning a flight route in advance;
2. in the process of constructing the three-dimensional map by the multiple unmanned aerial vehicles, the same areas constructed by any two unmanned aerial vehicles are automatically detected, the corresponding flight directions are automatically changed, the resource waste is reduced, and the construction of the three-dimensional map of the whole unknown area is completed as soon as possible;
3. the three-dimensional survey information and the three-dimensional map of the forest are obtained through detection of a plurality of unmanned aerial vehicles from different angles.
Drawings
FIG. 1 is a block flow diagram of a method of constructing a three-dimensional survey map according to the present invention;
FIG. 2 is a block diagram of the flow of the invention for the unmanned aerial vehicle to collect map data;
FIG. 3 is a block flow diagram of the fusion generation of a three-dimensional survey map of the present invention;
FIG. 4 is a block flow diagram of the path planning for a three-dimensional survey map based on fusion in accordance with the present invention;
FIG. 5 is a block diagram of an apparatus for constructing a three-dimensional survey map according to the present invention;
FIG. 6 is a block diagram of an apparatus for constructing a three-dimensional survey map according to an embodiment of the present invention;
fig. 7 is a block diagram of the general structure of a system for building a three-dimensional survey map by cooperation of multiple unmanned aerial vehicles according to the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings in conjunction with the following detailed description. It should be understood that the description is intended to be exemplary only, and is not intended to limit the scope of the present invention. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present invention.
The technical scheme of the invention can be applied to the aspect of three-dimensional map construction in unknown fields of high risk, high strength and the like. Specifically, the technical scheme of the invention mainly aims at exploring the forest condition of an unknown forest area, the three-dimensional survey map of the whole area is constructed through cooperation among a plurality of unmanned aerial vehicles, and the constructed three-dimensional survey map can clearly reflect the specific conditions of trees and the environment in the area.
A first aspect of the present invention provides a method 100 for building a three-dimensional survey map by cooperation of multiple drones, as shown in fig. 1, including the following steps:
The unmanned aerial vehicle mainly includes: the flight control unit, information acquisition unit, information identification unit, memory cell and information transmission unit. And the flight control unit controls the unmanned aerial vehicle to avoid obstacles and plan a flight path according to the position information of the unmanned aerial vehicle and the depth information of the forest environment. The depth information of the forest environment comprises specific types, diameters, heights, crown widths and the like of trees. The information acquisition unit comprises a distance sensor (such as a laser radar, a millimeter wave radar and the like), a visual sensor (such as a monocular camera, a binocular camera and the like), an inertial sensor (including a gyroscope and an accelerometer), a temperature sensor, a gas sensor and the like, and ambient parameter data and self position information are acquired according to the sensors. For example, image data around the unmanned aerial vehicle is collected through a vision sensor, distance information between surrounding obstacles and the unmanned aerial vehicle is collected through a distance sensor, and flight attitude information of the unmanned aerial vehicle is acquired through an inertial sensor. For example, the laser radar can receive reflected waves of the radar signals from objects in the detected region, and three-dimensional point cloud data (including x, y and z three-dimensional coordinate values) of the detected region is obtained according to the reflected waves; the binocular camera can capture image information of the tree and perform distance measurement on the tree in a binocular distance measurement mode; the gyroscope can acquire the flying angular velocity information of the unmanned aerial vehicle, and the accelerometer acquires the flying acceleration information of the unmanned aerial vehicle. And carrying out fusion processing on the data obtained by the sensor to obtain the flying mileage and direction information of the unmanned aerial vehicle. The information identification unit is used for calibrating and calibrating the laser radar and the vision sensor, so that point cloud data of the vision sensor and the laser radar can be aligned, depth information of an environment is identified based on surrounding environment image data and distance data acquired by the sensor, enhanced three-dimensional map information with specific tree growth conditions and relative position information of trees and the unmanned aerial vehicle are generated according to the spatial positions and the depth information of the trees. The storage unit is used for storing the data obtained by the information acquisition unit and the information identification unit. The information transmission unit is responsible for information interaction between different unmanned aerial vehicles in the whole system and between the unmanned aerial vehicle and the background ground station.
The unmanned aerial vehicle with the units finishes the avoidance and autonomous navigation of the obstacles in the flying process, continuously establishes a space three-dimensional map of the surrounding environment in the flying process, identifies the obstacles and finishes the interaction of unmanned aerial vehicle information and a ground station. And the unmanned aerial vehicles for constructing the three-dimensional survey map comprise at least two unmanned aerial vehicles, and each unmanned aerial vehicle acquires map data corresponding to the flight process of the unmanned aerial vehicle.
And 102, generating a fused three-dimensional survey map with the environmental survey data based on the at least two map data.
And fusing the at least two map data to generate a three-dimensional survey map with environment survey data, wherein the main body of the three-dimensional survey map is a ground station. The ground station mainly comprises: the device comprises a data storage unit, a data processing unit, an information transmission unit and a control unit. The data processing unit fuses and processes the three-dimensional map information and the forest survey information transmitted by the unmanned aerial vehicle and received by the information transmission unit, the three-dimensional survey map of the whole unknown region is built and stored in the data storage unit, and the control unit sends a control instruction to the unmanned aerial vehicle according to the condition that the three-dimensional survey map is completed.
In a specific embodiment of the invention, the environment survey data is forest environment survey data, and the three-dimensional survey map is a three-dimensional map displaying tree and environment information in a forest area; however, the method is not limited to the survey of forest areas, and the method of the present invention can be used to construct a three-dimensional survey map for any location area.
In a specific embodiment of the present invention, the method further comprises the steps of: judging whether the map data acquired by each unmanned aerial vehicle and the generated and fused three-dimensional survey map have overlapped parts in real time based on the generated and fused three-dimensional survey map; and if the overlapped part exists, sending a command for changing the flight direction to the corresponding unmanned aerial vehicle. The data storage unit of the ground station simultaneously stores three-dimensional map information constructed by different unmanned aerial vehicles and three-dimensional map information obtained by the whole system after the background ground station is fused, in order to avoid repeated flight of any two unmanned aerial vehicles to parts in the same area, the ground station is required to detect whether the three-dimensional map information constructed by each unmanned aerial vehicle and the fused three-dimensional map information have overlapping parts in real time, and if the overlapping parts exist, the automatic trigger control unit sends a command for changing the flight direction to the corresponding unmanned aerial vehicle through the information transmission unit.
The judgment of the overlapping portion includes: extracting a plurality of pieces of feature point information of map data acquired by each unmanned aerial vehicle, and judging that an overlapped part exists if a plurality of pieces of same feature point information exist in the map data and the generated and fused three-dimensional survey map; and/or comparing the relative position information of each unmanned aerial vehicle with a public position reference point, for example, setting the ground station as the public reference point, and if the relative position information is the same as the relative position information in the generated fused three-dimensional survey map, such as the coordinate information (x, y), determining that an overlapping part exists.
And when the construction of the three-dimensional survey map of the whole area is finished, sending an instruction for returning to a flight starting point to the unmanned aerial vehicle, wherein the instruction can be sent by a control unit of the ground station.
In a specific embodiment of the present invention, the flying height of the drone is included above and/or below the crown, i.e. divided into air flight and ground flight. Both air and ground may be multiple or one drone. The air flight refers to that the unmanned aerial vehicle flies on a canopy of a tree, the unmanned aerial vehicle acquires air information when flying above the crown, and the air information comprises at least one of height information, type information, geographical position information, canopy width information and canopy point cloud density information of the tree; the ground flight refers to the flight of an unmanned aerial vehicle under the canopy of a tree, the unmanned aerial vehicle acquires ground information when flying under the crown of the tree, and the ground information comprises at least one of diameter information, roundness information, species information, density information and geographical position information under the tree. The map data fusion between the aerial unmanned aerial vehicles and the map data fusion between the ground unmanned aerial vehicles are consistent with the method. And the ground station fuses map data and survey data constructed by the unmanned aerial vehicles on the air and the ground according to the geographic position information and the feature matching. When flying under the crown, the step of the map data with environmental survey data collected by the unmanned aerial vehicle comprises: obstacle avoidance flight is carried out through a sensor, and meanwhile obstacle information and the ground information are collected through the sensor so as to construct the map data.
As shown in fig. 2, a three-dimensional survey map construction process 200 for an unmanned aerial vehicle flying under a crown is provided, which includes the following steps:
and step 204, fusing the survey data and the map data by the unmanned aerial vehicle to obtain a three-dimensional survey map.
In a specific embodiment of the invention, the method further comprises performing cross validation based on the aerial information and the ground information to ensure accuracy and authenticity of the constructed three-dimensional survey map data.
In an embodiment of the present invention, the step of generating a fused three-dimensional survey map with the environmental survey data based on at least two map data comprises: and fusing the at least two map data through the determined fusion base point, wherein the fusion base point refers to a reference point capable of unifying the at least two maps to the same coordinate system. The reference points include absolute coordinate points and/or references possessing unique identifications. The step of performing fusion comprises: extracting features of map data acquired by each unmanned aerial vehicle, and merging overlapped areas with overlapped features; and splicing the map data acquired by the unmanned aerial vehicles to form a complete three-dimensional survey map. Setting the same geographical position reference point, determining the direction and mileage information of each unmanned aerial vehicle flying relative to the geographical position reference point, and fusing map data according to the direction and mileage information.
As shown in fig. 3, a construction process 300 of the fusion map is provided, which includes the following steps:
and step 304, the base station platform or a certain unmanned machine completes the fusion of the two maps through the determined fusion base point.
The fusion base point refers to a reference point which can be used for unifying two maps into the same coordinate system. The reference point may be an absolute coordinate point or a reference object with a unique identifier. For example, the unmanned aerial vehicle flies into a certain open area through path planning, and then obtains a more accurate GPS coordinate. At this point, the map of the drone has an absolute reference point. The second drone also obtains an absolute reference point by the same method. Based on the two reference points, the two survey maps complete the fusion. Or the two unmanned aerial vehicles take off at the same position and use one tree as a reference point. The unmanned aerial vehicle identifies the same tree through the sensor, and the tree is used as a fusion base point. The base station or a certain unmanned machine fuses the two survey maps based on the fusion base point. By the method, the system can fuse any number of survey maps, and further obtain a survey map completed by cooperation of a plurality of unmanned aerial vehicles.
In a specific embodiment of the invention, the flight path of the unmanned aerial vehicle is planned according to the generated and fused three-dimensional survey map. Fig. 4 shows a process 400 of multi-drone fusion map-based path planning, including the following steps:
and 403, replanning the flight path of the unmanned aerial vehicle according to the generated and fused three-dimensional survey map.
The invention aims to accelerate the investigation speed of a large-area forest area by using a plurality of unmanned aerial vehicles. Thus, in an initial phase, each drone performs autonomous path planning. With the fusion and expansion of survey maps generated by a plurality of unmanned aerial vehicles, each unmanned aerial vehicle can plan a path according to the fusion map. For example, a fused map-based plan may allow multiple drones to focus on flying to areas where surveys are not completed, while avoiding multiple drones investigating duplicate areas.
In a specific embodiment of the invention, the method further includes a step of judging whether the three-dimensional survey map is complete, obtaining a change range of adjacent frame image data of the map data acquired by the unmanned aerial vehicle in a machine learning manner, setting a first set threshold value of the change of the adjacent frame image data according to historical map data, and sending a command of changing a flight direction to the unmanned aerial vehicle when detecting that the change of the map data acquired by the unmanned aerial vehicle exceeds the first set threshold value.
The method for building a three-dimensional survey map by cooperation of multiple drones according to the present invention is described and further explained below by two specific embodiments.
Specific example 1: unmanned aerial vehicles (at least two) are distributed on the ground to acquire forest environment actual measurement data, namely, each unmanned aerial vehicle flies in an airspace under a forest crown to detect the specific situation of the area, the three-dimensional map construction, obstacle identification, obstacle avoidance and path planning of the unmanned aerial vehicle detection area are completed, and the background server performs distributed management and three-dimensional map fusion on each unmanned aerial vehicle.
In the process of building the three-dimensional map of each unmanned aerial vehicle, two aspects of fusion of different unmanned aerial vehicle building map data by a ground station and integrity confirmation of the whole map building are involved.
And (3) map information fusion aspect: the data processing unit of the ground station is used for extracting features of three-dimensional map information constructed by all unmanned aerial vehicles, three-dimensional image data of multiple unmanned aerial vehicle coincident regions are fused according to matching of feature points, three-dimensional maps constructed by different unmanned aerial vehicles are spliced together, and construction of the whole three-dimensional map is completed.
Furthermore, the same geographical position reference point can be set, namely a plurality of unmanned aerial vehicles carry out the environmental detection of an unknown forest region from the same GPS reference point, the specific direction and mileage information of the flight of each unmanned aerial vehicle are determined through the inertial navigation system carried by each unmanned aerial vehicle, and the fusion of a three-dimensional map is carried out according to the relative flight direction and mileage of each unmanned aerial vehicle and the GPS reference point.
Integrity confirmation aspect: automatically detecting whether the middle area of the fused three-dimensional map has the problems of missed flight, missed detection and the like according to the azimuth and the geographical position information of the fused three-dimensional map by a data processing unit of the ground station; further, the change range of the image data acquired by the unmanned aerial vehicle can be obtained through a machine learning mode according to the surrounding image data detected by the unmanned aerial vehicle, and then the change threshold between the front frame image data and the rear frame image data is set according to the historical data of different unmanned aerial vehicles, when the background cloud detects that the change of the unmanned aerial vehicle detection data exceeds the set threshold, the unmanned aerial vehicle is triggered to change the flight direction, and the edge detection of the position area is completed.
Specific example 2: the flight altitude of the drone system is comprised above the crown and below the crown. In the implementation process of the scheme, an unmanned aerial vehicle flight starting point with a real geographical position needs to be set at first, and the unmanned aerial vehicle flight is guaranteed to have a public geographical reference (x, y and z) coordinate. And flying through the same starting point to identify the flying direction, the mileage and the plane coordinate information of the unmanned aerial vehicle. For example, the x and y coordinates of the aerial drone detection data and the x and y coordinates of the ground drone detection data are aligned, and the two data are fused by three-dimensional map data.
The unmanned aerial vehicle system comprises two unmanned aerial vehicle systems with different flying heights, wherein the plane coordinates of the two unmanned aerial vehicle systems with different flying heights at the same moment are the same. Further, when the altitude difference value between a plurality of unmanned aerial vehicle systems at the same flying altitude at the same moment is smaller than a first set threshold value and the distance is smaller than a second set threshold value, an instruction for changing the flying direction is sent, and the instruction for changing the flying direction enables at least one unmanned aerial vehicle system in the plurality of unmanned aerial vehicle systems to change the flying direction. The distance (relative position) threshold value is set for between the different unmanned aerial vehicles of same flying height at the same moment, and the specific distance (relative position) between each other is detected through distance sensor in flight, when the distance between above-mentioned unmanned aerial vehicles is less than and sets for the threshold value, the flight unit of corresponding unmanned aerial vehicle of automatic trigger comes the change direction, and another unmanned aerial vehicle then does not change the flight direction, flies according to former planning route, avoids different unmanned aerial vehicles to survey same region.
Further, can adopt every ground unmanned aerial vehicle to correspond an aerial flying unmanned aerial vehicle, make both can survey the forest information of coplanar coordinate region at the same moment to the automatic processing that fuses of data information who will survey has saved alignment and the matching link of ground unmanned aerial vehicle and aerial unmanned aerial vehicle detection data.
Further, a flight mission of the drone system flying under the crown may be formulated according to the detection information obtained by the drone system flying above the crown, the flight mission including the range of the detection zone. Particularly, the flying sequence of the aerial unmanned aerial vehicle and the ground unmanned aerial vehicle can be set, the method that the aerial unmanned aerial vehicle flies first is adopted, the overall situation of the whole unknown area including the size and the range information is detected first, the information is transmitted to the background, and the background automatically formulates the flying tasks of the ground unmanned aerial vehicles according to the geographic position coordinates and the density of trees through the acquired overall area size and range information. The task of unmanned aerial vehicle flight includes the specific position range of the detection area, but does not include the specific flight path, and the flight path is independently planned by each unmanned aerial vehicle according to the specific position size of the obstacle in the detection project. Furthermore, the background server can automatically judge whether to detect a certain area according to the set tree density threshold, and the size and specific position range of the area without environment detection.
Further, an unmanned aerial vehicle is used for carrying sensors such as temperature and gas, environment data information such as temperature and smoke around the unmanned aerial vehicle is collected, and when the temperature and/or smoke data collected by the temperature and/or gas sensors of the unmanned aerial vehicle system exceed a set threshold value, the map data comprise position calibration information and/or alarm information of the unmanned aerial vehicle system. That is to say, when environmental data such as the temperature or smog around the unmanned aerial vehicle surpassed the settlement threshold value, then demarcated and reported to the police in order to avoid the dangerous condition such as conflagration to this position automation. Further, the data identification unit processes and analyzes the data, if the fire disaster is judged to happen, the unmanned aerial vehicle is automatically triggered to detect the environmental data such as the temperature, the smoke and the trees of the area around the position, the spreading direction and the smoke density of the fire disaster are identified, the information is transmitted to the ground station, and the ground station is helped to master the fire disaster situation in real time, so that the ground station can conveniently rescue according to the actual situation.
Furthermore, according to the tree growth data such as the diameter, the height and the type of the trees acquired by the unmanned aerial vehicle acquisition units, the economic data of the forest are automatically generated, and the user is helped to better manage the forest. Further, in combination with the surrounding data information detected by the drone, such as: the future growth data condition of trees in the forest is predicted according to the conditions of resources such as water, soil and sunlight, and the flight route of the corresponding unmanned aerial vehicle when detecting the forest is automatically planned according to specific time, so that the future detection of the region is facilitated.
A second aspect of the present invention provides an apparatus for building a three-dimensional survey map by cooperation of multiple drones, as shown in fig. 5, including: the map data acquisition module 2 is used for acquiring at least two pieces of map data with environment survey data, which are acquired by at least two unmanned aerial vehicles in the flight process; and the three-dimensional survey map fusion generation module 3 is used for generating a fused three-dimensional survey map with the environmental survey data based on the at least two map data.
In a specific embodiment, the apparatus further comprises, as shown in fig. 6: the three-dimensional survey map overlapping judgment module 4 is used for judging whether the map data acquired by each unmanned aerial vehicle and the generated and fused three-dimensional survey map have overlapping parts in real time based on the generated and fused three-dimensional survey map; and the instruction control module 5 is used for sending an instruction of the flight direction to the unmanned aerial vehicle. The three-dimensional survey map fusion generation module fuses the at least two map data through a determined fusion base point, wherein the fusion base point refers to a reference point which can unify the at least two maps to the same coordinate system. The reference points include absolute coordinate points and/or references possessing unique identifications. The fusion generation module extracts features of the map data acquired by the unmanned aerial vehicles and merges feature-overlapped regions; and splicing the map data acquired by the unmanned aerial vehicles to form a complete three-dimensional survey map. The fusion generation module sets a same geographical position reference point, determines the direction and mileage information of each unmanned aerial vehicle flying relative to the geographical position reference point, and fuses map data according to the direction and mileage information.
Further, the device also comprises a three-dimensional survey map integrity judging module 6, which is used for judging whether the three-dimensional survey map is complete; the path planning module is used for planning the flight path of the unmanned aerial vehicle according to the generated and fused three-dimensional survey map; and the alarm module 7 is used for sending an alarm when the temperature and/or smoke data collected by the temperature and/or gas sensor of the unmanned aerial vehicle exceed a third set threshold value, wherein the alarm comprises position calibration information and/or alarm information.
In a specific embodiment, a system for building a three-dimensional survey map by cooperation of multiple drones, as shown in fig. 7, comprises at least two drones and a ground station; shown are drones 1, … … drone N for collecting map data with environmental survey data during flight; the ground station is used for generating a fused three-dimensional survey map with the environmental survey data based on the map data; and information interaction is carried out between at least two unmanned aerial vehicles and between each unmanned aerial vehicle and the ground station through the information transmission unit. The specific components of the unmanned aerial vehicle and the ground station are as described above, and are not described herein again.
A third aspect of the present invention provides an electronic device comprising: one or more processors; storage means for storing one or more programs; when executed by the one or more processors, cause the one or more processors to implement a method for collaborative construction of a three-dimensional survey map by multiple drones as described above.
A fourth aspect of the invention provides a computer-readable storage medium having stored thereon computer-executable instructions operable, when executed by a computing device, to perform a method of collaborative construction of a three-dimensional survey map by multiple drones as described above.
In summary, the present invention provides a method, an apparatus, a system, an electronic device and a computer-readable storage medium for constructing a three-dimensional map based on multiple drones, wherein the method includes acquiring at least two pieces of map data with environmental survey data acquired by at least two drones in a flight process; generating a fused three-dimensional survey map with the environmental survey data based on the at least two map data. Through the technical scheme, the invention can realize that: in the process of detecting the forest environment by the unmanned aerial vehicle, the unmanned aerial vehicle autonomously finishes obstacle-shielding flight and route planning without planning a flight route in advance; in the process of constructing the three-dimensional map by the multiple unmanned aerial vehicles, the same areas constructed by any two unmanned aerial vehicles are automatically detected, the corresponding flight directions are automatically changed, the resource waste is reduced, and the construction of the three-dimensional map of the whole unknown area is completed as soon as possible; the method has the advantages that the three-dimensional survey information and the three-dimensional map of the forest are obtained through detection of multiple unmanned aerial vehicles from different angles, and future growth data of the area can be predicted to help the future detection of the area.
It is to be understood that the above-described embodiments of the present invention are merely illustrative of or explaining the principles of the invention and are not to be construed as limiting the invention. Therefore, any modification, equivalent replacement, improvement and the like made without departing from the spirit and scope of the present invention should be included in the protection scope of the present invention. Further, it is intended that the appended claims cover all such variations and modifications as fall within the scope and boundaries of the appended claims or the equivalents of such scope and boundaries.
Claims (17)
1. A method for building a three-dimensional survey map by cooperation of multiple unmanned aerial vehicles is characterized by comprising the following steps:
each unmanned aerial vehicle carries out autonomous path planning, and obstacle avoidance and autonomous navigation are completed in the flight process;
acquiring at least two map data with environment survey data acquired by at least two unmanned aerial vehicles in the flight process; the flight heights of the at least two unmanned aerial vehicles are above the crown and below the crown, air information is obtained when the unmanned aerial vehicles fly above the crown, and ground information is obtained when the unmanned aerial vehicles fly below the crown;
generating a fused three-dimensional survey map with the environmental survey data based on the at least two map data; performing cross validation based on the aerial information and the ground information, and fusing map data and survey data constructed by the aerial information and the ground information according to geographical position information and feature matching;
and replanning the flight path of the unmanned aerial vehicle according to the generated and fused three-dimensional survey map.
2. The method of claim 1, wherein the environmental survey data is forest environment survey data, and the three-dimensional survey map is a three-dimensional map showing information about trees and environment in a forest area.
3. The method of claim 1 or 2, further comprising: judging whether the map data acquired by each unmanned aerial vehicle and the generated and fused three-dimensional survey map have overlapped parts in real time based on the generated and fused three-dimensional survey map; and if the overlapped part exists, sending a command for changing the flight direction to the corresponding unmanned aerial vehicle.
4. The method of claim 3, wherein the determining of the overlap portion comprises: extracting a plurality of pieces of feature point information of map data acquired by each unmanned aerial vehicle, and judging that an overlapped part exists if a plurality of pieces of same feature point information exist in the map data and the generated and fused three-dimensional survey map; and/or comparing the relative position information of each unmanned aerial vehicle with the public position reference point, and if the relative position information is the same as the relative position information in the generated and fused three-dimensional survey map, judging that an overlapped part exists.
5. The method of claim 1, wherein the aerial information comprises at least one of tree altitude information, species information, geographic location information, crown breadth information, and crown layer point cloud density information; the ground information includes at least one of diameter information, roundness information, species information, density information, and geographical location information below the tree.
6. The method according to claim 1 or 2, wherein the step of generating a fused three-dimensional survey map with the environmental survey data based on at least two map data comprises: and fusing the at least two map data through the determined fusion base point, wherein the fusion base point refers to a reference point capable of unifying the at least two maps to the same coordinate system.
7. The method according to claim 6, characterized in that the reference points comprise absolute coordinate points and/or references possessing unique identifications.
8. The method of claim 6, wherein the fusing the at least two map data by the determined fusion base point comprises:
and determining the direction and mileage information of each unmanned aerial vehicle flying relative to the fusion base point, and fusing the at least two map data according to the direction and mileage information.
9. Method according to claim 1 or 2, characterized in that when the difference in flying height of the at least two drones is less than a first set threshold and the distance between the at least two drones is less than a second set threshold, an instruction to change the direction of flight is sent so that at least one drone changes the direction of flight.
10. Method according to claim 1 or 2, characterized in that the difference in the flying heights of said at least two drones is greater than a third set threshold, so that the ground plane coordinates of said at least two drones at the same moment are the same.
11. A method according to claim 1 or 2, wherein the map data comprises location calibration information and/or alarm information for the drone when the temperature and/or smoke data collected by the temperature and/or gas sensors of the drone exceeds a fourth set threshold.
12. The utility model provides a device of three-dimensional investigation map is found in cooperation of many unmanned aerial vehicles which characterized in that includes:
the path planning module is used for carrying out autonomous path planning on each unmanned aerial vehicle and finishing obstacle avoidance and autonomous navigation in the flight process;
the map data acquisition module is used for acquiring at least two pieces of map data with environment survey data, which are acquired by at least two unmanned aerial vehicles in the flight process; the flight heights of the at least two unmanned aerial vehicles are above the crown and below the crown, air information is obtained when the unmanned aerial vehicles fly above the crown, and ground information is obtained when the unmanned aerial vehicles fly below the crown;
the three-dimensional survey map fusion generation module is used for generating a fused three-dimensional survey map with the environmental survey data based on the at least two map data; performing cross validation based on the aerial information and the ground information, and fusing map data and survey data constructed by the aerial information and the ground information according to geographical position information and feature matching;
the path planning module is further used for replanning the flight path of the unmanned aerial vehicle according to the generated and fused three-dimensional survey map.
13. The device of claim 12, further comprising a three-dimensional survey map overlap determination module, configured to determine, in real time, whether there is an overlap between the map data collected by each drone and the generated and fused three-dimensional survey map based on the generated and fused three-dimensional survey map.
14. The apparatus of claim 12, further comprising an instruction control module configured to send an instruction of a flight direction to the drone.
15. The device of claim 12, further comprising an alarm module configured to issue an alarm when the temperature and/or smoke data collected by the temperature and/or gas sensor of the drone exceeds a fourth set threshold, the alarm including location calibration information and/or alarm information.
16. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-11.
17. A computer-readable storage medium having computer-executable instructions stored thereon, which, when executed by a computing device, are operable to perform the method of any of claims 1-11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810753172.4A CN109029422B (en) | 2018-07-10 | 2018-07-10 | Method and device for building three-dimensional survey map through cooperation of multiple unmanned aerial vehicles |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810753172.4A CN109029422B (en) | 2018-07-10 | 2018-07-10 | Method and device for building three-dimensional survey map through cooperation of multiple unmanned aerial vehicles |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109029422A CN109029422A (en) | 2018-12-18 |
| CN109029422B true CN109029422B (en) | 2021-03-05 |
Family
ID=64641942
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810753172.4A Active CN109029422B (en) | 2018-07-10 | 2018-07-10 | Method and device for building three-dimensional survey map through cooperation of multiple unmanned aerial vehicles |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109029422B (en) |
Families Citing this family (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020133415A1 (en) * | 2018-12-29 | 2020-07-02 | Beijing Didi Infinity Technology And Development Co., Ltd. | Systems and methods for constructing a high-definition map based on landmarks |
| CN111670339B (en) * | 2019-03-08 | 2024-01-26 | 深圳市大疆创新科技有限公司 | Techniques for collaborative mapping between unmanned aerial vehicles and ground vehicles |
| CN117310739A (en) | 2019-03-08 | 2023-12-29 | 深圳市大疆创新科技有限公司 | Technique for sharing drawing data between movable objects |
| US20200326203A1 (en) * | 2019-04-15 | 2020-10-15 | Qualcomm Incorporated | Real-world traffic model |
| CN110207691B (en) * | 2019-05-08 | 2021-01-15 | 南京航空航天大学 | Multi-unmanned vehicle collaborative navigation method based on data link ranging |
| CN110119147B (en) * | 2019-05-09 | 2022-07-08 | 深圳市速腾聚创科技有限公司 | Vehicle automatic driving method, device, computer equipment and storage medium |
| CN112148742A (en) * | 2019-06-28 | 2020-12-29 | Oppo广东移动通信有限公司 | Map updating method and device, terminal and storage medium |
| CN110415174B (en) * | 2019-07-31 | 2023-07-07 | 达闼科技（北京）有限公司 | Map fusion method, electronic device and storage medium |
| CN110825106B (en) * | 2019-10-22 | 2022-04-22 | 深圳市道通智能航空技术股份有限公司 | Obstacle avoidance method of aircraft, flight system and storage medium |
| CN113383283B (en) * | 2019-12-30 | 2024-06-18 | 深圳元戎启行科技有限公司 | Perceptual information processing method, apparatus, computer device, and storage medium |
| CN111208526B (en) * | 2020-01-17 | 2022-07-08 | 西北工业大学 | Multi-unmanned aerial vehicle cooperative positioning method based on laser radar and positioning vector matching |
| CN111551167B (en) * | 2020-02-10 | 2022-09-27 | 江苏盖亚环境科技股份有限公司 | Global navigation auxiliary method based on unmanned aerial vehicle shooting and semantic segmentation |
| CN112000130B (en) * | 2020-09-07 | 2023-04-25 | 哈尔滨工业大学 | Multi-machine collaborative high-precision map building positioning system of unmanned aerial vehicle |
| CN112268541B (en) * | 2020-10-16 | 2022-04-15 | 中国有色金属长沙勘察设计研究院有限公司 | Three-dimensional space detection method |
| CN112288634A (en) * | 2020-10-29 | 2021-01-29 | 江苏理工学院 | Splicing method and device for aerial images of multiple unmanned aerial vehicles |
| CN112949292B (en) * | 2021-01-21 | 2024-04-05 | 中国人民解放军61540部队 | Method, device, equipment and storage medium for processing return data of cluster unmanned aerial vehicle |
| CN113359700B (en) * | 2021-05-08 | 2022-12-20 | 安徽泗州拖拉机制造有限公司 | Intelligent operation system of unmanned tractor based on 5G |
| CN113362036A (en) * | 2021-06-24 | 2021-09-07 | 陕西地建土地工程技术研究院有限责任公司 | Land resource informatization management system and method based on Internet of things |
| CN114338332B (en) * | 2021-12-23 | 2023-05-05 | 深圳职业技术学院 | Efficient data transmission method in intelligent industrial Internet of things |
| CN114596362B (en) * | 2022-03-15 | 2023-03-14 | 云粒智慧科技有限公司 | High-point camera coordinate calculation method and device, electronic equipment and medium |
| CN115327568B (en) * | 2022-07-19 | 2023-10-20 | 哈尔滨工程大学 | PointNet network-based unmanned aerial vehicle cluster real-time target recognition method, system and map construction method |
| CN115683062B (en) * | 2023-01-04 | 2023-03-10 | 北京新兴科遥信息技术有限公司 | Territorial space planning detection analysis system |
| CN116205394B (en) * | 2023-05-05 | 2023-07-14 | 浙江茂源林业工程有限公司 | Forest resource investigation and monitoring method and system based on radio navigation |
| CN116824414B (en) * | 2023-08-29 | 2023-11-14 | 深圳市硕腾科技有限公司 | Method for rapidly deploying RTK (real time kinematic) by unmanned aerial vehicle |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102628690A (en) * | 2012-04-19 | 2012-08-08 | 清华大学 | Task collaborative visual navigation method of two unmanned aerial vehicles |
| CN106595659A (en) * | 2016-11-03 | 2017-04-26 | 南京航空航天大学 | Map merging method of unmanned aerial vehicle visual SLAM under city complex environment |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103941747B (en) * | 2014-03-31 | 2016-08-17 | 清华大学 | The control method of unmanned aerial vehicle group and system |
| ES2876449T3 (en) * | 2014-09-05 | 2021-11-12 | Sz Dji Technology Co Ltd | Multi-sensor environment mapping |
| CN105678754B (en) * | 2015-12-31 | 2018-08-07 | 西北工业大学 | A kind of unmanned plane real-time map method for reconstructing |
| CN105571588A (en) * | 2016-03-10 | 2016-05-11 | 赛度科技（北京）有限责任公司 | Method for building three-dimensional aerial airway map of unmanned aerial vehicle and displaying airway of three-dimensional aerial airway map |
| CN107504957B (en) * | 2017-07-12 | 2020-04-03 | 天津大学 | Method for rapidly constructing three-dimensional terrain model by using unmanned aerial vehicle multi-view camera shooting |
| CN107844129B (en) * | 2017-11-30 | 2021-03-23 | 北京大学深圳研究生院 | Multi-unmanned aerial vehicle collaborative search method and system based on path planning and information fusion |
-
2018
- 2018-07-10 CN CN201810753172.4A patent/CN109029422B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102628690A (en) * | 2012-04-19 | 2012-08-08 | 清华大学 | Task collaborative visual navigation method of two unmanned aerial vehicles |
| CN106595659A (en) * | 2016-11-03 | 2017-04-26 | 南京航空航天大学 | Map merging method of unmanned aerial vehicle visual SLAM under city complex environment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109029422A (en) | 2018-12-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109029422B (en) | Method and device for building three-dimensional survey map through cooperation of multiple unmanned aerial vehicles | |
| US20230359227A1 (en) | Method and System for Providing Route of Unmanned Air Vehicle | |
| US10962650B2 (en) | Polyhedral geofences | |
| CN110057373A (en) | For generating the method, apparatus and computer storage medium of fine semanteme map | |
| CN107783106B (en) | Data fusion method between unmanned aerial vehicle and barrier | |
| CN1945351B (en) | Robot navigation positioning system and navigation positioning method | |
| CN110609570A (en) | Autonomous obstacle avoidance inspection method based on unmanned aerial vehicle | |
| CN112506222A (en) | Unmanned aerial vehicle intelligent obstacle avoidance method and device | |
| CN107783545B (en) | Obstacle avoidance system of post-disaster rescue rotor unmanned aerial vehicle based on OODA (object oriented data acquisition) ring multi-sensor information fusion | |
| US20110285981A1 (en) | Sensor Element and System Comprising Wide Field-of-View 3-D Imaging LIDAR | |
| CN111624641A (en) | Explosion-proof type intelligent inspection robot for oil depot area | |
| KR20170111921A (en) | Method and system for controlling unmanned air vehicle | |
| CN109753084A (en) | A kind of unmanned device and method | |
| US10921825B2 (en) | System and method for perceptive navigation of automated vehicles | |
| EP3799618B1 (en) | Method of navigating a vehicle and system thereof | |
| CN106373433A (en) | Method and device for setting flight path of aircraft | |
| CN107783547A (en) | Post disaster relief rotor wing unmanned aerial vehicle obstacle avoidance system and method | |
| CN111796602A (en) | Plant protection unmanned aerial vehicle barrier is surveyed and early warning system | |
| WO2020087297A1 (en) | Unmanned aerial vehicle testing method and apparatus, and storage medium | |
| CN112985386B (en) | Automatic driving multi-source fusion positioning method, device, equipment and storage medium | |
| CN112034479A (en) | Positioning method and system applied to intelligent inspection unmanned aerial vehicle under coal mine | |
| CN107783119A (en) | Apply the Decision fusion method in obstacle avoidance system | |
| CN116755458A (en) | Unmanned aerial vehicle system of autonomous path planning and obstacle avoidance system | |
| CN110785355A (en) | Unmanned aerial vehicle testing method, device and storage medium | |
| CN116430901A (en) | Unmanned aerial vehicle return control method and system based on mobile parking apron |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |