CN113325870A - Three-dimensional visual coverage track planning method for unmanned aerial vehicle under photographic geometric constraint - Google Patents
Three-dimensional visual coverage track planning method for unmanned aerial vehicle under photographic geometric constraint Download PDFInfo
- Publication number
- CN113325870A CN113325870A CN202110627152.4A CN202110627152A CN113325870A CN 113325870 A CN113325870 A CN 113325870A CN 202110627152 A CN202110627152 A CN 202110627152A CN 113325870 A CN113325870 A CN 113325870A
- Authority
- CN
- China
- Prior art keywords
- planning
- aerial vehicle
- unmanned aerial
- coverage
- constraint
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000000007 visual effect Effects 0.000 title claims abstract description 28
- 238000000034 method Methods 0.000 title claims abstract description 26
- 230000000704 physical effect Effects 0.000 claims abstract description 6
- 230000008859 change Effects 0.000 claims abstract description 5
- 238000005457 optimization Methods 0.000 claims description 8
- 230000036314 physical performance Effects 0.000 claims description 2
- 238000012544 monitoring process Methods 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000002068 genetic effect Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The invention provides a three-dimensional visual coverage track planning method of an unmanned aerial vehicle under the constraint of shooting geometry, which comprises the following steps; firstly, designing a complete visual coverage track planning frame aiming at a three-dimensional visual coverage task; aiming at the problems of holder camera image deformation and resolution loss caused by natural terrain change, two new photogrammetric constraints are defined for solving viewpoints which must pass through in the air; and taking the flight path length and the flight time as performance indexes, and performing planning solution under the constraint of inherent physical properties of the quad-rotor unmanned aerial vehicle by using a two-step layered coverage planning algorithm to obtain a smooth path with optimal performance. The method has the characteristics of objectivity and strong universality in the aspect of planning the vision coverage track of the quad-rotor unmanned aerial vehicle, fully considers the inherent physical property constraint of the quad-rotor unmanned aerial vehicle, and effectively improves the vision coverage efficiency of the quad-rotor unmanned aerial vehicle.
Description
Technical Field
The invention belongs to the field of three-dimensional photogrammetry and aerial robot trajectory planning, and particularly relates to a three-dimensional visual coverage trajectory planning method for an unmanned aerial vehicle under the constraint of photography geometry.
Background
Currently, autonomous regional monitoring is an advanced technical approach involving many location-related tasks, such as ecological observation, environmental protection, emergency relief, etc. In open-air natural ecological environment, aerial robot can follow aerial visual angle to high accuracy, high speed, the motion form of independently planning, the main detection equipment of using the smart camera of loading gives play to advantages such as big visual angle, VTOL, fixed point hover, consequently, monocular four rotor unmanned aerial vehicle can be regarded as the unmanned platform of ideal, carries out effectual long-term regional monitoring.
In the existing solution, an unmanned aerial vehicle is usually flown with a vision sensor at a fixed height, and according to a preset flight mode, an area of interest is simply traversed once to realize coverage, and the method belongs to two-dimensional in-plane planning.
However, in some special task scenes, such as wide area, complex environment, and dynamic scenes, covering a city or field scene with large relief, simple two-dimensional in-plane planning cannot meet the task requirements, so the three-dimensional visual coverage planning method is more important, and the autonomous region monitoring using the quad-rotor unmanned aerial vehicle faces the following challenges:
first, battery life and payload capacity limitations make it difficult to achieve area coverage;
secondly, in order to obtain accurate reconstruction results of the three-dimensional geographic terrain, certain projection geometric constraints need to be set for the onboard camera.
Therefore, for the three-dimensional visual coverage of the unmanned aerial vehicle, a three-dimensional visual coverage trajectory planning method of the unmanned aerial vehicle under the constraint of the shooting geometry is urgently needed.
Disclosure of Invention
In order to solve the technical problem, the invention provides a three-dimensional visual coverage track planning method of an unmanned aerial vehicle under the constraint of shooting geometry, which comprises the following steps;
step 1, firstly, designing a complete visual coverage track planning frame aiming at a three-dimensional visual coverage task;
and 3, taking the flight path length and the flight time as performance indexes, performing planning solution under the constraint of inherent physical properties of the quad-rotor unmanned aerial vehicle by using a two-step layered coverage planning algorithm to obtain a smooth path with optimal performance, and acquiring the color image shot at each viewpoint.
Preferably, the visual coverage trajectory planning frame in step 1 is to select an experimental area to be monitored, divide the terrain surface into a plurality of grids under the condition that the requirement of the image overlapping rate is met, and obtain the geometric center of each photographic grid on the terrain surface.
Preferably, the first layer of the two-step hierarchical coverage planning algorithm in step 3 is to optimize the path length by using a heuristic algorithm to obtain a broken-line traversal path sequence.
Preferably, the second layer of the two-step hierarchical coverage planning algorithm in the step 3 is to obtain a smooth track with a timestamp by further optimizing by taking flight time as a performance index according to the traversal path sequence obtained by the first layer and combining with the inherent physical performance constraint of the quad-rotor unmanned aerial vehicle, and sacrificing a small part of the track length optimization rate to obtain the flight time optimization rate and meet the smoothness constraint.
Compared with the prior art, the invention has the beneficial effects that:
1. aiming at the three-dimensional visual coverage task, the invention designs a complete visual coverage track planning frame, solves the smooth track planning from the viewpoint to each viewpoint, and has universality in the processing flow.
2. According to the invention, two new photogrammetry constraints are defined when the viewpoint is solved, and the problems of image deformation and resolution loss of the pan-tilt camera caused by natural terrain change can be effectively solved.
3. According to the invention, under the constraint of inherent physical properties of the quad-rotor unmanned aerial vehicle, the flight path length and the flight time are used as performance indexes, and a smooth path with optimal performance can be obtained by innovatively using a two-step layered coverage planning algorithm for planning and solving.
Drawings
FIG. 1 is a schematic diagram of the effects of the visual overlay trajectory planning framework of the present invention;
FIG. 2 is a cross-sectional schematic view of the camera geometry constraints of the present invention;
FIG. 3 is a schematic view of the course overlap ratio requirement of each photographing region according to the present invention;
FIG. 4 is a schematic diagram of the lateral overlap ratio requirements of various imaging regions of the present invention;
FIG. 5 is a graph comparing performance of planning results for an embodiment of the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings:
example (b):
a three-dimensional visual coverage track planning experiment based on shooting geometric constraint is carried out on a certain mountain region by utilizing a quad-rotor unmanned aerial vehicle.
As shown in fig. 1, the method comprises the following steps:
step 1, selecting an experimental area to be monitored, dividing a terrain surface into a plurality of grids under the condition of meeting the requirement of image overlapping rate, and solving the geometric center of each photographic grid on the terrain surface;
vPi＝[xi,yi,zi],(i＝1,2,…,N)。
where N represents the number of in-air views.
xij∈{0，1}i，j＝1，2，...，N
wherein L represents the total traversal path length, L0kRepresenting the path length, L, between the ground origin and the first aerial viewpoint k to be traversedswRepresenting the path length, L, between the last aerial viewpoint s to be traversed and the ground end pointijRepresents the path length between the ith and jth aerial viewpoints, XijRepresents a binary selection variable when XijWhen the value is 1, the representation flies to the jth air viewpoint directly after the ith air viewpoint in the traversal order.
Considering that a heuristic algorithm such as a genetic algorithm cannot obtain a global optimal solution at one time, selecting an optimal one of local optimal solutions obtained by multiple times of solution as a final solution To represent the global optimal solution To obtain a broken-line type traversal path sequence, and if smoothness and continuity constraints are not considered, using a traditional Stop-To-Go method as a broken-line type flight trajectory for the layer of results.
And 4, using a second layer in the two-step hierarchical coverage planning algorithm, wherein the optimization problem is defined as:
s.t.||V(t)||≤vm,
||A(t)||≤am,
where T represents the total time of flight, Δ TjRepresents the time of flight of the j-th segment, V (t) represents the velocity vector, A (t) represents the acceleration vector, v (t) represents the velocity vectormRepresenting the maximum allowable flight speed, amRepresenting the maximum allowable flight acceleration, tjRepresenting the corresponding time stamp at any point in the space in the defined domain.
According to the sequence of the traversal path obtained by the first layer, the inherent physical property constraint of the quad-rotor unmanned aerial vehicle is combined, the flight time is further optimized by taking the flight time as a performance index, a small part of the optimization rate of the track length is sacrificed to obtain the optimization rate of the flight time and satisfy the constraints of smoothness and the like, and a smooth track with a timestamp is obtained.
And 5, carrying a video sensor by using a real unmanned aerial vehicle, and carrying out two-step layered coverage planning algorithm planning on the spot to obtain a smooth track so as to obtain a color image shot at each viewpoint.
And 6, reconstructing the color image by using a three-dimensional reconstruction algorithm to obtain a reconstruction result.
Specifically, the two-step hierarchical coverage planning algorithm provided by the invention can be respectively superimposed on the traditional method To better compare the planning results, as shown in fig. 5, the method is a traditional zigzag traversal and Stop-To-Go planning method, the method is a planning method Smooth BF in which the second layer is added To the traditional zigzag traversal, the method is a planning method GACPP + S & G in which the Stop-To-Go is added To the first layer, and the method is a planning method HO-CTP provided by the invention, and the method provided by the invention is superior To the traditional method in view of the performance data of fig. 5.
In the description of the present invention, it is to be understood that the terms "coaxial", "bottom", "one end", "top", "middle", "other end", "upper", "one side", "top", "inner", "front", "center", "both ends", and the like, indicate orientations or positional relationships based on those shown in the drawings, and are only for convenience of description and simplicity of description, and do not indicate or imply that the referenced device or element must have a particular orientation, be constructed and operated in a particular orientation, and thus, are not to be construed as limiting the present invention.
Furthermore, the terms "first", "second", "third", "fourth" are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated, whereby the features defined as "first", "second", "third", "fourth" may explicitly or implicitly include at least one such feature.
In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "disposed," "connected," "secured," "screwed" and the like are to be construed broadly, e.g., as meaning fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; the terms may be directly connected or indirectly connected through an intermediate, and may be communication between two elements or interaction relationship between two elements, unless otherwise specifically limited, and the specific meaning of the terms in the present invention will be understood by those skilled in the art according to specific situations.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (4)
1. A three-dimensional visual coverage track planning method of an unmanned aerial vehicle under the constraint of shooting geometry is characterized by comprising the following steps;
step 1, firstly, designing a complete visual coverage track planning frame aiming at a three-dimensional visual coverage task;
step 2, in the visual coverage track planning frame in the step 1, aiming at the problems of holder camera image deformation and resolution loss caused by natural terrain change, defining two new photogrammetry constraints for solving viewpoints which must pass through in the air;
and 3, taking the flight path length and the flight time as performance indexes, performing planning solution under the constraint of inherent physical properties of the quad-rotor unmanned aerial vehicle by using a two-step layered coverage planning algorithm to obtain a smooth path with optimal performance, and acquiring the color image shot at each viewpoint.
2. The method for planning the three-dimensional visual coverage locus of the unmanned aerial vehicle under the geometric constraint of photography according to claim 1, wherein the visual coverage locus planning framework in the step 1 is to select an experimental area to be monitored, divide the terrain surface into a plurality of grids under the condition of meeting the requirement of image overlapping rate, and obtain the geometric center of each photography grid on the terrain surface.
3. The method for planning the three-dimensional visual coverage locus of the unmanned aerial vehicle under the geometric constraint of photography as claimed in claim 1, wherein the first layer of the two-step layered coverage planning algorithm in the step 3 is to optimize the path length by using a heuristic algorithm to obtain a broken-line traversal path sequence.
4. The method for planning the three-dimensional visual coverage trajectory of the unmanned aerial vehicle under the geometric constraint of photography as claimed in claim 1, wherein the second layer of the two-step hierarchical coverage planning algorithm in the step 3 is to further optimize by taking flight time as a performance index according to the sequence of the traversal path obtained by the first layer and combining the inherent physical performance constraint of the quad-rotor unmanned aerial vehicle, sacrifice a small part of the trajectory length optimization rate to exchange the flight time optimization rate, and satisfy the smoothness constraint to obtain a smooth trajectory with a time stamp.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110627152.4A CN113325870A (en) | 2021-06-04 | 2021-06-04 | Three-dimensional visual coverage track planning method for unmanned aerial vehicle under photographic geometric constraint |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110627152.4A CN113325870A (en) | 2021-06-04 | 2021-06-04 | Three-dimensional visual coverage track planning method for unmanned aerial vehicle under photographic geometric constraint |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113325870A true CN113325870A (en) | 2021-08-31 |
Family
ID=77419814
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110627152.4A Pending CN113325870A (en) | 2021-06-04 | 2021-06-04 | Three-dimensional visual coverage track planning method for unmanned aerial vehicle under photographic geometric constraint |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113325870A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150314443A1 (en) * | 2014-04-30 | 2015-11-05 | Shenzhen Mercury Optoelectronics Research Institute | Visual-based obstacle detection method and apparatus for mobile robot |
| CN109976164A (en) * | 2019-04-25 | 2019-07-05 | 南开大学 | A kind of energy-optimised vision covering method for planning track of multi-rotor unmanned aerial vehicle |
| CN110675483A (en) * | 2019-07-17 | 2020-01-10 | 电子科技大学 | Dense vision SLAM-based rapid reconstruction method for three-dimensional map of unmanned aerial vehicle |
-
2021
- 2021-06-04 CN CN202110627152.4A patent/CN113325870A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150314443A1 (en) * | 2014-04-30 | 2015-11-05 | Shenzhen Mercury Optoelectronics Research Institute | Visual-based obstacle detection method and apparatus for mobile robot |
| CN109976164A (en) * | 2019-04-25 | 2019-07-05 | 南开大学 | A kind of energy-optimised vision covering method for planning track of multi-rotor unmanned aerial vehicle |
| CN110675483A (en) * | 2019-07-17 | 2020-01-10 | 电子科技大学 | Dense vision SLAM-based rapid reconstruction method for three-dimensional map of unmanned aerial vehicle |
Non-Patent Citations (1)
| Title |
|---|
| HONGPENG WANG: "Near-Optimal 3-D Visual Coverage for Quadrotor Unmanned Aerial Vehicles Under Photogrammetric Constraints", 《IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11086324B2 (en) | Structure from motion (SfM) processing for unmanned aerial vehicle (UAV) | |
| Hoppe et al. | Photogrammetric camera network design for micro aerial vehicles | |
| CN107194989B (en) | Traffic accident scene three-dimensional reconstruction system and method based on unmanned aerial vehicle aircraft aerial photography | |
| CN105203084B (en) | A kind of unmanned plane 3D panoramic vision devices | |
| JP4854819B2 (en) | Image information output method | |
| WO2018103408A1 (en) | Aerial image capturing method and system for unmanned aerial vehicle to survey traffic accident scene | |
| CN107514993A (en) | The collecting method and system towards single building modeling based on unmanned plane | |
| CN110243381B (en) | Cooperative sensing monitoring method for air-ground robot | |
| Zhang et al. | Continuous aerial path planning for 3D urban scene reconstruction. | |
| CN114066960B (en) | Three-dimensional reconstruction method, point cloud fusion method, device, equipment and storage medium | |
| JP2009217524A (en) | System for generating and browsing three-dimensional moving image of city view | |
| CN110706273B (en) | Real-time collapse area measurement method based on unmanned aerial vehicle | |
| Skarlatos et al. | Accuracy assessment of minimum control points for UAV photography and georeferencing | |
| CN115657706B (en) | Landform measurement method and system based on unmanned aerial vehicle | |
| Wischounig-Strucl et al. | Resource aware and incremental mosaics of wide areas from small-scale UAVs | |
| JP4418857B1 (en) | Image acquisition system for generating 3D video of routes | |
| CN114077249B (en) | Operation method, operation equipment, device and storage medium | |
| Majdik et al. | Micro air vehicle localization and position tracking from textured 3d cadastral models | |
| CN112785686A (en) | Forest map construction method based on big data and readable storage medium | |
| Darrah et al. | Optimized 3D mapping of a large area with structures using multiple multirotors | |
| CN113325870A (en) | Three-dimensional visual coverage track planning method for unmanned aerial vehicle under photographic geometric constraint | |
| CN112700530B (en) | Unmanned aerial vehicle rotary multi-angle railway existing line high-precision three-dimensional reconstruction route planning method | |
| Yamauchi et al. | Verification of model accuracy and photo shooting efficiency of large-scale SfM for flight path calculation | |
| Sharma et al. | Image Acquisition for High Quality Architectural Reconstruction. | |
| Dixit et al. | Evaluation of Cross-View Matching to Improve Ground Vehicle Localization with Aerial Perception |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210831 |