CN115661204B

CN115661204B - Collaborative searching and tracking positioning method for moving target by unmanned aerial vehicle cluster

Info

Publication number: CN115661204B
Application number: CN202211358543.1A
Authority: CN
Inventors: 聂一鸣; 孔凡杰; 葛超; 连政; 徐孝煜; 黄昊
Original assignee: National Defense Technology Innovation Institute PLA Academy of Military Science
Current assignee: National Defense Technology Innovation Institute PLA Academy of Military Science
Priority date: 2022-11-01
Filing date: 2022-11-01
Publication date: 2023-11-10
Anticipated expiration: 2042-11-01
Also published as: CN115661204A

Abstract

The invention provides a collaborative searching and tracking positioning method of an unmanned aerial vehicle cluster on multiple moving targets, and belongs to the technical field of unmanned aerial vehicle collaborative and tracking control. The invention pre-plans the cruising path of each unmanned plane; the unmanned aerial vehicle performs multi-target searching and detecting by using a monocular camera; the unmanned aerial vehicle adopts a multi-stage tracking strategy to intelligently track the target; in the tracking process, the unmanned aerial vehicle performs target position measurement, calculation, optimization and release sharing; after the single target tracking is completed, the unmanned aerial vehicle automatically plans a return path and continues to search the residual targets; and in the whole course of the task, the unmanned aerial vehicle autonomously carries out real-time obstacle avoidance between the static obstacle and the unmanned aerial vehicle. The invention solves the repeated tracking problem of a plurality of unmanned aerial vehicles on the same target and the stable continuous tracking problem of the short-term loss of the target, and effectively improves the overall efficiency of collaborative searching and tracking positioning of a plurality of moving targets.

Description

Collaborative searching and tracking positioning method for moving target by unmanned aerial vehicle cluster

Technical Field

The invention belongs to the technical field of unmanned aerial vehicle coordination and tracking control, and particularly relates to a coordination searching and tracking positioning method for a moving target by an unmanned aerial vehicle cluster.

Background

A drone is a type of unmanned aerial vehicle that may be used multiple times and perform certain tasks by carrying sensors or other loads. Due to the characteristics of small weight, low cost, strong autonomy, no risk of casualties and the like, the unmanned aerial vehicle has wide application in aspects of disaster relief and rescue, aerial photography, real-time detection, remote striking, post-disaster rescue, geological investigation and the like. Unmanned aerial vehicles perform target searching, detection tracking and positioning tasks in a certain area, and are always an important means for acquiring information in military and civil fields. In recent years, with the rapid development of technologies such as artificial intelligence, sensors, signal processing, image recognition, mobile communication and the like, an unmanned aerial vehicle cluster cooperation scheme becomes an important means for completing target cooperation searching and tracking positioning tasks.

Under a complex scene, a single unmanned aerial vehicle often faces the problems of large search range, more tracking objects and the like, so that tasks such as target searching, tracking and positioning cannot be completed quickly. The unmanned aerial vehicle clusters can achieve higher efficiency through mutual cooperation, and compared with the single unmanned aerial vehicle when executing collaborative searching and tracking positioning tasks, the single unmanned aerial vehicle has the following advantages: (1) The task area is expanded, and a larger area can be searched by utilizing cooperation; (2) Robustness is increased, and the situation that tasks fail due to single unmanned aerial vehicle faults is avoided; (3) The task execution efficiency is improved, the unmanned aerial vehicle clusters execute tasks in parallel, and the search time is reduced. However, when the unmanned aerial vehicle cluster executes the collaborative search task, improving the rationality of task allocation, global information sharing, guaranteeing the safety of the unmanned aerial vehicle cluster, mutual interference of multiple unmanned aerial vehicles and the like are still key problems to be solved urgently. The unmanned aerial vehicle cluster collaborative search, tracking and positioning tasks can be divided into three parts: (1) task area coverage search: planning a search path of each unmanned aerial vehicle, and realizing intelligent search of the unmanned aerial vehicle cluster on a task area; (2) real-time target detection: the unmanned aerial vehicle detects targets in the search area in real time by using an airborne sensor; (3) target tracking and localization: the unmanned aerial vehicle can track the target intelligently and calculate the position of the target accurately.

Unmanned aerial vehicle cluster collaborative search and tracking positioning need to solve the following problems: (1) Selecting a proper task planning method to adapt to various complex scenes; (2) How to plan a proper cruising path by collaborative searching, and quickly searching, so as to avoid low searching efficiency; (3) The stable tracking method is designed, so that the obstacle can be avoided and the tracking process is not hindered.

The task planning method of the unmanned aerial vehicle cluster is mainly divided into two types, namely centralized type and distributed type. The centralized type is that a central control node performs task planning on the whole unmanned aerial vehicle cluster, so that the efficiency is high but the robustness is poor; the distributed unmanned aerial vehicle has no central control node, can independently search and detect, can also perform local information interaction and decision to achieve the aim of completing a search task, has stronger robustness and expandability, and has higher requirements on algorithms. The existing unmanned aerial vehicle cluster collaborative search problem can be mainly divided into a static programming method and a dynamic programming method at home and abroad. In the aspect of static planning, the method mainly comprises the area processing and path planning methods such as Voronoi diagram, graham's method, dubins path planning and the like. The operation adopts the divide-and-conquer idea, divides the search area, determines the optimal search path of the unmanned aerial vehicle in each sub-area, solves the problem of low cooperative efficiency of unmanned aerial vehicle clusters, but is difficult to face the emergency situations of unmanned aerial vehicle faults, incapability of taking off and the like. The other dynamic planning method is mainly to construct a pheromone map, a target probability map and other maps, and then to conduct real-time dynamic decision by adopting optimization methods such as distributed predictive control, rolling time domain optimization and the like. Unmanned aerial vehicle cluster cooperative target tracking is mainly divided into two modes of standby off tracking and persistence tracking. The stand off tracking needs to control the unmanned aerial vehicle to keep a specific distance from the target, and also needs to keep a relatively proper observation angle between the unmanned aerial vehicle and the target, so that the unmanned aerial vehicle usually performs circular motion around the target, and the more mature method comprises the methods of an optimal guidance parameter searching algorithm, a roll angle, a speed control law and the like. While performing a persistence tracking task, it is often manifested as performing object tracking in a particular formation. And a proper unmanned aerial vehicle formation tracking method is designed, and formation tracking control on the target is realized through mutual cooperation among unmanned aerial vehicles.

Disclosure of Invention

Therefore, aiming at the problem of low efficiency of multi-unmanned aerial vehicle collaborative search and target tracking positioning in a complex urban environment, the invention provides a multi-unmanned aerial vehicle collaborative rapid search and tracking positioning method with self-adaptive strategy switching. According to the method, through a real-time high-precision target detection algorithm, a dynamic obstacle avoidance strategy and a quick coordinate measuring and calculating method, the technical problems that a single target is repeatedly tracked, multiple unmanned aerial vehicles are prevented from being bumped and targets are blocked for a short time under the condition of multiple obstacles can be effectively solved, and the quick search, stable tracking and accurate positioning of the known multiple moving targets in a task area by the unmanned aerial vehicle cluster are realized.

In order to achieve the above object, the present invention provides the following technical solutions:

a collaborative searching and tracking positioning method of unmanned aerial vehicle clusters to multiple moving targets is shown in fig. 1, and comprises the following steps:

step S1, a gridding environment information and an unmanned aerial vehicle cruising model are constructed, as shown in FIG. 6;

step S2, performing target detection on a search area by using an airborne monocular camera based on a deep learning method;

s21, training a target detection algorithm network model;

firstly, N task area images containing targets are collected through an unmanned plane monocular camera, and targets expected to be tracked in the images are marked in a manual marking mode to generate corresponding multi-category labels;

Taking the image and the labeled target label data as a training set, training a target detection algorithm depth target detection model, and completing training when one of the following conditions is met:

condition 1: the accuracy of the target detection algorithm model is not improved any more;

condition 2: the training times reach the preset maximum set times E _max ；

Secondly, collecting M task areas containing targetsInputting the domain image into a target detection algorithm model with complete training to generate a new target label, manually checking and correcting the detection result to finally obtain a target label set L of the region to be searched _O The method comprises the steps of carrying out a first treatment on the surface of the Wherein M is much greater than N;

m images and tag set L _O Inputting the target detection algorithm network model for training, and ending training when one of the following conditions is met:

condition 2, training times reaching a predetermined maximum set time E' _max ；

S22, acquiring an image of a region to be searched in real time;

cruising search is carried out on the unmanned aerial vehicle after taking off according to the path obtained in the step S1, RGB image data of a search area are collected in real time through an onboard monocular camera, and the image data collected by different unmanned aerial vehicles at the same time t are integrated into an image set P _t ={p ₁ ,p ₂ ,…,p _n P, where _i Searching area images acquired at t moment of the ith unmanned aerial vehicle;

s23, performing target detection on the monocular camera image by using a target detection algorithm model;

utilizing the S21 pre-trained multi-classification target detection algorithm model to real-time acquire the image set P obtained in the step S22 _t Image detection is carried out to obtain a target information set U of each image _O Which includes the length and width (H, L) of each target bounding box, the center point coordinates (x _o ,y _o ) Confidence CONF _O And class number cls;

step S3, after the unmanned aerial vehicle detects the target, the unmanned aerial vehicle autonomously selects different tracking strategies to intelligently track the target;

s31, when the unmanned aerial vehicle detects the target, judging whether the newly found target is tracked by other unmanned aerial vehicles, and eliminating the tracked target

The unmanned aerial vehicle flies along the cruising path, and a target detection algorithm is utilized to detect a target of a real-time image of the monocular camera in the cruising process; meanwhile, the unmanned aerial vehicle acquires and stores the current tracking and releasing target positions of other unmanned aerial vehicles in real time, carries out multi-frame clustering according to the acquired target position information released by other unmanned aerial vehicles, and analyzes to obtain the tracking and discovering conditions of the current targets;

s32, traversing U 'by unmanned aerial vehicle' _O,n According to the distance between the detected target and the unmanned plane, selecting the nearest target, and judging to enter a long-distance tracking or short-distance tracking program:

taking the size of a bounding box of a target detection algorithm as a judgment basis of a far target and a near target, wherein h is _i And l _i Respectively, currently detects U' _O,n The height and width of the ith target bounding box;

h _i <T _h or l _i <T _l Or y _min,i <T _y Step S33 is entered, otherwise step S34 is entered;

wherein T is _h 、T _l 、T _y Numerical selection is set according to the imaging resolution of the monocular camera used, y _min,i Is U' _O,n The y value of the pixel point at the upper left corner of the ith bounding box;

s33, when the unmanned aerial vehicle judges that the target position is far away from the unmanned aerial vehicle, entering a remote tracking program;

when unmanned aerial vehicle is tracked remotely, cloud platform angle keeps T _g1 The unmanned aerial vehicle adjusts the advancing speed and the steering speed of the unmanned aerial vehicle according to the position of the target bounding box to be tracked in the imaging of the monocular camera;

first, a relative distance (u) of the target bounding box from the center point imaged by the monocular camera is calculated _x ,u _y )：

u _x ＝x _o -u _cx

u _y ＝y _o -u _cy

Wherein (u) _cx ,u _cy ) The position of the center point for imaging the monocular camera is determined according to y _o Setting the forward speed v of the unmanned aerial vehicle in a pixel distance segmentation way from the imaging center point of the monocular camera _x Is of the size of (2):

v _x ＝k _fi ×y _o +b _fi

wherein k is _fi 、b _fi Parameters are adjusted for different speeds during remote tracking, i representing y at this time _o An ith section of pixel distance from the monocular camera imaging center point;

rotational speed w of unmanned aerial vehicle _z The size of (2) is:

wherein k is _f Yaw rate w for unmanned aerial vehicle _z Wherein the left turn is positive and the right turn is negative with the viewing angle of the unmanned aerial vehicle;

s34, when the unmanned aerial vehicle judges that the target position is close to the unmanned aerial vehicle, entering close-range tracking;

unmanned aerial vehicle closely tracks, and cloud platform angle adjustment is T _g2 Wherein T is _g2 Greater than T _g1 The unmanned aerial vehicle adjusts the advancing speed and the steering speed of the unmanned aerial vehicle according to the position of the target bounding box to be tracked in the imaging of the monocular camera;

calculating the relative distance (u) of the object bounding box from the center point of monocular camera imaging _x ,u _y )：

u _x ＝x _o -u _cx

u _y ＝y _o -u _cy

Wherein (u) _cx ,u _cy ) Center point position for monocular camera imaging, speed of advance of unmanned aerial vehicle v _x According to the size y _o And u is equal to _cy Ratio u of (2) _tc Segment setting is performed, wherein forward is positive and backward is negative:

v _x ＝k _ni ×y _o +b _ni

wherein k is _ni 、b _ni Parameters are adjusted for different speeds during close-range tracking, i representing thisTime y _o An ith section of pixel distance from the monocular camera imaging center point;

rotational speed w of unmanned aerial vehicle _z The size of (2) is:

wherein k is _n For unmanned aerial vehicle short distance yaw rate w _z Wherein the left turn is positive and the right turn is negative with the viewing angle of the unmanned aerial vehicle;

S35, in the unmanned aerial vehicle tracking process, after a target is temporarily lost due to shielding, the unmanned aerial vehicle pre-judges and tracks the target motion trail based on the motion state before the target is lost;

step S4, measuring and calculating the position of the target while tracking, and sharing the target position in real time

According to the perspective transformation principle, mapping the position of the target on the camera image into an unmanned aerial vehicle body coordinate system to obtain the relative position with the unmanned aerial vehicle, converting the position of the target into the world coordinate system through the world coordinate of the unmanned aerial vehicle, so as to obtain the coordinate of the target in the world coordinate system, and releasing and sharing the category and the coordinate of the target into a global network by the unmanned aerial vehicle;

s41, eliminating the wrong target position;

the target detection network has good detection effect and can continuously detect the appointed target; when an sporadic detection result appears, namely, no target is detected in continuous frames, the detection is judged to be error detection, and the tracking program does not respond to the detection result;

the current unmanned aerial vehicle records the coordinates of each category target detection frame in the nearest n frames on the image; frame number n when a certain class of object is detected _d If the number is less than n, the target is not processed, and the sporadic detection result is used as the detection of possible errors to be eliminated;

S42, automatically correcting the target class errors in target detection by utilizing the front and rear frame information;

in unmanned aerial vehicle close range tracking, the class of the tracked object is locked to be cls _tr The method comprises the steps of carrying out a first treatment on the surface of the At t _s In the time range of seconds, when the target class is C _tr Is detected to disappear and other target class C is detected _e When comparing object class C _tr Coordinates of last moment (x _t ,y _t ) Coordinates (x) of other object classes currently detected _e ,y _e ) If the distance d between them _err ＜d _tr Then the target class C _e Modified to C _tr ；

Wherein,d _tr judging a threshold value with similar distance between two coordinates for the image coordinate system;

s43, smoothing target detection result by using multi-frame sequential information

When a target is detected in continuous frames, a multi-frame smoothing technology is utilized to exclude a detection result with larger error, the target detection positions of the multi-frames are fused, and an average value of target coordinates is calculated;

after the false detection is eliminated, the average coordinate value (x _m ,y _m )：

Wherein x is _o,i 、y _o,i Respectively an x-axis coordinate and a y-axis coordinate of a target detection frame in an ith frame;

s44, calculating the position of the target in the world coordinate system and publishing the position to the global network

Under the camera image coordinate system and the unmanned plane body coordinate system, n and n >4 are found, the corresponding coordinates above are brought into the following perspective transformation formula, and the homography matrix Trans of the perspective transformation is calculated:

[x _u y _u 1]＝[x _m y _m 1]·Trans

Wherein [ x ] _u y _u 1]Representing the coordinates of a point under the unmanned aerial vehicle body coordinate system, [ x ] _m y _m 1]Representing coordinates of points under a camera image coordinate system;

obtaining coordinates of points corresponding to the unmanned aerial vehicle body coordinate system under the image coordinate system through perspective transformation homography matrix Trans, and obtaining the relative position of the target and the unmanned aerial vehicle;

and then, converting the relative positions of the target and the unmanned aerial vehicle into absolute positions in a world coordinate system through a transformation matrix and the pose of the unmanned aerial vehicle:

[x _w y _w 1 1]＝[x _u y _u 1 1]·Tr _{w_uav}

wherein,yaw angle, x of unmanned aerial vehicle _uav And y _uav The coordinates of the unmanned aerial vehicle in a world coordinate system;

object class cls to be tracked and coordinates x of the object in the world coordinate system _w ,y _w Publishing to a global network;

s5, planning a path returning to the cruising point by using a dynamic path planning algorithm;

after the unmanned aerial vehicle finishes tracking the target, a collision-free path is planned by adopting dynamic path planning rules by taking the current position as a starting point and taking a cruising path point closest to the unmanned aerial vehicle as an end point;

after reaching the end point, returning to the step S2, and continuing searching and detecting until the searching, tracking and positioning tasks of all the moving targets are completed;

and S6, in the whole process of the search task, the unmanned aerial vehicle in the steps S1-S5 needs to carry out obstacle avoidance treatment on the static obstacle and other unmanned aerial vehicles.

Compared with the prior art, the invention has the advantages and positive effects that:

1. according to the invention, the cooperative tracking mode of the unmanned aerial vehicle clusters on the targets is comprehensively considered and optimized, different targets in the same class can be autonomously identified, different unmanned aerial vehicles are automatically allocated to track the different targets, and the problem of repeated tracking of a single target is solved;

2. the invention provides a static obstacle avoidance algorithm and a dynamic obstacle avoidance algorithm, solves the problem of collision avoidance of multiple unmanned aerial vehicles under the condition of multiple obstacles, and prevents the impact of collision on an overall task;

3. according to the invention, the target prior motion state is utilized to carry out fitting calculation on the target motion track, so that the problem of short-time shielding loss of the target is solved, and the tracking continuity and stability are improved;

4. in the invention, the tracking of the moving target is split into a long-distance tracking mode and a short-distance tracking mode, and different camera visual field ranges and unmanned aerial vehicle tracking strategies are adaptively switched in different modes, so that the efficiency of tracking the moving target is effectively improved.

Drawings

The foregoing and/or additional aspects and advantages of the invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:

fig. 1 is a flowchart of a method for collaborative searching and tracking positioning of a moving object by a cluster of unmanned aerial vehicles according to an embodiment of the present invention;

FIG. 2 is a block diagram of a tracking strategy for a drone to multiple moving targets according to one embodiment of the invention;

fig. 3 is a flowchart of a remote tracking strategy of a moving object by a drone according to one embodiment of the present invention;

FIG. 4 is a flowchart of a close-range tracking strategy of a drone to a moving target, according to one embodiment of the present invention;

FIG. 5 is a block diagram of a process for locating and publishing a moving object by a drone according to one embodiment of the present invention;

FIG. 6 is a schematic diagram of a simulation test environment rasterized modeling in accordance with an embodiment of the invention;

fig. 7 is a schematic diagram illustrating dynamic obstacle avoidance among unmanned aerial vehicle clusters according to an embodiment of the present invention;

FIG. 8 is a schematic view of an obstacle avoidance of a static obstacle by a drone according to one embodiment of the present invention;

fig. 9 is a schematic diagram of a speed change of a drone tracking process according to one embodiment of the present invention.

Detailed Description

In order to make the technical scheme and advantages of the present invention more clear, the technical scheme in the embodiment of the present invention is clearly and completely described below with reference to the accompanying drawings in the embodiment of the present invention:

the embodiment of the invention relates to a collaborative searching and tracking positioning method for a moving target by an unmanned aerial vehicle cluster, which comprises the following steps.

A collaborative searching and tracking positioning method of unmanned aerial vehicle clusters to multiple moving targets comprises the following steps:

step S1, a gridding environment information and an unmanned aerial vehicle cruising model are built according to specific tasks;

the person skilled in the art can choose to construct the gridding environment information and the unmanned aerial vehicle cruising model according to specific task requirements, and a construction mode is provided in the embodiment.

Further, the implementation process of step S1 is as follows:

s11, acquiring map information of a search task area according to a search task requirement, and rasterizing environment information of the task area;

explicitly searching task space boundary range { O _x ,O _y ,O _z }, wherein O _x ,O _y Refers to the boundary range of the XY axis plane, O _z Refers to the range of elevation boundaries;

rasterizing modeling is carried out on an XY axis plane of a search task space, and the area size of a single grid is set to be S ² The number of grids q=o _x ×O _y /S ² ；

Classifying different grids according to actual obstacle informationAggregate phi for obstacle grid _obs And a trafficable grid set Φ _ava ；

S12, searching a cruising path in a passable area;

setting a search condition set U _cond ＝{C ₁ ,C ₂ ,C ₃ }, wherein C ₁ ：Uav _i Refers to a grid covered by a monocular camera view of an ith unmanned aerial vehicle, C ₁ Meaning that the total grid covered by the monocular camera vision of each unmanned aerial vehicle is equal to the trafficable grid set phi _ava ；

C ₂ ：Where i.noteq.j, uav _i,t Refers to a grid covered by the field of view of the ith unmanned aerial vehicle at the moment t, C ₂ Meaning that each unmanned aerial vehicle cannot search for repeated grids at the same time t;

C ₃ ：Z _uav ＝H _cru ,0＜H _cru ＜O _z ，C ₃ finger unmanned aerial vehicle keeps high H _cru ；

To search for condition sets U _cond Based on, in the trafficable grid set phi _ava Planning grid areas which are required to be searched for and covered by each unmanned aerial vehicle, and connecting the centers of the planned areas to form an area search cruising path of each unmanned aerial vehicle;

s13, after the cruising path is determined, the unmanned aerial vehicle takes off and flies along the cruising path at the speed v and the height H _cru Cruising.

a person skilled in the art can select a suitable target detection algorithm according to task requirements to perform target detection in the step 2, wherein the target detection algorithm is YOLOv4 or YOLOv7, and can achieve a good detection effect, and the performance of YOLOv7 is most prominent, and the embodiment is described by taking YOLOv7 as an example.

S21, training a YOLOv7 network model;

Taking the image and the labeled target label data as a training set, training the Yolov7 depth target detection model, and completing training when one of the following conditions is met:

condition 1: the accuracy of the YOLOv7 model is not improved any more;

condition 2: the training times reach the preset maximum set times E _max ；

Secondly, collecting M task area images containing targets, inputting a training-completed YOLOv7 model to generate new target labels, manually checking and correcting detection results to finally obtain a target label set L of the area to be searched _O The method comprises the steps of carrying out a first treatment on the surface of the Wherein M is much greater than N;

m images and tag set L _O Inputting the YOLOv7 network model for training, and ending training when one of the following conditions is met:

condition 1: the accuracy of the YOLOv7 model is not improved any more;

S22, acquiring an image of a region to be searched in real time;

cruising search is carried out on the unmanned aerial vehicle after taking off according to the path obtained in the step S1, RGB image data of a search area are collected in real time through an onboard monocular camera, and the image data collected by different unmanned aerial vehicles at the same time t are integrated into an image set P _t ＝{p ₁ ,p ₂ ,…,p _n P, where _i Searching area images acquired at t moment of the ith unmanned aerial vehicle;

S23, performing target detection on the monocular camera image by using a YOLOv7 model;

utilizing the S21 pre-trained multi-classification YOLOv7 model to real-time acquire the image set P obtained in the step S22 _t Image detection is carried out to obtain a target information set U of each image _O Containing each target bounding boxLength and width (H, L), center point coordinates (x _o ,y _o ) Confidence CONF _O And class number cls;

further, step S2 further includes S24, optimizing the filtering target detection result;

the filtration conditions were set as follows:

condition 1, detection frame confidence CONF _O ≥T _C Wherein T is _C Referring to a lowest confidence threshold constant, discarding the bounding box if less than the constant;

condition 2, target bounding box area H.L is greater than or equal to T _S Wherein T is _S Referring to the lowest area threshold constant, discarding the bounding box if less than the constant;

condition 3, overlapping area S of two adjacent bounding boxes _IOU ≤T _IOU Wherein T is _IOU Referring to a maximum overlapping area threshold constant, if the maximum overlapping area threshold constant is larger than the maximum overlapping area threshold constant, discarding bounding boxes with lower confidence;

condition 4, target class number Cls is not in target class set Cls to be tracked _ex In, i.eDiscarding the bounding box;

according to the filtering condition, the target information set U _O Filtering and updating to obtain a final target information set U 'to be tracked in the task area' _O Better results are obtained.

Step S3, after the unmanned aerial vehicle detects the target, the unmanned aerial vehicle autonomously selects different tracking strategies to intelligently track the target, as shown in fig. 2;

The unmanned aerial vehicle flies along the cruising path, and the real-time image of the monocular camera is subjected to target detection by utilizing YOLOv7 in the cruising process; meanwhile, the unmanned aerial vehicle acquires and stores the current tracking and releasing target positions of other unmanned aerial vehicles in real time, carries out multi-frame clustering according to the acquired target position information released by other unmanned aerial vehicles, and analyzes to obtain the tracking and discovering conditions of the current targets;

further, the step S31 is specifically implemented as follows:

s311, each unmanned aerial vehicle acquires the target positions of other unmanned aerial vehicles which are tracked and released at present, stores the acquired continuous positions in a classification mode, and stores the target positions in an array C respectively _k Wherein k is a category label, C _k The size is L, and if the front and rear position information acquisition interval is greater than s seconds, the current category target is determined to be lost, and C is determined to be _k Initializing and zeroing the array;

s312 if the current time C _k The last digit value of the array is non-zero, and C _k Continuously updating the data in the array, and then classifying the object into the class C _k The array performs k-means clustering, and the purpose of the k-means clustering is to determine that a plurality of different targets under the category are being tracked currently, and the specific method is as follows: determining an initial centroid k value of the cluster by adopting an elbow rule, clustering the initial centroid k value of the cluster under the condition of different values k=2, 3,4, and calculating the sum k of clustering distances according to the following formula _dist ；

C _k Sample X in array _i The clustering result is C _(i) Clustering is performed to obtain distance statistics W (in) of different samples among the same target:

sample distance statistic W (out) between different targets:

the sample statistics for the entire cluster are:

K _dist ＝W(in)+W(out)；

selecting the most proper k value by using an ellaw so as to determine the number of tracked targets in the current category; meanwhile, taking the clustering center position of each category as the coordinates of each target to obtain the tracking condition of other unmanned aerial vehicles on different targets in different categories;

s313 as the nth unmanned aerial vehicle, the target detection set obtained by S24I.e. when the target is detected, if C _k If the distance E is not the same as T, comparing the target position obtained by remote measurement with the target positions of the same class which are currently tracked by other unmanned aerial vehicles, and if the Euclidean distance E is less than T _l The target is the target which is tracked by other unmanned aerial vehicles, and the target is collected U' _O,n The target is deleted.

taking the size of a YOLOv7 bounding box as a judgment basis of a far and near target, wherein h is _i And l _i Respectively, currently detects U' _O,n The height and width of the ith target bounding box;

h _i ＜T _h or l _i ＜T _l Or y _min,i ＜T _y Step S33 is entered, otherwise step S34 is entered;

s33, when the unmanned aerial vehicle judges that the target position is far away from the unmanned aerial vehicle, entering a long-distance tracking program, as shown in fig. 3;

u _x ＝x _o -u _cx

u _y ＝y _o -u _cy

Wherein (u) _cx ,u _cy ) The position of the center point for imaging the monocular camera is determined according to y _o Setting the forward speed v of the unmanned aerial vehicle in a pixel distance segmentation way from the imaging center point of the monocular camera _x The size of (2) is shown in fig. 9:

v _x ＝k _fi ×y _o +b _fi

rotational speed w of unmanned aerial vehicle _z The size of (2) is:

s34, when the unmanned aerial vehicle judges that the target position is close to the unmanned aerial vehicle, entering close-range tracking, as shown in FIG. 4;

u _x ＝x _o -u _cx

u _y ＝y _o -u _cy

v _x ＝k _ni ×y _o +b _ni

wherein k is _ni 、b _ni For different speed adjustment parameters in close-range tracking, i represents y at this time _o An ith section of pixel distance from the monocular camera imaging center point;

rotational speed w of unmanned aerial vehicle _z The size of (2) is:

there are various manners of prejudging and tracking, and those skilled in the art can choose according to specific situations, and the present invention provides an implementation manner as follows:

further, the specific implementation of step S35 includes the following steps:

s351, creating a queue q, wherein the queue stores the current tracked target position of the unmanned aerial vehicle in real time, the last data of the queue q is the latest motion position of the current tracked target, and the data in the queue q are continuous in time;

s352 automatically acquires data in a queue q when the target is lost due to occlusion of the current tracked target and the unmanned aerial vehicle, calculates the advancing direction theta of the target before disappearance, and fits the motion trail of the target:

wherein y is _act,n Y-value, x, representing the last target position in queue q _act,n X value, y representing last target position in queue q _act,i Y-value, x, representing the first target position in queue q _act,1 X-value, v, representing the first target position in queue q _act Representing the target movement speed, t _q Representing the time interval from the acquisition of the first position to the last position in the whole queue q;

s353, predicting the movement track of the blocked target according to the calculated movement direction and movement speed of the target:

x _act,pre ＝y _act,n +cos(q)×(t _now -t _lose )

y _act,pre ＝y _act,n +sin(q)×(t _now -t _lose )

wherein x is _act,pre 、y _act,pre Representing the x and y coordinates, t, of the current predicted target position _now Indicating the current time, t _lose Indicating the target losing moment; the predicted target position is used as the target position of the tracked target after being temporarily lost due to shielding;

s354, the unmanned aerial vehicle adjusts the tracking speed according to the target moving direction and speed obtained through prediction, predicts and tracks, and when the target is not shielded any more, the unmanned aerial vehicle enters S34

v _wx ＝cos(θ)×v _act

v _wy =sin(θ)×v _act

Wherein v is _wx 、v _wy Respectively representing the speeds of the unmanned aerial vehicle in the x axis and the y axis of the world coordinates at the current moment.

S41, eliminating the wrong target position;

the current unmanned aerial vehicle records the coordinates of each category target detection frame in the nearest n frames on the image; when a certain category Frame number n of detected target _d If the number is less than n, the target is not processed, and the sporadic detection result is used as the detection of possible errors to be eliminated;

s43, smoothing the target detection result by utilizing multi-frame sequential information;

s44, calculating the position of the target in the world coordinate system and publishing the position to the global network, as shown in FIG. 5;

[x _u y _u 1]＝[x _m y _m 1]·Trans

[x _w y _w 1 1]＝[x _u y _u 1 1]·Tr _{w_uav}

further, in the embodiment of the present invention, the dynamic path planning algorithm in S5 adopts D ^* The algorithm is used for planning the path returning to the cruising point, and other path planning algorithms can be adopted by the person skilled in the art according to the requirements, D ^* The algorithm plans a path returning to the cruising point as follows:

s51, rasterizing and dividing a task area into an unviewable area, a difficult-to-pass area and an unviewable area;

rasterizing a task area; according to the passing difficulty, the area is divided into 3 main parts: passable areas, non-passable areas and difficult-to-pass areas; the passable area refers to an area with an open visual field and normal passing of the unmanned aerial vehicle, the non-passable area refers to an area with static barriers, and the difficult-to-pass area refers to an area with an unopened visual field and easy collision of the unmanned aerial vehicle;

s52, setting a loss value of each grid, and planning an optimal path for returning to the cruising point;

after the unmanned aerial vehicle completes tracking one target, D is used ^* Calculating a path returning to a cruising point by an algorithm; d (D) ^* The algorithm comprises heuristic functions f (n) =g (n) +h (n), wherein g (n) is the shortest path value currently known from the starting point to the point n, and h (n) is the loss value from the point n to the target;

Loss values of passable areas, inaccessible areas and inaccessible areas in adjacent nodes are respectively set to N _QYS 、N _QYS +n and Inf; at the same time set to 1.4 x N in diagonally adjacent nodes _QYS 、1.4*(N _QYS +n) and Inf; the purpose of setting the loss value is to enable the return route of the unmanned aerial vehicle to be contained in the passable area as much as possible in the completion of the route planning of returning to the cruising point.

Step S6, in the whole process of searching the task, the unmanned aerial vehicle in steps S1-S5 needs to avoid the static obstacle and other unmanned aerial vehicles, and a person skilled in the art can adopt a proper method to avoid the obstacle according to the requirement, and the step S6 is realized according to the following steps:

s61, a static obstacle avoidance algorithm of the unmanned aerial vehicle according to the position change speed of surrounding static obstacles is shown in fig. 7;

according to the positions of static obstacles (such as objects threatening the unmanned aerial vehicle to fly, such as buildings, street lamps, hills and the like), the static obstacle points closest to the unmanned aerial vehicle are obtained by traversing the area around the unmanned aerial vehicle within a certain range;

at speed v when the drone is in close range tracking ₀ ＜v _{avo_m} When flying, starting a fine adjustment obstacle avoidance program; traversing omega around unmanned aerial vehicle _s Points within range find distance unmanned plane U ₀ Nearest obstacle point, Ω _s The calculation can be made by the following formula:

Ω _s ＝{(x,y)x ₀ -D _{avo_s} <x<x ₀ +D _{avo_s} ,y ₀ -D _{avo_s} <y<y ₀ +D _{avo_s} }

wherein D is _{avo_s} The response distance of the unmanned aerial vehicle to the obstacle when the unmanned aerial vehicle is in close-range tracking obstacle avoidance;

if omega _s Points within the range contain points M marked as obstacles _obs Starting a small-speed static obstacle avoidance program; under the unmanned aerial vehicle coordinate system, the forward direction of the unmanned aerial vehicle is taken as the x axis, the leftward translation direction of the unmanned aerial vehicle is taken as the y axis, and then the obstacle point M _obs Distance from unmanned plane in x-axis direction d _x The distance from the unmanned plane in the y-axis direction is d _y The Euclidean distance to the unmanned aerial vehicle is

After obstacle avoidance, the flying speed of the unmanned aerial vehicle in the advancing direction is as follows:

the flying speed of the unmanned aerial vehicle in the left moving direction is as follows:

wherein d _{avo_sx} And d _{avo_sy} In the small-speed static obstacle avoidance program, speed adjustment parameters of the unmanned aerial vehicle in the forward and leftward movement directions are respectively set;

at a speed v when the unmanned aerial vehicle is in the process of remote tracking ₀ ≥v _{avo_m} During flight, starting an emergency obstacle avoidance program; traversing omega around unmanned aerial vehicle _h Points within range find distance unmanned plane U ₀ Nearest obstacle point, Ω _h The calculation can be made by the following formula:

Ω _h ＝{(x,y)|x ₀ -D _{avo_h} ＜x＜x ₀ +D _{avo_h} ,y ₀ -D _{avo_h} ＜y＜y ₀ +D _{avo_h} }

wherein D is _{avo_h} The response distance of the unmanned aerial vehicle to the obstacle when the unmanned aerial vehicle is used for remotely tracking and avoiding the obstacle is provided;

if omega _h Points within the range contain points M marked as obstacles _obs Starting a high-speed static obstacle avoidance program; under the unmanned aerial vehicle coordinate system, the forward direction of the unmanned aerial vehicle is taken as the x axis, the leftward translation direction of the unmanned aerial vehicle is taken as the y axis, and then the obstacle point M _obs Distance from unmanned plane in x-axis direction d _x The distance from the unmanned plane in the y-axis direction is d _y The Euclidean distance to the unmanned aerial vehicle is

v _y ＝abs(v ₀ )·k _{avo_hx} ·(D _{avo_h} -d _e )·(D _{avo_h} -|d _y |)

wherein d _{avo_hx} And k _{avo_hx} Respectively adjusting parameters of the speed of the unmanned aerial vehicle in the directions of an x axis and a y axis in a high-speed static obstacle avoidance program;

for the conditions of different speeds of the unmanned aerial vehicle, v _{avo_m} In order to distinguish boundary values of large-speed flight and small-speed flight of the unmanned aerial vehicle, parameter settings of obstacle avoidance programs are different under different speeds, and the general conditions are as follows:

d _{avo_hx} ＞d _{avo_sx} ,d _{avo_sy} ＞1,k _{avo_hx} <<1,D _{avo_h} ＞D _{avo_s}

s62, the unmanned aerial vehicle changes the inter-machine dynamic obstacle avoidance algorithm of the height according to the position relation between other unmanned aerial vehicles, as shown in fig. 8;

unmanned plane Uav ₀ Acquiring Uav of other N-1 unmanned aerial vehicles in real time ₁ ,Uav ₂ ,...,Uav _N-1 And calculates the distance d with other unmanned aerial vehicles ₁ ,d ₂ ,...,d _N-1 The method comprises the steps of carrying out a first treatment on the surface of the When min { d ₁ ,d ₂ ,...,d _N-1 }＜D _{avo_d} During the starter obstacle avoidance procedure, D _{avo_d} The response distance for dynamic obstacle avoidance between unmanned aerial vehicles is;

in general, the initial cruising altitude of the unmanned aerial vehicle is H _cru After the obstacle avoidance process of the starter, the expected cruising altitude of the unmanned aerial vehicle is modified to H _avo The calculation method is as follows:

H _avo ＝H _cru +N _higher ·H _bias -N _lower ·H _bias

wherein N is _higher Is unmanned plane Uav ₀ The distance between the two is smaller than D _{avo_d} And height z _i ＞z ₀ (i=1, 2,.,. The number of unmanned aerial vehicles of N-1) (number ratio unmanned aerial vehicle Uav when the heights are the same ₀ Number of unmanned aerial vehicles with large numbers), N _lower Is unmanned plane Uav ₀ The distance between the two is smaller than D _{avo_d} And height z _i ＜z ₀ (i=1, 2,.,. The number of unmanned aerial vehicles of N-1) (number ratio unmanned aerial vehicle Uav when the heights are the same ₀ Unmanned aerial vehicle number with small number)；H _bias The height adjustment parameter of the obstacle avoidance among the unmanned aerial vehicle is related to the maximum height of the unmanned aerial vehicle body, and is generally more than half of the maximum height of the unmanned aerial vehicle body;

after the expected cruising altitude of the unmanned aerial vehicle is modified, the speed v of the unmanned aerial vehicle in the vertical direction is changed _h To a desired height, v _h The calculation method is as follows:

v _h ＝(H _avo -z ₀ )·v _bias

wherein v is _bias The unmanned aerial vehicle height adjustment method is an adjustment parameter of the unmanned aerial vehicle height direction speed during dynamic obstacle avoidance, and the larger the parameter is, the faster the unmanned aerial vehicle height adjustment is;

when min { d ₁ ,d ₂ ,...,d _N-1 }≥D _{avo_d} When the unmanned aerial vehicle exits from the inter-aircraft dynamic obstacle avoidance program, the expected cruising altitude of the unmanned aerial vehicle is changed into H _cru At the same time with v _h ＝(H _cru -z ₀ )·v _bias Is returned to the original desired height.

Claims

1. The collaborative searching and tracking positioning method for the multiple moving targets by the unmanned aerial vehicle cluster is characterized by comprising the following steps:

step S1, a gridding environment information and an unmanned aerial vehicle cruising model are constructed according to specific task requirements; the S1 is realized according to the following steps:

Classifying different grids into a set of obstacle grids phi according to actual obstacle information _obs And a trafficable grid set Φ _ava ；

S12, searching a cruising path in a passable area;

setting a search condition set U _cond ＝{C ₁ ,C ₂ ,C ₃ }, wherein C ₁ ：Uav _n Refers to a grid covered by a monocular camera view of an nth unmanned aerial vehicle, C ₁ Meaning that the total grid covered by the monocular camera vision of each unmanned aerial vehicle is equal to the trafficable grid set phi _ava ；

S13, after the cruising path is determined, the unmanned aerial vehicle takes off and flies along the cruising path at the speed v and the height H _cru Cruising;

s21, training a target detection algorithm network model;

condition 2: the training times reach the preset maximum set times E _max ；

Secondly, collecting M task area images containing targets, inputting a trained target detection algorithm model to generate new target labels, manually checking and correcting detection results to finally obtain a target label set L of the area to be searched _O The method comprises the steps of carrying out a first treatment on the surface of the Wherein M is much greater than N;

S22, acquiring an image of a region to be searched in real time;

cruising search is carried out on the unmanned aerial vehicle after taking off according to the path obtained in the step S1, RGB image data of a search area are collected in real time through an onboard monocular camera, and the image data collected by different unmanned aerial vehicles at the same time t are integrated into an image set P _t ＝{p ₁ ,p ₂ ,…,p _n P, where _n Searching area images acquired at the moment t of the nth unmanned aerial vehicle;

s24, optimizing a filtering target detection result;

the filtration conditions were set as follows:

according to the filtering condition, the target information set U _O Filtering and updating to obtain a final target information set U 'to be tracked in the task area' _O ；

S32, traversing U 'by unmanned aerial vehicle' _O According to the distance between the detected target and the unmanned plane, selecting the nearest target, and judging to enter a long-distance tracking or short-distance tracking program:

surrounding large frames with target detection algorithmsThe small is taken as the judgment basis of the near-far target, wherein h _i And l _i Respectively, currently detects U' _O The height and width of the ith target bounding box;

wherein T is _h 、T _l 、T _y Numerical selection is set according to the imaging resolution of the monocular camera used, y _min,i Is U' _O The y value of the pixel point at the upper left corner of the ith bounding box;

u _x ＝x _o -u _cx

u _y ＝y _o -u _cy

v _x ＝k _fi ×y _o +b _fi

rotational speed w of unmanned aerial vehicle _z The size of (2) is:

u _x ＝x _o -u _cx

u _y ＝y _o -u _cy

v _x ＝k _ni ×y _o +b _ni

rotational speed w of unmanned aerial vehicle _z The size of (2) is:

Wherein k is _n Yaw for unmanned aerial vehicle in short distanceAngular velocity w _z Wherein the left turn is positive and the right turn is negative with the viewing angle of the unmanned aerial vehicle;

S41, eliminating the wrong target position;

after false detection is eliminated, calculating average sitting of target detection frames in n framesScale value (x) _m ,y _m )：

Wherein x is _i ，y _i Respectively an x-axis coordinate and a y-axis coordinate of a target detection frame in an ith frame;

s44, calculating the position of the target in the world coordinate system and publishing the position to the global network;

under the camera image coordinate system and the unmanned aerial vehicle body coordinate system, corresponding coordinates of n and n >4 are found out and brought into the following perspective transformation formula, and a homography matrix Trans of the perspective transformation is calculated:

[x _u y _u 1]＝[x _m y _m 1]·Trans

[x _w y _w 1 1]＝[x _u y _u 1 1]·Tr _{w_uav}

step S6, in the whole process of searching tasks, the unmanned aerial vehicle in the steps S1-S5 needs to perform obstacle avoidance treatment on static obstacles and other unmanned aerial vehicles;

s61, a static obstacle avoidance algorithm of the unmanned aerial vehicle according to the position change speed of surrounding static obstacles;

according to the position of the static obstacle, traversing the area in a certain range around the unmanned aerial vehicle to obtain a static obstacle point nearest to the unmanned aerial vehicle;

at speed v when the drone is in close range tracking ₀ ＜v _{avo_m} When flying, starting a fine adjustment obstacle avoidance program; traversing omega around unmanned aerial vehicle _s Points within range find distance unmanned plane Uav ₀ Nearest obstacle point, Ω _s The calculation can be made by the following formula:

Ω _s ＝{(x,y)|x ₀ -D _{avo_s} ＜x＜x ₀ +D _{avo_s} ,y ₀ -D _{avo_s} ＜y＜y ₀ +D _{avo_s} }

wherein D is _{avo_s} Avoidance for unmanned aerial vehicle tracking in close rangeResponse distance to the obstacle at the time of obstacle;

at a speed v when the unmanned aerial vehicle is in the process of remote tracking ₀ ≥v _{avo_m} During flight, starting an emergency obstacle avoidance program; traversing omega around unmanned aerial vehicle _h Points within range find distance unmanned plane Uav ₀ Nearest obstacle point, Ω _h The calculation can be made by the following formula:

v _y ＝abs(v ₀ )·k _{avo_hx} ·(D _{avo_h} -d _e )·(D _{avo_h} -|d _y |)

d _{avo_hx} ＞d _{avo_sx} ,d _{avo_sy} ＞1,k _{avo_hx} ＜＜1,D _{avo_h} ＞D _{avo_s}

s62, the unmanned aerial vehicle changes the inter-machine dynamic obstacle avoidance algorithm of the height according to the position relation between other unmanned aerial vehicles;

H _avo ＝H _cru +N _higher ·H _bias -N _lower ·H _bias

wherein N is _higher Is unmanned plane Uav ₀ The distance between the two is smaller than D _{avo_d} And height z _i ＜z ₀ (i=1, 2,., N-1) the number of unmanned aerial vehicles, the number being the same when the heights are the same than the unmanned aerial vehicle Uav ₀ Number of unmanned aerial vehicles with large number N _lower Is unmanned plane Uav ₀ The distance between the two is smaller than D _{avo_d} And height z _i ＜z ₀ (i=1, 2,., N-1) the number of unmanned aerial vehicles, the number being the same when the heights are the same than the unmanned aerial vehicle Uav ₀ Numbering the number of unmanned aerial vehicles with small numbers; h _bias The height adjustment parameter of the obstacle avoidance among the unmanned aerial vehicle is related to the maximum height of the unmanned aerial vehicle body, and is generally more than half of the maximum height of the unmanned aerial vehicle body;

v _h ＝(H _avo -z ₀ )·v _bias

2. The method for collaborative searching and tracking positioning of multiple moving objects by an unmanned aerial vehicle cluster according to claim 1, wherein the object detection algorithm is YOLOv4 or YOLOv7.

3. The method for collaborative searching and tracking positioning of multiple moving objects by an unmanned aerial vehicle cluster according to claim 2, wherein the step S31 specifically comprises the following steps:

s312 if the current time C _k The last digit value of the array is non-zero, and C _k Continuously updating the data in the array, and then classifying the object into the class C _k K-means clustering is carried out on the array, and it is determined that a plurality of different targets under the category are currently tracked;

s313 as the nth unmanned aerial vehicle, the target detection set obtained by S24I.e. when the target is detected, if C _k If the distance E is not the same as T, comparing the target position obtained by remote measurement with the target positions of the same class which are currently tracked by other unmanned aerial vehicles, and if the Euclidean distance E is less than T _l The target is the target which is tracked by other unmanned aerial vehicles, and the target is collected U' _O The target is deleted.

4. The method for collaborative searching and tracking positioning of multiple moving objects by an unmanned aerial vehicle cluster according to claim 1, wherein the step S35 specifically comprises the following steps:

x _act,pre ＝x _act,n +cos(q)×(t _now -t _lose )

y _act,pre ＝y _act,n +sin(q)×(t _now -t _lose )

wherein x is _act,pre 、y _act,pre Representing the x, y coordinates, t, of the current predicted target position _now Indicating the current time, t _lose Indicating the target losing moment; the predicted target position is used as the target position of the tracked target after being temporarily lost due to shielding;

s354, the unmanned aerial vehicle adjusts the tracking speed according to the predicted target moving direction and speed, predicts and tracks, and when the target is not shielded any more, the unmanned aerial vehicle enters S34;

v _wx =cos(θ)×v _act

v _wy =sin(θ)×v _act

5. The method for collaborative searching and tracking positioning of unmanned aerial vehicle clusters on multiple moving targets according to claim 1, wherein the dynamic path planning algorithm in S5 adopts D ^* The algorithm is used for planning a path returning to the cruising point, and the specific steps are as follows:

loss values of passable areas, inaccessible areas and inaccessible areas in adjacent nodes are respectively set to N _QYS 、N _QYS +n and Inf; at the same time set to 1.4 x N in diagonally adjacent nodes _QYS 、1.4*(N _QYS +n) and Inf.

6. The method for collaborative searching and tracking positioning of multiple moving objects by an unmanned aerial vehicle cluster according to claim 3, wherein the specific method for k-means clustering in S312 is as follows: determining initial centroids of clusters using elbow's law _k Values, k=2, 3,4 for different valuesClustering is performed separately, and the sum k of the clustering distances is calculated by the following formula _dist ；

C _k Sample A in array _i The clustering result is C _(i) Clustering is performed to obtain distance statistics W (in) of different samples among the same target:

sample distance statistic W (out) between different targets:

the sample statistics for the entire cluster are:

K _dist =W(in)+W(out)；

selecting the most proper k value by using an ellaw so as to determine the number of tracked targets in the current category; and simultaneously, taking the clustering center position of each category as the coordinates of each target to obtain the tracking condition of other unmanned aerial vehicles on different targets in different categories.