CN111311641B - Unmanned aerial vehicle target tracking control method - Google Patents
Unmanned aerial vehicle target tracking control method Download PDFInfo
- Publication number
- CN111311641B CN111311641B CN202010116432.4A CN202010116432A CN111311641B CN 111311641 B CN111311641 B CN 111311641B CN 202010116432 A CN202010116432 A CN 202010116432A CN 111311641 B CN111311641 B CN 111311641B
- Authority
- CN
- China
- Prior art keywords
- target
- aerial vehicle
- unmanned aerial
- video sequence
- tracking
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/254—Analysis of motion involving subtraction of images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20224—Image subtraction
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The invention relates to an unmanned aerial vehicle target tracking control method, and belongs to the field of unmanned aerial vehicle target tracking. The method is that an unmanned plane transmits an acquired video sequence back to a ground tracking control system through a mobile network link; the ground tracking control system selects a target to be tracked in the returned first frame video sequence; then, performing feature learning on the target to obtain a tracking template; calculating the position of the estimated target in the video sequence of the next frame by tracking the template and the video sequence of the next frame; and calculating the vector difference of the centers of mass of the targets of the front frame and the rear frame, and transmitting the vector difference as the input of the unmanned aerial vehicle control system to the unmanned aerial vehicle board through the mobile network link to control the unmanned aerial vehicle to track the target to fly. The invention can realize ultra-remote tracking control on the unmanned aerial vehicle, and solves the problem of intelligent tracking of moving targets in the airport scene of the unmanned aerial vehicle.
Description
Technical Field
The invention belongs to the field of unmanned aerial vehicle target tracking, and relates to an unmanned aerial vehicle target tracking control method.
Background
With the development of computer vision technology, the detection and tracking of moving objects are receiving more and more attention from students. It involves many related disciplines such as image processing, linear algebra, probability and statistics, etc. And have been widely used in the military, civilian and commercial fields. Unmanned aerial vehicle technology development is also faster and faster, and current unmanned aerial vehicle is little, the quality is light, and production and operation maintenance cost are lower, can reduce a lot of manual labor. Especially, the rotor unmanned aerial vehicle can realize tasks such as vertical lifting and hovering in the sky in a flexible way, and along with the development of the automatic control field, the unmanned aerial vehicle and computer vision are combined to be applied to the fields such as inspection, search and rescue, aerial photography and the like more and more widely.
At present, in the field of unmanned aerial vehicle target tracking, the unmanned aerial vehicle target tracking is limited by the calculation power of an onboard board card, real-time target tracking cannot be achieved, and the limitation of communication distance cannot be achieved, and when the target is far away, ultra-remote tracking control cannot be achieved.
Disclosure of Invention
Therefore, the invention aims to provide the unmanned aerial vehicle target tracking control method, which is characterized in that target tracking calculation is put in a tracking control system at the ground end, and then control information is transmitted to an onboard board through a 4G link, so that the unmanned aerial vehicle is subjected to ultra-remote tracking control, and the problem of intelligent tracking of a moving target under an unmanned airport scene is solved.
In order to achieve the above purpose, the present invention provides the following technical solutions:
the unmanned aerial vehicle target tracking control method comprises the steps that an unmanned aerial vehicle transmits an acquired video sequence back to a ground tracking control system through a 4G link; the ground tracking control system selects a target to be tracked in the returned first frame video sequence; then, performing feature learning on the target to obtain a tracking template; calculating the position of the estimated target in the video sequence of the next frame by tracking the template and the video sequence of the next frame; and calculating the vector difference of the centers of mass of the front frame and the rear frame, and transmitting the vector difference as the input of the unmanned aerial vehicle control system to the unmanned aerial vehicle board through a 4G link to control the unmanned aerial vehicle to track the target to fly.
Further, the tracking control method specifically includes the following steps:
s1: acquiring a video sequence through a camera fixed on the unmanned aerial vehicle;
s2: the acquired video sequence is transmitted back to the ground tracking control system through a 4G link;
s3: selecting a target to be tracked in the returned first frame video sequence through a ground tracking control system;
s4: the ground tracking control system performs feature learning on the target to obtain a tracking template;
s5: calculating the position of the estimated target in the video sequence of the next frame, namely the position of the mass center of the target in the image, through the tracking template obtained in the step S4 and the video sequence of the next frame;
s6: calculating vector difference of the mass centers of the targets of the front frame and the rear frame according to the target position obtained in the step S5;
s7: transmitting the vector difference obtained in the step S6 as input of an unmanned aerial vehicle control system to an unmanned aerial vehicle board through a 4G link to control the unmanned aerial vehicle to track a target to fly;
s8: taking the target predicted in the step S5 as a new target, and returning to the step S4;
s9: repeating the steps S5-S8 to continuously track the target until the video sequence is terminated.
Further, the step S2 specifically includes:
s21: transplanting a network card drive on the carrier plate to connect the network card drive with a mobile network;
s22: inserting the mobile network card into the record edition, then dialing and networking, and connecting to a mobile network;
s23: the video sequence acquired by the camera is transmitted back to the ground heel control system by the board through the 4G network.
Further, the step S4 specifically includes:
s41: extracting HOG features and CN features of a video sequence to obtain a target sample z;
s42: the filter with the largest output response to the target sample z is found by calculation, namely the target tracking template. The calculation formula of the tracking template is as follows:
where F is the input image and G is the output response.
Further, the step S5 specifically includes:
s51: circularly shifting the target area obtained in the previous frame to obtain a large number of areas to be detected;
s52: and calculating the maximum response by using the region to be detected and the tracking template to obtain the target position of the current frame of the target.
Further, the step S7 specifically includes:
s71: transmitting the vector difference calculated in the step S6 as a control instruction to an airborne board of the unmanned aerial vehicle through a mobile network;
s72: the vector difference transmitted to the airborne board is used as an input control quantity of the unmanned aerial vehicle PID control, and the unmanned aerial vehicle is controlled to fly along with the target; the unmanned aerial vehicle position control formula is as follows:
wherein m is the mass of the unmanned aerial vehicle,
is the sum of the lifting forces provided by 4 motors of the unmanned aerial vehicle, K f Is unmanned plane resistance, K p Is a specific control parameter, v is the unmanned aerial vehicle speed.
The invention has the beneficial effects that: according to the invention, the calculation of target tracking is put in the ground tracking control system, and the large calculation force of the ground system relative to the onboard board card can update the target in real time. And the feedback of the video sequence and the control instruction of the ground tracking control system are transmitted through a 4G link, the influence of the transmitted video high definition on the tracking algorithm is small, and the ultra-remote tracking control can be performed through a 4G network.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objects and other advantages of the invention may be realized and obtained by means of the instrumentalities and combinations particularly pointed out in the specification.
Drawings
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in the following preferred detail with reference to the accompanying drawings, in which:
FIG. 1 is a block flow diagram of a tracking control system of the present invention.
Detailed Description
Other advantages and effects of the present invention will become apparent to those skilled in the art from the following disclosure, which describes the embodiments of the present invention with reference to specific examples. The invention may be practiced or carried out in other embodiments that depart from the specific details, and the details of the present description may be modified or varied from the spirit and scope of the present invention. It should be noted that the illustrations provided in the following embodiments merely illustrate the basic idea of the present invention by way of illustration, and the following embodiments and features in the embodiments may be combined with each other without conflict.
Referring to fig. 1, the method for controlling target tracking of an unmanned aerial vehicle specifically comprises the following steps:
s1: and selecting an M100 unmanned aerial vehicle in the large area, and fixing a camera below the unmanned aerial vehicle to acquire a video sequence.
S2: the video sequence acquired by the camera is transmitted back to the ground tracking control system through the 4G network by the carrier plate. The step S2 specifically comprises the following steps:
s21: and transplanting a network card drive on the carrier board so that the network card drive can be connected with a network.
S22: and inserting the mobile network card into the record edition, and then performing dial-up networking to connect to the 4G network.
S23: the video sequence acquired by the camera is transmitted back to the ground heel control system by the board through the 4G network.
S3: and after the first frame of video sequence is returned to the ground tracking control system, a target to be tracked is selected by using a mouse picture frame.
S4: and obtaining a target tracking template through learning the target.
S41: and extracting HOG features and CN features of the video sequence to obtain a target sample z.
S42: the filter with the largest output response to the target sample z is found by calculation, namely the target tracking template.
S5: and (3) calculating the position of the estimated target in the video sequence of the next frame, namely the position of the mass center of the target in the image, through the tracking template obtained in the step (S4) and the video sequence of the next frame.
S51: and circularly shifting the target area obtained in the previous frame to obtain a large number of areas to be detected.
S52: and calculating the maximum response by using the region to be detected and the tracking template to obtain the target position of the current frame of the target.
S6: and (5) calculating the vector difference of the mass centers of the targets of the front frame and the rear frame according to the target position obtained in the step (S5).
S7: and (3) transmitting the vector difference obtained in the step (S6) as input of an unmanned aerial vehicle control system to an unmanned aerial vehicle board through a 4G link to control the unmanned aerial vehicle to track the target to fly.
S71: and (3) transmitting the vector difference calculated in the step (S6) as a control instruction to an airborne board of the unmanned aerial vehicle through a 4G network.
S72: the vector difference transmitted to the airborne board is used as an input control quantity of the unmanned aerial vehicle PID control, and the unmanned aerial vehicle is controlled to fly along with the target.
S8: taking the target area predicted in the step S5 as a new target, and returning to the step S4;
s9: the steps S5-S8 are then repeated to continue tracking the target until the video sequence is terminated.
Finally, it is noted that the above embodiments are only for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made thereto without departing from the spirit and scope of the present invention, which is intended to be covered by the claims of the present invention.
Claims (2)
1. The unmanned aerial vehicle target tracking control method is characterized by comprising the following steps of:
s1: acquiring a video sequence through a camera fixed on the unmanned aerial vehicle;
s2: the acquired video sequence is transmitted back to the ground tracking control system through a 4G link;
s3: selecting a target to be tracked in the returned first frame video sequence through a ground tracking control system;
s4: the ground tracking control system performs feature learning on the target to obtain a tracking template, and specifically comprises the following steps:
s41: extracting HOG features and CN features of a video sequence to obtain a target sample z;
s42: and calculating to find a filter with the largest output response with the target sample z, namely a target tracking template, wherein the calculation formula of the tracking template is as follows:
where F is the input image and G is the output response;
s5: calculating the position of a predicted target in the video sequence of the current frame, namely the position of the mass center of the target in the image, through the tracking template obtained in the step S4 and the video sequence of the previous frame; the method specifically comprises the following steps:
s51: circularly shifting the target area obtained in the previous frame to obtain a large number of areas to be detected;
s52: calculating to obtain the maximum response, namely the target position of the current frame of the target, by using the region to be detected and the tracking template;
s6: calculating vector difference of the mass centers of the targets of the front frame and the rear frame according to the target position obtained in the step S5;
s7: transmitting the vector difference obtained in the step S6 as input of an unmanned aerial vehicle control system to an unmanned aerial vehicle board through a 4G link to control the unmanned aerial vehicle to track a target to fly; the method specifically comprises the following steps:
s71: transmitting the vector difference calculated in the step S6 as a control instruction to an airborne board of the unmanned aerial vehicle through a mobile network;
s72: the vector difference p transmitted to the airborne board is used as an input control quantity of the unmanned aerial vehicle PID control, and the unmanned aerial vehicle is controlled to fly along with the target; the unmanned aerial vehicle position control formula is as follows:
wherein m is the mass of the unmanned aerial vehicle,
is the sum of the lifting forces provided by 4 motors of the unmanned aerial vehicle, K f Is unmanned plane resistance, K p Is a proportional control parameter, v is the unmanned aerial vehicle speed;
s8: taking the target predicted in the step S5 as a new target, and returning to the step S4;
s9: repeating the steps S5-S8 to continuously track the target until the video sequence is terminated.
2. The unmanned aerial vehicle target tracking control method according to claim 1, wherein the step S2 specifically comprises:
s21: transplanting a network card drive on the carrier plate to connect the network card drive with a mobile network;
s22: inserting the mobile network card into the record edition, then dialing and networking, and connecting to a mobile network;
s23: the video sequence acquired by the camera is transmitted back to the ground heel control system by the board through the 4G network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010116432.4A CN111311641B (en) | 2020-02-25 | 2020-02-25 | Unmanned aerial vehicle target tracking control method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010116432.4A CN111311641B (en) | 2020-02-25 | 2020-02-25 | Unmanned aerial vehicle target tracking control method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111311641A CN111311641A (en) | 2020-06-19 |
| CN111311641B true CN111311641B (en) | 2023-06-09 |
Family
ID=71146034
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010116432.4A Active CN111311641B (en) | 2020-02-25 | 2020-02-25 | Unmanned aerial vehicle target tracking control method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111311641B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109859242B (en) * | 2019-01-16 | 2022-09-09 | 重庆邮电大学 | Target tracking method for prediction adaptive learning |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105117720A (en) * | 2015-09-29 | 2015-12-02 | 江南大学 | Object scale self-adaption tracking method based on spatial-temporal model |
| CN108009494A (en) * | 2017-11-30 | 2018-05-08 | 中山大学 | A kind of intersection wireless vehicle tracking based on unmanned plane |
| CN108090919A (en) * | 2018-01-02 | 2018-05-29 | 华南理工大学 | Improved kernel correlation filtering tracking method based on super-pixel optical flow and adaptive learning factor |
| CN108776974A (en) * | 2018-05-24 | 2018-11-09 | 南京行者易智能交通科技有限公司 | A kind of real-time modeling method method suitable for public transport scene |
| CN108875547A (en) * | 2018-04-16 | 2018-11-23 | 中国科学院信息工程研究所 | A kind of method for tracking target and device of resourceoriented constrained environment |
| CN109325961A (en) * | 2018-08-27 | 2019-02-12 | 北京悦图遥感科技发展有限公司 | UAV Video multi-object tracking method and device |
| CN109445453A (en) * | 2018-09-12 | 2019-03-08 | 湖南农业大学 | A kind of unmanned plane Real Time Compression tracking based on OpenCV |
| WO2020019353A1 (en) * | 2018-07-27 | 2020-01-30 | 深圳市大疆创新科技有限公司 | Tracking control method, apparatus, and computer-readable storage medium |
-
2020
- 2020-02-25 CN CN202010116432.4A patent/CN111311641B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105117720A (en) * | 2015-09-29 | 2015-12-02 | 江南大学 | Object scale self-adaption tracking method based on spatial-temporal model |
| CN108009494A (en) * | 2017-11-30 | 2018-05-08 | 中山大学 | A kind of intersection wireless vehicle tracking based on unmanned plane |
| CN108090919A (en) * | 2018-01-02 | 2018-05-29 | 华南理工大学 | Improved kernel correlation filtering tracking method based on super-pixel optical flow and adaptive learning factor |
| CN108875547A (en) * | 2018-04-16 | 2018-11-23 | 中国科学院信息工程研究所 | A kind of method for tracking target and device of resourceoriented constrained environment |
| CN108776974A (en) * | 2018-05-24 | 2018-11-09 | 南京行者易智能交通科技有限公司 | A kind of real-time modeling method method suitable for public transport scene |
| WO2020019353A1 (en) * | 2018-07-27 | 2020-01-30 | 深圳市大疆创新科技有限公司 | Tracking control method, apparatus, and computer-readable storage medium |
| CN109325961A (en) * | 2018-08-27 | 2019-02-12 | 北京悦图遥感科技发展有限公司 | UAV Video multi-object tracking method and device |
| CN109445453A (en) * | 2018-09-12 | 2019-03-08 | 湖南农业大学 | A kind of unmanned plane Real Time Compression tracking based on OpenCV |
Non-Patent Citations (1)
| Title |
|---|
| 特征融合的尺度自适应相关滤波跟踪算法;李聪 等;《光学学报》;20180531;第1-8页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111311641A (en) | 2020-06-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110443827B (en) | Unmanned aerial vehicle video single-target long-term tracking method based on improved twin network | |
| CN106981073B (en) | A kind of ground moving object method for real time tracking and system based on unmanned plane | |
| Xiong et al. | Visual detection of green mangoes by an unmanned aerial vehicle in orchards based on a deep learning method | |
| Chamoso et al. | UAVs applied to the counting and monitoring of animals | |
| CN108255198B (en) | Shooting cradle head control system and control method under unmanned aerial vehicle flight state | |
| CN111765974B (en) | Wild animal observation system and method based on miniature refrigeration thermal infrared imager | |
| CN112162565B (en) | Uninterrupted self-main-pole tower inspection method based on multi-machine collaborative operation | |
| CN107886523A (en) | Vehicle target movement velocity detection method based on unmanned plane multi-source image | |
| CN109116298B (en) | Positioning method, storage medium and positioning system | |
| CN106210647A (en) | Based on the method and system building base station coverage area full-view image of taking photo by plane | |
| CN112927264A (en) | Unmanned aerial vehicle tracking shooting system and RGBD tracking method thereof | |
| CN111311641B (en) | Unmanned aerial vehicle target tracking control method | |
| CN115649501B (en) | Unmanned aerial vehicle night lighting system and method | |
| CN107703936A (en) | Automatic Guided Vehicle system and dolly localization method based on convolutional neural networks | |
| CN111831010A (en) | Unmanned aerial vehicle obstacle avoidance flight method based on digital space slice | |
| CN111354016A (en) | Unmanned aerial vehicle ship tracking method and system based on deep learning and difference value hashing | |
| CN111161305A (en) | Intelligent unmanned aerial vehicle identification tracking method and system | |
| CN107458619B (en) | Method and system for automatically landing and charging full-automatic micro-miniature four-rotor wing | |
| CN112241180B (en) | Visual processing method for landing guidance of unmanned aerial vehicle mobile platform | |
| CN112364854B (en) | Airborne target approaching guidance system and method based on detection, tracking and fusion | |
| CN114463301A (en) | Intelligent accompanying flight method for unmanned aerial vehicle video target in multi-shielding environment | |
| CN114620218A (en) | Unmanned aerial vehicle landing method based on improved YOLOv5 algorithm | |
| Sung et al. | Onboard pattern recognition for autonomous UAV landing | |
| Bakken et al. | End-to-end learning for autonomous navigation for agricultural robots | |
| CN110751694B (en) | Image navigation method based on three-color channel fusion mutual information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |