CN117130383A - Unmanned aerial vehicle vision tracking method and system, unmanned aerial vehicle and readable storage medium - Google Patents
Unmanned aerial vehicle vision tracking method and system, unmanned aerial vehicle and readable storage medium Download PDFInfo
- Publication number
- CN117130383A CN117130383A CN202311186940.XA CN202311186940A CN117130383A CN 117130383 A CN117130383 A CN 117130383A CN 202311186940 A CN202311186940 A CN 202311186940A CN 117130383 A CN117130383 A CN 117130383A
- Authority
- CN
- China
- Prior art keywords
- unmanned aerial
- aerial vehicle
- target
- image
- cradle head
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 48
- 230000002787 reinforcement Effects 0.000 claims abstract description 31
- 230000009471 action Effects 0.000 claims description 23
- 230000006870 function Effects 0.000 claims description 23
- 238000001514 detection method Methods 0.000 claims description 19
- 238000012549 training Methods 0.000 claims description 19
- 230000000007 visual effect Effects 0.000 claims description 19
- 238000001914 filtration Methods 0.000 claims description 8
- 238000004088 simulation Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000001629 suppression Effects 0.000 claims description 5
- 230000008878 coupling Effects 0.000 abstract description 2
- 238000010168 coupling process Methods 0.000 abstract description 2
- 238000005859 coupling reaction Methods 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 238000011217 control strategy Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000005764 inhibitory process Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
Abstract
The application provides an unmanned aerial vehicle vision tracking method, an unmanned aerial vehicle vision tracking system, an unmanned aerial vehicle and a readable storage medium. Acquiring a target image acquired by a camera on a cradle head fixedly connected with the unmanned aerial vehicle; identifying a relative offset position of the tracked object in the target image relative to a central region of the image; according to the relative offset position, obtaining control quantity information of the unmanned aerial vehicle and the cradle head through a reinforcement learning algorithm; and according to the control quantity information, taking the image center area as a reference, and jointly controlling the unmanned aerial vehicle and the cradle head to enable the tracked target to be positioned in the image center area. In this way, according to the relative offset position of the tracked target in the target image relative to the central area of the image, the control quantity information of the unmanned aerial vehicle and the cradle head is obtained, the coupling of the cradle head gesture and the unmanned aerial vehicle gesture is not needed, the model of the unmanned aerial vehicle and the model of the cradle head are not needed to be established, and decoupling is not needed, so that the difficulty of decoupling does not exist.
Description
Technical Field
The application relates to the technical field of unmanned aerial vehicle control, in particular to an unmanned aerial vehicle vision tracking method, an unmanned aerial vehicle vision tracking system, an unmanned aerial vehicle and a readable storage medium.
Background
With the continuous progress of technology, unmanned aerial vehicles are widely used in both civil and military fields. The fixed wing unmanned aerial vehicle has the advantages of high sailing speed, long sailing time and the like, and is widely applied to the tracking of ground targets.
The method of carrying a visual sensor on an unmanned aerial vehicle is generally adopted to acquire image information of a ground target, so that the target is tracked. The vision sensor may be a pan-tilt camera. In the prior art, when the unmanned aerial vehicle is controlled, a model of the unmanned aerial vehicle and a model of a cradle head are required to be established, and the cradle head gesture is required to be coupled with the unmanned aerial vehicle gesture, so that the decoupling difficulty is high.
Disclosure of Invention
The application provides an unmanned aerial vehicle vision tracking method, an unmanned aerial vehicle vision tracking system, an unmanned aerial vehicle and a readable storage medium, wherein a model of the unmanned aerial vehicle and a model of a cradle head do not need to be established, and decoupling is not needed.
The application provides an unmanned aerial vehicle vision tracking method, which comprises the following steps:
acquiring a target image acquired by a camera on a cradle head fixedly connected with the unmanned aerial vehicle;
identifying a relative offset position of the tracked object in the object image relative to a central region of the image;
obtaining control quantity information of the unmanned aerial vehicle and the cradle head according to the relative offset position;
and according to the control quantity information, taking the image center area as a reference, and jointly controlling the unmanned aerial vehicle and the cradle head to enable the tracked target to be positioned in the image center area.
Further, the obtaining the control amount information of the unmanned aerial vehicle and the cradle head according to the relative offset position includes:
inputting the relative offset position, the current position of the cradle head, flight information and the position of the unmanned aerial vehicle into an unmanned aerial vehicle control model, and outputting to obtain control quantity information of the unmanned aerial vehicle and the cradle head; the unmanned aerial vehicle control model is obtained by training a target image sample formed by visual information and physical parameters of an unmanned aerial vehicle flight environment.
Further, the method further comprises: the unmanned aerial vehicle control model is obtained through training in the following mode: the visual information in the simulation environment and the physical parameters of the unmanned aerial vehicle flight environment are randomized through a domain randomization method, so that the sample size of the unmanned aerial vehicle control model is increased; based on a reinforcement learning algorithm, training an unmanned aerial vehicle control model by using the increased sample size to obtain the unmanned aerial vehicle control model; the unmanned aerial vehicle control model is used for migrating to the real unmanned aerial vehicle.
Further, the inputting the relative offset position, the current pose of the cradle head, the flight information and the pose of the unmanned aerial vehicle to the unmanned aerial vehicle control model, and outputting and obtaining the control quantity information of the unmanned aerial vehicle and the cradle head includes:
taking the relative offset position, the current pose of the cradle head, the flight information and the pose of the unmanned aerial vehicle as observables of a reinforcement learning algorithm;
taking the yaw angular velocity of the unmanned aerial vehicle and the target pose of the cradle head as actions of the reinforcement learning algorithm;
taking the square of the relative offset position as a penalty term for a reward function in the reinforcement learning algorithm;
taking the navigation speed of the unmanned aerial vehicle as a reward item of the reward function;
and obtaining the control quantity information of the unmanned aerial vehicle and the cradle head through the reinforcement learning algorithm by using the observed quantity, the action and the reward function.
Further, the obtaining, by using the observed quantity, the action and the reward function and the reinforcement learning algorithm, control quantity information of the unmanned aerial vehicle and the pan-tilt includes:
determining a reward value of the reward function in a plurality of iterations under the conditions of the observed quantity and the action;
and selecting a state space and an action space which correspond to the largest accumulated rewarding value as the control quantity information through the rewarding values iterated for a plurality of times.
Further, the unmanned aerial vehicle is a fixed wing unmanned aerial vehicle;
the cradle head carries a monocular camera; the cradle head has two degrees of freedom and is used for tracking a target by adjusting a pitch angle and a yaw angle;
and fixedly connecting the cloud deck to the fixed wing unmanned aerial vehicle.
Further, the identifying the relative offset position of the tracked object in the object image relative to the image center area includes:
dividing the target image into a plurality of grid cells using a lightweight model; the plurality of grid cells detect and locate an object containing the grid;
estimating probability distribution of a target tag and a target corresponding to the target tag in each cell of the grid cells by using the plurality of grid cells;
performing non-maximum suppression according to the probability distribution to obtain a target detection result of a target; the target detection result comprises the size and the center position of a target;
receiving an interesting target selected by a user aiming at the target of the target image as a tracked target;
the relative offset position of the current image frame of the tracked object in the object image relative to the central region of the image is tracked.
Further, the lightweight model includes a network structure using YOLO algorithm, wherein the network structure comprises 24 convolutional layers and 2 fully-connected layers;
the dividing the target image into a plurality of grid cells using a lightweight model includes:
extracting image features of the target image by using the convolutional layer through the YOLO algorithm;
the estimating, by using the plurality of grid cells, a probability distribution of a target tag and a target corresponding to the target tag appearing in each cell of the grid cells, includes:
predicting the image features through the full connection layer, and converting the image features into predicted values of the targets; the predicted value comprises the boundary coordinates of the target and the probability of the target label corresponding to the target.
Further, the tracking the relative offset position of the current image frame of the tracked object in the object image relative to the central area of the image comprises:
a nuclear correlation filtering algorithm is adopted, and related information is determined according to the information of the current image frame and the information of the previous image frame in the target image;
and carrying out correlation calculation with the image frame after the current image frame which is newly acquired by utilizing the correlation information to obtain a tracking result of the tracked target.
The application provides an unmanned aerial vehicle vision tracking system, which comprises:
the target image acquisition module is used for acquiring a target image acquired by a camera on a cradle head fixedly connected with the unmanned aerial vehicle;
a relative offset position identifying module for identifying the relative offset position of the tracked target relative to the central area of the image in the target image;
the control quantity information determining module is used for obtaining control quantity information of the unmanned aerial vehicle and the cradle head according to the relative offset position;
and the joint control module is used for jointly controlling the unmanned aerial vehicle and the cradle head by taking the image center area as a reference according to the control quantity information so as to enable the tracked target to be positioned in the image center area.
The present application provides a drone comprising one or more processors for implementing the method as described in any one of the preceding claims.
The present application provides a computer readable storage medium having stored thereon a program which, when executed by a processor, implements a method as described in any of the above.
In some embodiments, the unmanned aerial vehicle vision tracking method of the application obtains a target image collected by a camera on a cradle head fixedly connected with the unmanned aerial vehicle; identifying a relative offset position of the tracked object in the target image relative to a central region of the image; according to the relative offset position, obtaining control quantity information of the unmanned aerial vehicle and the cradle head through a reinforcement learning algorithm; and according to the control quantity information, taking the image center area as a reference, and jointly controlling the unmanned aerial vehicle and the cradle head to enable the tracked target to be positioned in the image center area. In this way, according to the relative offset position of the tracked target in the target image relative to the central area of the image, the control quantity information of the unmanned aerial vehicle and the cradle head is obtained, the model of the unmanned aerial vehicle and the model of the cradle head do not need to be established, the gesture of the cradle head and the gesture of the unmanned aerial vehicle do not need to be coupled, and decoupling is not needed, so that decoupling difficulty does not exist.
Drawings
Fig. 1 is a schematic flow chart of a visual tracking method of an unmanned aerial vehicle according to an embodiment of the application;
fig. 2 is a schematic diagram of a tracking flow of a tracked target in the unmanned aerial vehicle visual tracking method shown in fig. 1;
fig. 3 is a schematic flow chart of a kernel-related filtering algorithm in the unmanned aerial vehicle visual tracking method shown in fig. 2;
fig. 4 is a schematic structural diagram of a visual tracking system of an unmanned aerial vehicle according to an embodiment of the present application;
fig. 5 is a block diagram of a unmanned aerial vehicle according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with one or more embodiments of the present specification. Rather, they are merely examples of apparatus and methods consistent with aspects of one or more embodiments of the present description as detailed in the accompanying claims.
It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than described in this specification. Furthermore, individual steps described in this specification, in other embodiments, may be described as being split into multiple steps; while various steps described in this specification may be combined into a single step in other embodiments.
In order to solve the technical problem of greater decoupling difficulty, the embodiment of the application provides an unmanned aerial vehicle vision tracking method. The method comprises the steps of obtaining a target image collected by a camera on a cradle head fixedly connected with an unmanned aerial vehicle; identifying a relative offset position of the tracked object in the target image relative to a central region of the image; according to the relative offset position, obtaining control quantity information of the unmanned aerial vehicle and the cradle head through a reinforcement learning algorithm; and according to the control quantity information, taking the image center area as a reference, and jointly controlling the unmanned aerial vehicle and the cradle head to enable the tracked target to be positioned in the image center area.
According to the embodiment of the application, the control quantity information of the unmanned aerial vehicle and the cradle head is obtained according to the relative offset position of the tracked target in the target image relative to the central area of the image, the model of the unmanned aerial vehicle and the model of the cradle head are not required to be established, the coupling between the cradle head gesture and the unmanned aerial vehicle gesture is not required, and the decoupling difficulty is avoided. The tracking speed of the tracked target is high, the precision is high, and the robustness is high.
Fig. 1 is a schematic flow chart of a visual tracking method of an unmanned aerial vehicle according to an embodiment of the application.
As shown in fig. 1, the unmanned aerial vehicle vision tracking method may include the following steps 110 to 140:
step 110, acquiring a target image acquired by a camera on a cradle head fixedly connected with the unmanned aerial vehicle.
The target image may reflect content collected by the pan-tilt camera, and the content may include a target, so as to achieve subsequent target tracking. The target image may include, but is not limited to, one or more of a ground target image and an overhead target image.
Wherein, unmanned aerial vehicle can be fixed wing unmanned aerial vehicle; the cradle head carries a monocular camera; the cradle head has two degrees of freedom, and is used for tracking a target by adjusting a pitch angle and a yaw angle. And fixedly connecting the cradle head on the fixed wing unmanned aerial vehicle. Therefore, compared with a multi-rotor unmanned aerial vehicle, the fixed-wing unmanned aerial vehicle can fly according with the dynamics principle, so that the flying with a higher speed can be realized, and the object with a higher moving speed on the ground can be tracked. Meanwhile, the fixed wing unmanned aerial vehicle has long standby time, so that long-time tracking can be realized. In addition, the flight control system of the fixed wing unmanned aerial vehicle adopts the flight principle similar to an airplane, so that the stability can be realized.
Furthermore, the fixed wing unmanned aerial vehicle flies to the upper space near the target, and performs uniform spiral flight, and simultaneously performs target searching and identification. The position of the target in the image is thus subsequently recognized and detected in real time by means of a lightweight YOLO detection network, also referred to as the YOLO model hereinafter.
At step 120, the relative offset position of the tracked object in the target image with respect to the central region of the image is identified.
The tracked objects may refer to one or more objects in an object image, which are selected by a user from all objects in the object image.
And 130, obtaining control quantity information of the unmanned aerial vehicle and the cradle head according to the relative offset position.
The control quantity is used for an optimal strategy for controlling the unmanned aerial vehicle and the cradle head so as to track the tracked target. The control amount may include one or more of a yaw rate of the unmanned aerial vehicle and a target pose of the pan/tilt. Furthermore, the optimal strategy for controlling the unmanned aerial vehicle and the cradle head is obtained by using a reinforcement learning algorithm. According to the fixed wing unmanned aerial vehicle target tracking method based on reinforcement learning, ground image information is collected through the monocular vision sensor mounted on the two-degree-of-freedom cradle head, target detection is carried out through the lightweight YOLO network, and further the yaw rate, the cradle head pitch angle and the cradle head yaw angle of the fixed wing unmanned aerial vehicle are controlled simultaneously through the reinforcement learning algorithm, so that long-time stable tracking of a ground target is achieved. The details are as follows.
And 140, based on the control quantity information and taking the image center area as a reference, jointly controlling the unmanned aerial vehicle and the cradle head to enable the tracked target to be positioned in the image center area.
The image center area is used for enabling the tracked object to be located in the image center area as much as possible so as to better track and shoot the tracked object. The image center region may be a region where an image center point is located, or the image center region may be a predetermined region extending outward from the center of the image center point.
The step 140 may further include establishing a joint equation of motion of the fixed-wing unmanned aerial vehicle and the two-degree-of-freedom cradle head by:
wherein V is t Is the navigation speed of the unmanned aerial vehicle, V x And V y Components along the x-axis and y-axis of the inertial coordinate system, respectively;yaw angle of unmanned aerial vehicle, theta p And theta t Respectively a yaw angle and a pitch angle of the cradle head. Three control amounts when the fixed wing unmanned aerial vehicle implements target tracking are determined from the control amounts: />And take them as the action variables of the subsequent reinforcement learning algorithm.
Fig. 2 is a schematic diagram of a tracking flow of a tracked target in the unmanned aerial vehicle visual tracking method shown in fig. 1.
As shown in fig. 2, the step 120 may further include the following steps 121 to 125:
step 121, dividing the target image into a plurality of grid cells by using a lightweight model; a plurality of grid cells detect and locate an object containing the grid.
Wherein the target may be represented by a target detection box. The lightweight model described above may include, but is not limited to, a lightweight YOLO model for achieving target detection. In this way, the user selects the target of interest from the targets obtained according to the YOLO algorithm as the tracked target, and then tracks the position (sigma) of the selected tracked target in the image in real time through the kernel correlation filtering algorithm x ,σ y )。
Further, the lightweight model comprises a network structure using a YOLO algorithm, wherein the network structure comprises 24 convolution layers and 2 full connection layers. Identifying and detecting the position of the object in the image by the YOLO detection network: and extracting image characteristics by adopting a convolution network, and obtaining a predicted value through a full connection layer. The network structure contains 24 convolutional layers and 2 fully-connected layers. For each layer, a corresponding activation function is employed. The detection results are the size and the center position of each target. The specific implementation is as follows: first, a sample dataset of the ground target image of interest is collected and labeled with a labeling tool. Second, modify data files, model files, weight files, etc. And running a training program to train. And obtaining a final weight file after training. Thirdly, real-time identification of ground targets is carried out by utilizing the weight file obtained through training and combining image information acquired by an unmanned aerial vehicle-mounted vision sensor.
Step 122, estimating probability distribution of the target label and the target corresponding to the target label in each cell of the grid cells by using a plurality of grid cells.
The step 121 may include the following steps (1); accordingly, the step 122 may include the following steps (2):
(1) and extracting the image characteristics of the target image by using a convolution layer through a YOLO algorithm. (2) Predicting the image features through the full connection layer, and converting the image features into predicted values of targets; the predicted value includes the boundary coordinates of the target and the probability of the target tag corresponding to the target. Thus, the user selects the target of interest from the target detection results to track according to the target detection results obtained by the YOLO algorithm. The tracking algorithm adopts a kernel correlation filtering method, a correlation filter is trained according to the information of the current image frame and the information of the previous image frame, then a response image is obtained by convolving the characteristics of the target, and then the position of the target is determined according to the response image. The generation of training samples is performed based on a circulant matrix. Thus, a lightweight YOLO algorithm is used for target detection. Reducing the number of model parameters by deleting unimportant convolution kernels, thereby reducing the size of the model; and then, a convolution network is adopted to extract image characteristics, and the position of a target in the image is recognized and detected in real time.
Step 123, performing non-maximum suppression according to the probability distribution to obtain a target detection result of the target; the target detection result includes the size and center position of the target.
Predicting the target according to probability distribution through non-maximum suppression to obtain a target detection result of the target; the target detection result includes the size and center position of the target as follows: in a first step, the probability distribution is identified. And 2, removing the probability smaller than a preset threshold value to obtain a target detection result of the target. By way of example, the present application budgets the 10 targets needed, without non-maximum suppression, 15 targets may be identified; and removing the 5 targets which are identified by non-maximum value inhibition, and finally obtaining the 10 required targets. In this way, invalid calculations during detection can be reduced.
And 124, receiving the target of interest selected by the user aiming at the target of the target image as a tracked target.
At step 125, the relative offset position of the current image frame of the tracked object in the target image with respect to the central region of the image is tracked.
Fig. 3 is a flow chart of a kernel-related filtering algorithm in the unmanned aerial vehicle visual tracking method shown in fig. 2.
As shown in fig. 3, the step 125 may further include determining the relevant information according to the information of the current image frame and the information of the previous image frame in the target image using a kernel correlation filtering algorithm; and performing correlation calculation with the image frame after the newly acquired current image frame by utilizing the correlation information to obtain a tracking result of the tracked target. Therefore, the unmanned aerial vehicle can track the target under the complex ground background by adopting a kernel correlation filter algorithm, and can solve the problems of scale change, rotation change and the like of the ground target. In addition, multi-objective tracking can be performed by means of a kernel correlation filtering algorithm.
Continuing with fig. 1 and 2, the step 130 may include inputting the relative offset position, the current pose of the pan-tilt, the flight information, and the pose of the unmanned aerial vehicle into the unmanned aerial vehicle control model, and outputting the control amount information of the unmanned aerial vehicle and the pan-tilt. The unmanned aerial vehicle control model is obtained by training a target image sample formed by visual information and physical parameters of an unmanned aerial vehicle flight environment.
Further, the step 130 may obtain the control amount information by the following steps 1 to 5:
step 1, using the relative offset position, the current pose of the cradle head, flight information and the pose of the unmanned aerial vehicle as observables of a reinforcement learning algorithm; wherein the observed quantity comprises a state observation space; the state space comprises the unmanned aerial vehicle gesture, the flying speed, the flying height and the current gesture of the cradle head; the action space comprises the yaw rate of the unmanned aerial vehicle, the yaw rate of the cradle head and the pitch rate of the cradle head.
And obtaining the optimal strategy for controlling the unmanned aerial vehicle and the cradle head by using a reinforcement learning algorithm, wherein the specific Markov decision process tuple is selected as follows.
State observation space:wherein->Respectively a pitch angle, a roll angle and a yaw angle of the fixed wing unmanned aerial vehicle; v (V) t The navigation speed of the unmanned aerial vehicle is the navigation speed of the unmanned aerial vehicle; h is the flying height of the unmanned aerial vehicle; θ p 、θ t Is the yaw angle and pitch angle of the cradle head.
Action space:wherein->Yaw rate for the drone; u (u) p Yaw rate for cradle head; u (u) t Is the pitch angle rate of the cradle head.
Bonus function:wherein sigma xe Sum sigma ye The deviation of the selected tracked object in the x-direction and the y-direction of the image, respectively. The specific calculation formula is as follows: />Where W is the image width and H is the image height.
And (5) carrying out loop iteration and learning an optimal strategy. The loop iteration process is as follows:
1) Initializing: giving an initial strategy pi;
2) Solving a linear equation, and calculating a value function V corresponding to pi:
where s is the state, R is the value of the bonus function, T is the value of the transfer function, V is the value of the state value function, a is the action, and s' is the transferred state.
3) Based on the calculated value function V, the strategy is updated according to the following formula:
π(s)＝argmax a {R(s,a)+γ∑ s′ T(s,a,s′)V(s′)}。
wherein argmax a In order to maximize the value function, γ is the discount factor.
4) Steps 2) and 3) are repeated until convergence conditions, i.e. old action = pi(s).
5) The iteration number does not exceed |A| |S| Where A is the size of the action space and S is the size of the state space.
Therefore, the fixed wing unmanned aerial vehicle and the cradle head are controlled in a combined mode through the reinforcement learning algorithm, and long-time stable spiral tracking of the fixed wing unmanned aerial vehicle on a target with a high moving speed on the ground is achieved.
Step 2, taking the yaw rate of the unmanned aerial vehicle and the target pose of the cradle head as actions of a reinforcement learning algorithm; wherein the actions include an action space.
Step 3, the square of the relative offset position is taken as a penalty term for the reward function in the reinforcement learning algorithm.
And 4, taking the navigation speed of the unmanned aerial vehicle as a rewarding item of a rewarding function.
And 5, obtaining control quantity information of the unmanned aerial vehicle and the cradle head through a reinforcement learning algorithm by using observance quantity, action and rewarding functions. Therefore, unmanned aerial vehicle control adaptability based on reinforcement learning algorithm is strong, and control strategies do not need to be designed in advance: unmanned aerial vehicle control based on reinforcement learning algorithm can adapt to the control problem of uncertainty environment and higher complexity, and has good adaptability to the problems of nonlinearity and non-certainty; and the combined control strategy of the unmanned aerial vehicle and the cradle head does not need to be designed in advance, and the model of the unmanned aerial vehicle and the model of the cradle head do not need to be established in advance, so that the optimal strategy can be obtained through interactive learning with the environment, and the method has higher flexibility.
The step 5 may include determining the prize value of the prize function in a plurality of iterations under the condition of the observance and the action;
and selecting a state space and an action space which correspond to the maximum accumulated rewarding value as control quantity information through the rewarding values of multiple iterations.
Wherein, the unmanned aerial vehicle control model is obtained through the training in various modes:
in one mode of obtaining the unmanned aerial vehicle control model, randomizing visual information in an analog environment and physical parameters of the unmanned aerial vehicle flight environment by a domain randomization method, and increasing the sample size of the unmanned aerial vehicle control model; based on a reinforcement learning algorithm, training the unmanned aerial vehicle control model by using the increased sample size to obtain the unmanned aerial vehicle control model; the unmanned aerial vehicle control model is used for migrating to a real unmanned aerial vehicle. In this way, by the domain randomization method, visual information in the simulation environment and physical parameters related to the unmanned aerial vehicle flight environment and the like are randomized, so that the intelligent agent learns in the simulation environment with random variation. Therefore, the unmanned aerial vehicle control model training sampling efficiency based on the reinforcement learning algorithm is high, the training is safe, and the problems that the sampling efficiency is low, the sample collection is time-consuming, damage is easily brought to an unmanned aerial vehicle or the surrounding environment and the like when the unmanned aerial vehicle control model is directly trained in the real environment can be solved.
For example, simulation training is performed in an Isaac Gym simulator, starting a physical engine in which a plurality of sub-agents, i.e., fixed-wing drones, are run in parallel. After the fixed-wing unmanned aerial vehicle control model is trained, transfer learning is carried out, and the obtained fixed-wing unmanned aerial vehicle control model is transferred to an actual fixed-wing unmanned aerial vehicle platform, so that the fixed-wing unmanned aerial vehicle can track a ground target in a spiral mode. Thus, simulation training is performed through the Isaac ym simulation environment, and the safety and the iteration speed in the learning process are improved. And then, performing migration learning through domain randomization and other means, and migrating the obtained model to an actual fixed-wing unmanned aerial vehicle platform, thereby realizing the spiral tracking of the fixed-wing unmanned aerial vehicle on a ground target.
In another mode of obtaining the unmanned aerial vehicle control model, a target image sample formed by visual information and physical parameters of the unmanned aerial vehicle flight environment in a real environment is obtained; based on a reinforcement learning algorithm, training an unmanned aerial vehicle control model by utilizing the increased sample size to obtain the unmanned aerial vehicle control model; the unmanned aerial vehicle control model is used for migrating to a real unmanned aerial vehicle. Thus, the control method is more suitable for the control of the surrounding actual environment. Therefore, the unmanned plane control model training sampling efficiency based on the reinforcement learning algorithm is high, and the training is safe.
According to the embodiment of the application, the problems that the target tracking method in the related technology has larger proposal error, the target is easy to deviate from the field of view of the on-board visual sensor, the accurate modeling and the design control rate are difficult to realize, and the long-time stable tracking of the fixed-wing unmanned aerial vehicle on the ground moving target can be realized.
Fig. 4 is a schematic structural diagram of a visual tracking system of an unmanned aerial vehicle according to an embodiment of the present application.
As shown in fig. 4, the unmanned aerial vehicle vision tracking system may include the following modules:
the target image acquisition module 41 is used for acquiring a target image acquired by a camera on a cradle head fixedly connected with the unmanned aerial vehicle;
a relative offset position identifying module 42 for identifying a relative offset position of the tracked object in the target image with respect to the center region of the image;
the control amount information determining module 43 is configured to obtain control amount information of the unmanned aerial vehicle and the pan-tilt according to the relative offset position;
and the joint control module 44 is used for jointly controlling the unmanned aerial vehicle and the cradle head based on the image central area according to the control quantity information so as to enable the tracked target to be positioned in the image central area.
The implementation process of the functions and actions of each module in the device is specifically detailed in the implementation process of the corresponding steps in the method, so that the same technical effects can be achieved, and the detailed description is omitted herein.
The unmanned aerial vehicle provided by the embodiment of the application comprises the unmanned aerial vehicle vision tracking system.
Fig. 5 is a block diagram of a unmanned aerial vehicle 50 according to an embodiment of the present application.
As shown in fig. 5, the drone 50 includes one or more processors 51 for implementing the drone vision tracking method described above.
In some embodiments, the drone 50 may include a computer-readable storage medium 59, which computer-readable storage medium 59 may store programs that may be invoked by the processor 51, and may include a non-volatile storage medium. In some embodiments, the drone 50 may include a memory 58 and an interface 57. In some embodiments, the drone 50 may also include other hardware depending on the application.
The computer readable storage medium 59 of an embodiment of the present application has stored thereon a program which, when executed by the processor 51, is adapted to carry out the unmanned aerial vehicle vision tracking method as described above.
The present application may take the form of a computer program product embodied on one or more computer-readable storage media 59 (including but not limited to disk storage, CD-ROM, optical storage, etc.) having program code embodied therein. Computer-readable storage media 59 include both non-transitory and non-transitory, removable and non-removable media, and may be implemented in any method or technology for information storage. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer readable storage media 59 include, but are not limited to: phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, may be used to store information that may be accessed by the computing device.
In the present application, unless explicitly specified and limited otherwise, the terms "mounted," "connected," "secured," and the like are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; either directly or indirectly, through intermediaries, or both, may be in communication with each other or in interaction with each other, unless expressly defined otherwise. The specific meaning of the above terms in the present application can be understood by those of ordinary skill in the art according to the specific circumstances.
The foregoing description of the preferred embodiments is provided for the purpose of illustration only, and is not intended to limit the scope of the disclosure, since any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the disclosure are intended to be included within the scope of the disclosure.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, the phrase "comprising one … …" does not exclude the presence of additional identical elements in a process, method, article, or apparatus that comprises the depicted element.
Claims (11)
1. The unmanned aerial vehicle vision tracking method is characterized by comprising the following steps of:
acquiring a target image acquired by a camera on a cradle head fixedly connected with the unmanned aerial vehicle;
identifying a relative offset position of the tracked object in the object image relative to the image center region;
obtaining control quantity information of the unmanned aerial vehicle and the cradle head according to the relative offset position;
and according to the control quantity information, taking the image center area as a reference, and jointly controlling the unmanned aerial vehicle and the cradle head to enable the tracked target to be positioned in the image center area.
2. The unmanned aerial vehicle vision tracking method of claim 1, wherein the obtaining control amount information of the unmanned aerial vehicle and a cradle head according to the relative offset position comprises:
inputting the relative offset position, the current position of the cradle head, flight information and the position of the unmanned aerial vehicle into an unmanned aerial vehicle control model, and outputting to obtain control quantity information of the unmanned aerial vehicle and the cradle head; the unmanned aerial vehicle control model is obtained by training a target image sample formed by visual information and physical parameters of an unmanned aerial vehicle flight environment.
3. The unmanned aerial vehicle vision tracking method of claim 2, wherein the method further comprises: the unmanned aerial vehicle control model is obtained through training in the following mode: the visual information in the simulation environment and the physical parameters of the unmanned aerial vehicle flight environment are randomized through a domain randomization method, so that the sample size of the unmanned aerial vehicle control model is increased; based on a reinforcement learning algorithm, training an unmanned aerial vehicle control model by using the increased sample size to obtain the unmanned aerial vehicle control model; the unmanned aerial vehicle control model is used for migrating to the real unmanned aerial vehicle.
4. The method for visual tracking of an unmanned aerial vehicle according to claim 2, wherein inputting the relative offset position, the current pose of the pan-tilt, the flight information and the pose of the unmanned aerial vehicle into an unmanned aerial vehicle control model, and outputting control quantity information of the unmanned aerial vehicle and the pan-tilt comprises:
taking the relative offset position, the current pose of the cradle head, the flight information and the pose of the unmanned aerial vehicle as observables of a reinforcement learning algorithm;
taking the yaw angular velocity of the unmanned aerial vehicle and the target pose of the cradle head as actions of the reinforcement learning algorithm;
taking the square of the relative offset position as a penalty term for a reward function in the reinforcement learning algorithm;
taking the navigation speed of the unmanned aerial vehicle as a reward item of the reward function;
and obtaining the control quantity information of the unmanned aerial vehicle and the cradle head through the reinforcement learning algorithm by using the observed quantity, the action and the reward function.
5. The method of claim 4, wherein the obtaining the control amount information of the unmanned aerial vehicle and the pan-tilt by the reinforcement learning algorithm using the observables, the actions, and the bonus functions comprises:
determining a reward value of the reward function in a plurality of iterations under the conditions of the observed quantity and the action;
and selecting a state space and an action space which correspond to the largest accumulated rewarding value as the control quantity information through the rewarding values iterated for a plurality of times.
6. The unmanned aerial vehicle vision tracking method of any of claims 1 to 5, wherein the unmanned aerial vehicle is a fixed-wing unmanned aerial vehicle;
the cradle head carries a monocular camera; the cradle head has two degrees of freedom and is used for tracking a target by adjusting a pitch angle and a yaw angle;
and fixedly connecting the cloud deck to the fixed wing unmanned aerial vehicle.
7. The unmanned aerial vehicle vision tracking method of any of claims 1 to 5, wherein the identifying the relative offset position of the tracked object in the target image with respect to the image center region comprises:
dividing the target image into a plurality of grid cells using a lightweight model; the plurality of grid cells detect and locate an object containing the grid;
estimating probability distribution of a target tag and a target corresponding to the target tag in each cell of the grid cells by using the plurality of grid cells;
performing non-maximum suppression according to the probability distribution to obtain a target detection result of a target; the target detection result comprises the size and the center position of a target;
receiving an interesting target selected by a user aiming at the target of the target image as a tracked target;
the relative offset position of the current image frame of the tracked object in the object image relative to the central region of the image is tracked.
8. The unmanned aerial vehicle vision tracking method of claim 7, wherein the lightweight model comprises a network structure using a YOLO algorithm, wherein the network structure comprises 24 convolutional layers and 2 fully-connected layers; the dividing the target image into a plurality of grid cells using a lightweight model includes: extracting image features of the target image by using the convolutional layer through the YOLO algorithm; the estimating, by using the plurality of grid cells, a probability distribution of a target tag and a target corresponding to the target tag appearing in each cell of the grid cells, includes: predicting the image features through the full connection layer, and converting the image features into predicted values of the targets; the predicted value comprises the boundary coordinates of the target and the probability of the target label corresponding to the target;
and/or the number of the groups of groups,
the tracking of the relative offset position of the current image frame of the tracked object in the object image relative to the central area of the image comprises the following steps: a nuclear correlation filtering algorithm is adopted, and related information is determined according to the information of the current image frame and the information of the previous image frame in the target image; and carrying out correlation calculation with the image frame after the current image frame which is newly acquired by utilizing the correlation information to obtain a tracking result of the tracked target.
9. An unmanned aerial vehicle vision tracking system, comprising:
the target image acquisition module is used for acquiring a target image acquired by a camera on a cradle head fixedly connected with the unmanned aerial vehicle;
a relative offset position identifying module for identifying the relative offset position of the tracked target relative to the central area of the image in the target image;
the control quantity information determining module is used for obtaining control quantity information of the unmanned aerial vehicle and the cradle head according to the relative offset position;
and the joint control module is used for jointly controlling the unmanned aerial vehicle and the cradle head by taking the image center area as a reference according to the control quantity information so as to enable the tracked target to be positioned in the image center area.
10. A drone comprising one or more processors configured to implement the drone vision tracking method of any one of claims 1-8.
11. A computer readable storage medium, having stored thereon a program which, when executed by a processor, implements the unmanned aerial vehicle vision tracking method of any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311186940.XA CN117130383B (en) | 2023-09-14 | 2023-09-14 | Unmanned aerial vehicle vision tracking method and system, unmanned aerial vehicle and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311186940.XA CN117130383B (en) | 2023-09-14 | 2023-09-14 | Unmanned aerial vehicle vision tracking method and system, unmanned aerial vehicle and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117130383A true CN117130383A (en) | 2023-11-28 |
| CN117130383B CN117130383B (en) | 2024-03-29 |
Family
ID=88858159
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311186940.XA Active CN117130383B (en) | 2023-09-14 | 2023-09-14 | Unmanned aerial vehicle vision tracking method and system, unmanned aerial vehicle and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117130383B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111596693A (en) * | 2020-06-17 | 2020-08-28 | 中国人民解放军国防科技大学 | Ground target tracking control method and system of unmanned aerial vehicle based on pan-tilt camera |
| CN111738189A (en) * | 2020-06-29 | 2020-10-02 | 广东电网有限责任公司 | Transmission line crimping hardware inspection control method, device, terminal and medium |
| CN111932588A (en) * | 2020-08-07 | 2020-11-13 | 浙江大学 | Tracking method of airborne unmanned aerial vehicle multi-target tracking system based on deep learning |
| CN113408510A (en) * | 2021-08-23 | 2021-09-17 | 中科方寸知微(南京)科技有限公司 | Transmission line target deviation rectifying method and system based on deep learning and one-hot coding |
| CN114281101A (en) * | 2021-12-03 | 2022-04-05 | 南京航空航天大学 | Unmanned aerial vehicle and holder interference source joint search method based on reinforcement learning |
-
2023
- 2023-09-14 CN CN202311186940.XA patent/CN117130383B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111596693A (en) * | 2020-06-17 | 2020-08-28 | 中国人民解放军国防科技大学 | Ground target tracking control method and system of unmanned aerial vehicle based on pan-tilt camera |
| CN111738189A (en) * | 2020-06-29 | 2020-10-02 | 广东电网有限责任公司 | Transmission line crimping hardware inspection control method, device, terminal and medium |
| CN111932588A (en) * | 2020-08-07 | 2020-11-13 | 浙江大学 | Tracking method of airborne unmanned aerial vehicle multi-target tracking system based on deep learning |
| CN113408510A (en) * | 2021-08-23 | 2021-09-17 | 中科方寸知微(南京)科技有限公司 | Transmission line target deviation rectifying method and system based on deep learning and one-hot coding |
| CN114281101A (en) * | 2021-12-03 | 2022-04-05 | 南京航空航天大学 | Unmanned aerial vehicle and holder interference source joint search method based on reinforcement learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117130383B (en) | 2024-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111932588B (en) | Tracking method of airborne unmanned aerial vehicle multi-target tracking system based on deep learning | |
| Cesetti et al. | A vision-based guidance system for UAV navigation and safe landing using natural landmarks | |
| Polvara et al. | Autonomous vehicular landings on the deck of an unmanned surface vehicle using deep reinforcement learning | |
| CN114048889A (en) | Aircraft trajectory prediction method based on long-term and short-term memory network | |
| CN108594848A (en) | A kind of unmanned plane of view-based access control model information fusion autonomous ship method stage by stage | |
| Zhang et al. | An intruder detection algorithm for vision based sense and avoid system | |
| CN106875403B (en) | A kind of imitative hawkeye visual movement object detection method for air refuelling | |
| CN111831010A (en) | Unmanned aerial vehicle obstacle avoidance flight method based on digital space slice | |
| CN105066998A (en) | Quantum-behaved pigeon inspired optimization-based unmanned aerial vehicle autonomous aerial refueling target detection method | |
| Sandström et al. | Fighter pilot behavior cloning | |
| CN112733971B (en) | Pose determination method, device and equipment of scanning equipment and storage medium | |
| CN117130383B (en) | Unmanned aerial vehicle vision tracking method and system, unmanned aerial vehicle and readable storage medium | |
| CN115861860B (en) | Target tracking and positioning method and system for unmanned aerial vehicle | |
| CN105930766A (en) | Unmanned plane | |
| CN117036989A (en) | Miniature unmanned aerial vehicle target recognition and tracking control method based on computer vision | |
| CN107742295A (en) | A kind of cube star docking reconstructing method of view-based access control model | |
| Yuan et al. | Eagle vision-based coordinate landing control framework of unmanned aerial vehicles on an unmanned surface vehicle | |
| CN112489118B (en) | Method for quickly calibrating external parameters of airborne sensor group of unmanned aerial vehicle | |
| CN105760813A (en) | Unmanned aerial vehicle target detection method based on plant branch and root evolution behaviors | |
| Depenbusch | Atmospheric energy harvesting for small uninhabited aircraft by gust soaring | |
| Lv et al. | Target recognition algorithm based on optical sensor data fusion | |
| CN114428517B (en) | End-to-end autonomous landing control method for unmanned plane and unmanned ship cooperative platform | |
| Li et al. | Object recognition through UAV observations based on YOLO and generative adversarial network | |
| CN114326759B (en) | Multi-agent formation control method and device and multi-agent system | |
| CN117593340B (en) | Method, device and equipment for determining swing angle of carrier rocket servo mechanism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |