CN114020024B - Unmanned aerial vehicle path planning method based on Monte Carlo tree search - Google Patents
Unmanned aerial vehicle path planning method based on Monte Carlo tree search Download PDFInfo
- Publication number
- CN114020024B CN114020024B CN202111305350.5A CN202111305350A CN114020024B CN 114020024 B CN114020024 B CN 114020024B CN 202111305350 A CN202111305350 A CN 202111305350A CN 114020024 B CN114020024 B CN 114020024B
- Authority
- CN
- China
- Prior art keywords
- node
- unmanned aerial
- aerial vehicle
- monte carlo
- carlo tree
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000012549 training Methods 0.000 claims abstract description 45
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 23
- 230000000875 corresponding effect Effects 0.000 claims abstract description 18
- 238000010845 search algorithm Methods 0.000 claims abstract description 16
- 230000009471 action Effects 0.000 claims description 26
- 238000005265 energy consumption Methods 0.000 claims description 22
- 230000008569 process Effects 0.000 claims description 18
- 238000004088 simulation Methods 0.000 claims description 16
- 238000004364 calculation method Methods 0.000 claims description 13
- 230000005540 biological transmission Effects 0.000 claims description 5
- 230000001133 acceleration Effects 0.000 claims description 4
- 239000000654 additive Substances 0.000 claims description 2
- 230000000996 additive effect Effects 0.000 claims description 2
- 239000003990 capacitor Substances 0.000 claims description 2
- 230000005484 gravity Effects 0.000 claims description 2
- 238000004891 communication Methods 0.000 description 7
- 230000008859 change Effects 0.000 description 4
- 230000002787 reinforcement Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000005012 migration Effects 0.000 description 2
- 238000013508 migration Methods 0.000 description 2
- 230000002349 favourable effect Effects 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses an unmanned aerial vehicle path planning method based on Monte Carlo tree search, which is high in algorithm efficiency, good in performance and capable of better adapting to a dynamic environment. The method comprises the following steps: (10) Establishing a Monte Carlo tree, initializing a root node and initializing the position of the unmanned aerial vehicle; (20) Setting the total training times of the Monte Carlo tree search algorithm according to experimental data; (30) Within the set total training times, carrying out search algorithm training on the Monte Carlo tree, enabling Monte Carlo tree parameters to iterate according to specific steps, and enabling the unmanned aerial vehicle to make corresponding actions; (40) When the training times are equal to the total training times, finishing the training to obtain a trained Monte Carlo tree; and continuously selecting the child node with the maximum UCT value downwards by using the UCT algorithm from the root node until reaching one leaf node according to the tree structure of the trained Monte Carlo tree, and executing corresponding actions by the unmanned aerial vehicle according to the selected node to obtain the optimal unmanned aerial vehicle path.
Description
Technical Field
The invention belongs to the technical field of unmanned aerial vehicle path planning, and particularly relates to an unmanned aerial vehicle path planning method based on Monte Carlo tree search.
Background
In recent years, drones have proven to be one of the most challenging and potential technologies in aeronautics. Due to its high mobility and low cost, drones have gained widespread use in the field of communications over the past few decades. Meanwhile, the task of unmanned aerial vehicle path planning is becoming one of the key technologies of unmanned aerial vehicles, and has been widely studied by scholars all over the world. The main objective of unmanned aerial vehicle path planning is to design an optimal flight path pointing to a target, that is, to satisfy the performance requirements of the unmanned aerial vehicle and also satisfy the given target conditions.
Drone-assisted wireless communication may provide wireless connectivity to devices that do not have communication infrastructure coverage, e.g., areas that cannot cover infrastructure due to severe architectural shadows, natural disaster damage, etc. In a communication system, a deployed drone may operate as a mobile relay or as a flying base station. In certain military scenarios, drone-assisted relay may provide reliable wireless connectivity between two or more remote devices. Since the size and weight of the wireless network devices are greatly reduced by the drone and the communication with the ground devices employs line of sight (LoS), this is very attractive to wireless device providers.
The mobile edge computing is to deploy task computing and storage resource services at the edge of a mobile network, provide cloud computing services for the mobile network and users at the edge, and accelerate the rapid downloading of various contents and applications in the network, thereby providing a network service solution with ultra-low time delay and high bandwidth for the users. After the mobile edge server is deployed, the transmission requirement on the core network can be reduced, and the pressure of the core network is reduced. In a mobile edge computing service scenario, the drone may serve as a mobile cloud server to provide services to users at the edge of the network. When the current task downloading speed of the user is low, the user can upload the task to the unmanned aerial vehicle, and the unmanned aerial vehicle finishes the task unloading and calculating.
The communication distance between the user and the unmanned aerial vehicle can be shortened through the appropriate flight path, and the improvement on the system performance is very important. In a moving edge computing scenario, in addition to designing the optimal flight trajectory of the drone, migration throughput is often maximized as much as possible. Furthermore, the path planning problem is also limited by various constraints, such as energy consumption constraints, user quality of service constraints, power constraints, etc., which are typically NP-hard problems. However, some conventional geometric algorithms such as an a-star algorithm, a genetic algorithm, an ant colony algorithm and the like have great limitations, and it becomes very difficult to solve the problem with constraint conditions, so that researchers apply machine learning and reinforcement learning to the field of unmanned plane path planning to solve the path planning problem under these complex conditions.
The path planning technology of drones in static environments is becoming more and more mature, and the path planning problem becomes more and more complex when the environment changes. For example, real-time changing user locations, user mission requirements, drone-user channel conditions can affect the design of drone flight trajectories. The task demand of the user in each time slot affects the migration throughput of the system, and the movement of the user affects the state of the communication channel and the energy consumption of the drone. Traditional reinforcement learning methods, such as Q-learning, can take considerable time cost and space cost when handling a large number of state-action pairs, which will reduce the efficiency of unmanned aerial vehicle path planning and cannot be directly applied to dynamic environments.
Disclosure of Invention
The invention aims to provide an unmanned aerial vehicle path planning method based on Monte Carlo tree search, which is high in algorithm efficiency, good in performance and capable of better adapting to a dynamic environment.
The technical solution for realizing the purpose of the invention is as follows:
an unmanned aerial vehicle path planning method based on Monte Carlo tree search comprises the following steps:
(10) Initializing unmanned aerial vehicles and Monte Carlo trees: establishing a Monte Carlo tree, initializing a root node and initializing the position of the unmanned aerial vehicle;
(20) Setting the total training times: setting the total training times of the Monte Carlo tree search algorithm according to experimental data;
(30) Training a Monte Carlo tree search algorithm: within the set total training times, carrying out search algorithm training on the Monte Carlo tree, enabling Monte Carlo tree parameters to iterate according to specific steps, and enabling the unmanned aerial vehicle to make corresponding actions according to the specific steps;
(40) Obtaining an optimal unmanned aerial vehicle path: when the training times are equal to the total training times, finishing the training to obtain a trained Monte Carlo tree; and continuously selecting a child node with the maximum UCT value downwards by using a UCT algorithm from a root node until reaching a leaf node according to the tree structure of the trained Monte Carlo tree, and executing corresponding actions by the unmanned aerial vehicle according to the selected node to obtain the optimal unmanned aerial vehicle path.
Compared with the prior art, the invention has the remarkable advantages that:
1. more applicable to dynamic environments: in the flight environment of the invention, the requirements of the position of the user, the number of tasks and the change of the channel state are changed along with the change of time, the Monte Carlo tree search can make better action balance in the dynamic unknown environment, and an enlightening solution can be obtained. Compared with a method for recording state-action pairs by using a table for Q-learning, the Monte Carlo tree search uses a tree structure to store the action and the state of the current situation, and a strategy for randomly selecting the action is added, so that the method is more suitable for a dynamically changing environment and solves the problem of unmanned aerial vehicle path planning in a complex dynamic environment;
2. the path planning efficiency is high: compared with other traditional reinforcement learning algorithms, the path planning method can effectively reduce training time, greatly reduces the time complexity of the algorithm in a top-down tree search mode, and is more suitable for real-time unmanned aerial vehicle path planning.
The invention is described in further detail below with reference to the figures and the detailed description.
Drawings
Fig. 1 is a main flow chart of the path planning method based on monte carlo tree search according to the present invention.
FIG. 2 is a flow chart of the training steps of the Monte Carlo tree search algorithm of FIG. 1.
FIG. 3 is a flow chart of the step of calculating the prize value of FIG. 2.
FIG. 4 is a schematic diagram of a Monte Carlo tree search algorithm.
Fig. 4 (1) shows the selection of a node, fig. 4 (2) shows the expansion of a node, fig. 4 (3) shows the simulation of a node, and fig. 4 (4) shows the update of the simulation result upward.
Fig. 5 is an exemplary diagram of a flight trajectory of the drone.
Detailed Description
The path planning method based on the Monte Carlo tree search is implemented based on the following scenes:
and establishing a scene model of mobile edge calculation, wherein the unmanned aerial vehicle serves as a mobile edge server to provide services for a group of users on the ground. For the convenience of calculation, the unmanned aerial vehicle can fly only at given K fixed points, and the flying time is dispersed into M time slots. In each time slot, each user sends a task unloading request to the unmanned aerial vehicle, and the number of tasks of the users obeys Gaussian distribution; the drone may fly from the current fixed point to another fixed point and provide service to one of the users, typically selecting the user closest to the drone. And the position of the user in each time slot is dynamically changed.
As shown in fig. 1, the path planning method based on monte carlo tree search of the present invention includes the following steps: (10) initializing the unmanned aerial vehicle and the Monte Carlo tree: and establishing a Monte Carlo tree, initializing a root node and initializing the position of the unmanned aerial vehicle.
The tree structure initially contains only one root node. The information in each node comprises a state S, a quality value Q, access times N and a father node N p Child node n c And the state S comprises the selected action a, the position l of the user currently served, the residual battery capacity E of the unmanned plane and the like. Action a can only be selected in a given action set a, which is a set of all actions that the drone can perform. In the scenario of the present invention, the action set a is the coordinates of the given K fixed points.
The drone is initially positioned at the location of coordinates (0, 0).
(20) Setting the total training times: and setting the total training times of the Monte Carlo tree search algorithm according to the experimental data.
Setting the total training times of the Monte Carlo tree search algorithm to be N according to conventional experimental data e The parameters of the sub, i.e. Monte Carlo tree, require iteration N e Next, the process is carried out. When the training times do not reach N e The iteration needs to be performed all the time. Preferably, according to experimental experience, N e 500 to 1000 times can be taken.
(30) Training a Monte Carlo tree search algorithm: and (3) within the set total training times, carrying out search algorithm training on the Monte Carlo tree, enabling Monte Carlo tree parameters to iterate according to specific steps, and enabling the unmanned aerial vehicle to make corresponding actions according to the specific steps.
N on Monte Carlo Tree e And (5) performing secondary training. In the training process, the parameters of the tree are continuously iterated according to specific steps of the algorithm, and the unmanned aerial vehicle can make corresponding actions according to the specific steps, so that the actions of the unmanned aerial vehicle can influence the iteration and the change of the parameters.
As shown in fig. 2, the monte carlo tree search algorithm training comprises the following steps:
(31) And (3) selecting a node: from a root node, selecting a child node with the largest UCT value downwards by using a UCT algorithm; the unmanned aerial vehicle executes corresponding actions contained in the node; and continuing to select the child nodes downwards by using the UCT algorithm until reaching a node which is not fully expanded, and stopping the selection of the node.
As shown in fig. 4 (1), a child node having the largest UCT value is selected downward from the root node by the UCT algorithm. The specific formula is as follows:
wherein the constant C is a trade-off factor, Q (N ') is the Q value of the child node, N (N') is the number of accesses of the child node, and N (N) is the number of accesses of the current node.
After the selection of the child node, the drone executes the corresponding actions contained in the node, i.e. flies from the fixed point where it is now located to another fixed point, and serves a user. The selection of child nodes continues down with the UCT algorithm until a node is reached that is not fully expanded. A fully expanded node means that the number of child nodes of the node is equal to the number of actions contained in action set a.
(32) And (3) expanding the nodes: and establishing a new node as a child node of the current node, and randomly selecting an action a' from the action set A to be bound with the new node.
As shown in fig. 4 (2), the current node needs to perform a node expansion operation. And establishing a new node as a child node of the current node, and randomly selecting an action a' from the action set A to be bound with the node, wherein the selected action and the action of the child node of the same layer cannot be repeated.
(33) And (3) simulation of the nodes: and starting from the state corresponding to the new node, performing simulation action selection and the flight process of the unmanned aerial vehicle.
As shown in fig. 4 (3), from the state corresponding to the newly expanded node, the simulated action selection and the flight process of the drone are performed. In the simulation process, all the actions of the unmanned aerial vehicle are randomly selected from the action set A, namely, the unmanned aerial vehicle continuously flies to another random fixed point and selects a user for service until the battery reaches a lower level, and the unmanned aerial vehicle returns to a charging place for charging. In the process, the unmanned aerial vehicle can consume hovering energy, calculation energy and flight energy, wherein the hovering energy consumption is energy consumption generated when the unmanned aerial vehicle hovers over the fixed points when serving the user, the calculation energy consumption is energy consumption generated when the unmanned aerial vehicle is used for task calculation after a task is unloaded to the unmanned aerial vehicle, and the flight energy consumption is energy consumption generated when the unmanned aerial vehicle flies between the fixed points. After the simulation process is completed, the corresponding reward value needs to be calculated to evaluate the whole simulation process. This step does not create new tree nodes in order to update the parameters of the existing tree nodes, which facilitates trajectory optimization and more favorable action selection by the drone.
(34) Calculating the reward value:
the prize value obtained in step (33) is calculated. In the present invention, the reward value is defined to be related to the energy consumption of the drone and the throughput of the user, as shown in fig. 3, the specific calculation method is as follows:
(341) Calculating the energy consumption of the unmanned aerial vehicle:
and (4) calculating the total energy consumption according to the hovering energy, the calculation energy and the flight energy consumed by the unmanned aerial vehicle in the step (33).
The specific calculation formula of hovering energy consumption is as follows:
wherein p is h (t) is the drone hover power,
is the data transmission rate, P, between the ith user and the UAV at the kth fixed point u Representing transmission power, σ 2 Is the power of additive white gaussian noise,
channel power gain, p, of a path loss model 0 Is the channel gain at a reference distance of 1m, H is the altitude of the drone, D uav (t) coordinates of the drone at time slot t, U i (t) is the coordinates of the ith user. S (t) = μ i (t)R i,k (t) Δ tB is the total number of bits of user tasks offloaded by the drone, μ i (t) is the number of tasks for the user, B is the channel bandwidth, Δ t is the drone hover time.
The specific calculation formula for calculating the energy consumption is as follows:
e c (t)＝γ c CS(t)f c 2
wherein, γ c Is effectively a switched capacitor, C is the number of CPU cycles required for each bit calculation, f c Is the frequency of the CPU.
The specific calculation formula of the flight energy consumption is as follows:
wherein, κ 1 、κ 2 Is a constant parameter, a uav The acceleration of the unmanned aerial vehicle during takeoff, and g is the gravity acceleration.
Therefore, the total energy consumption generated by the drone in a single time slot is:
e general assembly ＝e h (t)+e c (t)+e f (t)
(342) Calculating the throughput of the user:
according to the user served in step (33), assuming that the number of task requests of the user is mu i (t) each task has N number of bits b Bit, then user throughput is μ i (t)N b 。
(343) Calculating a specific prize value according to the formula:
the calculation formula of the generated prize value in a single time slot is as follows:
wherein, mu max Is the maximum value of the number of tasks, W max Is the maximum value of the total energy consumption.
The reward value of the whole simulation process in the step (33) is as follows:
where T is the number of slots at the beginning of the simulation process and T is the total number of slots.
(35) Updating the simulation result upwards: and starting from the current newly expanded node, and updating the quality value and the access times of all nodes on the path upwards along the path opposite to the path during node selection until the root node.
As shown in fig. 4 (4), starting from the node that is newly expanded currently, the Q values and N values of all nodes on the path are updated upward along the path of the node selected in step (31) until the root node. Updating the quality value Q of the node according to the reward value calculated in the step (34), specifically: q (N) = Q (N) + R, the number of accesses N of a node is updated by the formula: n (N) = N (N) +1.
By this point, a training process is ended. The second training also begins with the root node and repeats steps (31) - (35). The training process is carried out by N e After that time, the training is ended.
(40) Obtaining an optimal unmanned aerial vehicle path: when the training times are equal to the total training times, finishing the training to obtain a trained Monte Carlo tree; and continuously selecting the child node with the maximum UCT value downwards by utilizing the UCT algorithm from the root node until reaching one leaf node according to the tree structure of the trained Monte Carlo tree. And the unmanned plane executes corresponding actions according to the selected nodes, so that the optimal unmanned plane path can be obtained.
N e And after the secondary training is finished, obtaining a trained Monte Carlo tree. At the moment, the parameters and the structure of the Monte Carlo tree are complete, and the optimal unmanned aerial vehicle track can be obtained more accurately. And according to the trained tree structure, selecting a child node with the largest UCT value downwards by using a UCT algorithm from the root node, and executing a corresponding action by the unmanned aerial vehicle according to an action a contained in the node, namely flying from the current fixed point to another fixed point and serving one user. Continuing to select the child nodes downwards until a leaf node is reached, and executing corresponding actions by the unmanned aerial vehicle during each node selection, namely obtaining the unmanned aerial vehicle track shown in fig. 5.
As shown in the above flow, the unmanned aerial vehicle path planning method based on monte carlo tree search aims to continuously optimize the flight trajectory of the unmanned aerial vehicle and find an optimal path, so that the average throughput of a user is maximized.
In the environment set by the invention, the position of the user, the requirement of the user on the task quantity and the channel state between the user and the unmanned aerial vehicle all change in real time. Unlike other traditional reinforcement learning algorithms, the Monte Carlo Tree search utilizes a tree structure to store the state and actions of the current situation, and adds randomness in the steps of the algorithm, speeding up the training process. Therefore, the path planning method provided by the invention can effectively reduce the training time, reduce the time complexity and the space complexity of the algorithm, and is more suitable for real-time unmanned aerial vehicle path planning in a dynamic environment.
Claims (1)
1. An unmanned aerial vehicle path planning method based on Monte Carlo tree search is characterized by comprising the following steps:
(10) Initializing unmanned aerial vehicles and Monte Carlo trees: establishing a Monte Carlo tree, initializing a root node and initializing the position of the unmanned aerial vehicle;
(20) Setting the total training times: setting the total training times of the Monte Carlo tree search algorithm according to experimental data;
(30) Training a Monte Carlo tree search algorithm: within the set total training times, carrying out search algorithm training on the Monte Carlo tree, enabling Monte Carlo tree parameters to iterate according to specific steps, and enabling the unmanned aerial vehicle to make corresponding actions according to the specific steps;
(40) Obtaining an optimal unmanned aerial vehicle path: when the training times are equal to the total training times, finishing training to obtain a trained Monte Carlo tree; continuously selecting a child node with the maximum UCT value downwards by using a UCT algorithm from a root node until reaching a leaf node according to the tree structure of the trained Monte Carlo tree, and executing corresponding actions by the unmanned aerial vehicle according to the selected node to obtain an optimal unmanned aerial vehicle path;
the (30) Monte Carlo tree search algorithm training step comprises:
(31) And (3) node selection: starting from a root node, selecting a child node with the maximum UCT value downwards by using a UCT algorithm; the unmanned aerial vehicle executes corresponding actions contained in the node; continuing to select the child nodes downwards by using a UCT algorithm until reaching a node which is not completely expanded, and stopping the selection of the nodes;
(32) And (3) node expansion: establishing a new node as a child node of the current node, and randomly selecting an action a' from the action set A to be bound with the new node;
(33) And (3) node simulation: starting from the state corresponding to the new node, performing simulation action selection and an unmanned aerial vehicle flight process;
(34) Calculating the reward value: after the simulation process is finished, calculating the reward value obtained in the process;
(35) The simulation result is updated upwards: starting from the current newly expanded node, updating the quality values and the access times of all nodes on the path upwards along the path opposite to the path during node selection until reaching the root node;
the step of (34) calculating a prize value comprises:
(341) Calculating the energy consumption of the unmanned aerial vehicle: the total energy consumption of the drone in a single time slot is calculated as follows,
e general assembly ＝e h (t)+e c (t)+e f (t)，
Wherein hovering energy consumption
p h (t) is the drone hover power,
is the data transmission rate, P, between the ith user and the UAV at the kth fixed point u Representing transmission power, σ 2 Is the power of additive white gaussian noise,
channel power gain, p, of a path loss model 0 Is the channel gain at a reference distance of 1m, H is the altitude of the drone, D uav (t) coordinates of the drone at time slot t, U i (t) is the coordinates of the ith user; s (t) = μ i (t)R i,k (t) Δ tB is the total number of bits of user tasks offloaded by the drone, μ i (t) is the number of tasks for the user, B is the channel bandwidth, Δ t is the drone hover time;
calculating energy consumption
e c (t)＝γ c CS(t)f c 2 ，
γ c Is effectively a switched capacitor, C is the number of CPU cycles required for each bit calculation, f c Is the frequency of the CPU;
flight energy consumption
κ 1 、κ 2 Is a constant parameter, a uav The acceleration of the unmanned aerial vehicle during takeoff, and g is the gravity acceleration;
(342) Computing user throughputQuantity: suppose the user has a number of task requests of mu i (t) each task has N number of bits b Bit, then user throughput is μ i (t)N b ；
(343) Calculating the reward value: the prize value for the entire simulation process is calculated as follows,
wherein the prize value generated in a single time slot
Wherein, mu max Is the maximum value of the number of tasks, W max Is the maximum value of the total energy consumption, T is the number of time slots at the beginning of the simulation process, and T is the total number of time slots.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111305350.5A CN114020024B (en) | 2021-11-05 | 2021-11-05 | Unmanned aerial vehicle path planning method based on Monte Carlo tree search |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111305350.5A CN114020024B (en) | 2021-11-05 | 2021-11-05 | Unmanned aerial vehicle path planning method based on Monte Carlo tree search |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114020024A CN114020024A (en) | 2022-02-08 |
| CN114020024B true CN114020024B (en) | 2023-03-31 |
Family
ID=80061708
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111305350.5A Active CN114020024B (en) | 2021-11-05 | 2021-11-05 | Unmanned aerial vehicle path planning method based on Monte Carlo tree search |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114020024B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115494840B (en) * | 2022-08-20 | 2024-04-12 | 安徽工程大学 | Monte Carlo factor-based MC-IACO welding robot path planning method |
| CN116185079B (en) * | 2023-04-28 | 2023-08-04 | 西安迈远科技有限公司 | Unmanned aerial vehicle construction inspection route planning method based on self-adaptive cruising |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9417070B1 (en) * | 2013-04-01 | 2016-08-16 | Nextgen Aerosciences, Inc. | Systems and methods for continuous replanning of vehicle trajectories |
| CN106959700A (en) * | 2017-03-21 | 2017-07-18 | 北京航空航天大学 | A kind of unmanned aerial vehicle group collaboration patrol tracing path planing method based on upper limit confidential interval algorithm |
| CN110166110A (en) * | 2019-05-22 | 2019-08-23 | 南京理工大学 | Unmanned plane paths planning method based on edge calculations |
| CN110297490A (en) * | 2019-06-17 | 2019-10-01 | 西北工业大学 | Heterogeneous module robot via Self-reconfiguration planing method based on nitrification enhancement |
| CN110989352A (en) * | 2019-12-06 | 2020-04-10 | 上海应用技术大学 | Group robot collaborative search method based on Monte Carlo tree search algorithm |
| CN111367317A (en) * | 2020-03-27 | 2020-07-03 | 中国人民解放军国防科技大学 | Unmanned aerial vehicle cluster online task planning method based on Bayesian learning |
| CN111552313A (en) * | 2020-04-29 | 2020-08-18 | 南京理工大学 | Multi-unmanned aerial vehicle path planning method based on edge calculation dynamic task arrival |
| CN113110592A (en) * | 2021-04-23 | 2021-07-13 | 南京大学 | Unmanned aerial vehicle obstacle avoidance and path planning method |
-
2021
- 2021-11-05 CN CN202111305350.5A patent/CN114020024B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9417070B1 (en) * | 2013-04-01 | 2016-08-16 | Nextgen Aerosciences, Inc. | Systems and methods for continuous replanning of vehicle trajectories |
| CN106959700A (en) * | 2017-03-21 | 2017-07-18 | 北京航空航天大学 | A kind of unmanned aerial vehicle group collaboration patrol tracing path planing method based on upper limit confidential interval algorithm |
| CN110166110A (en) * | 2019-05-22 | 2019-08-23 | 南京理工大学 | Unmanned plane paths planning method based on edge calculations |
| CN110297490A (en) * | 2019-06-17 | 2019-10-01 | 西北工业大学 | Heterogeneous module robot via Self-reconfiguration planing method based on nitrification enhancement |
| CN110989352A (en) * | 2019-12-06 | 2020-04-10 | 上海应用技术大学 | Group robot collaborative search method based on Monte Carlo tree search algorithm |
| CN111367317A (en) * | 2020-03-27 | 2020-07-03 | 中国人民解放军国防科技大学 | Unmanned aerial vehicle cluster online task planning method based on Bayesian learning |
| CN111552313A (en) * | 2020-04-29 | 2020-08-18 | 南京理工大学 | Multi-unmanned aerial vehicle path planning method based on edge calculation dynamic task arrival |
| CN113110592A (en) * | 2021-04-23 | 2021-07-13 | 南京大学 | Unmanned aerial vehicle obstacle avoidance and path planning method |
Non-Patent Citations (1)
| Title |
|---|
| 结合跳点引导的无人机随机搜索避撞决策方法;李安醍;《航空学报》;20200825;第41卷(第8期);第2-13页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114020024A (en) | 2022-02-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109286913B (en) | Energy consumption optimization method of unmanned aerial vehicle mobile edge computing system based on cellular network connection | |
| CN111786713B (en) | Unmanned aerial vehicle network hovering position optimization method based on multi-agent deep reinforcement learning | |
| Zhang et al. | Energy-efficient trajectory optimization for UAV-assisted IoT networks | |
| CN109831797B (en) | Unmanned aerial vehicle base station bandwidth and track joint optimization method with limited push power | |
| US20210165405A1 (en) | Multiple unmanned aerial vehicles navigation optimization method and multiple unmanned aerial vehicles system using the same | |
| Bayerlein et al. | Trajectory optimization for autonomous flying base station via reinforcement learning | |
| CN109067490B (en) | Resource allocation method for multi-unmanned aerial vehicle cooperative mobile edge computing system under cellular network connection | |
| KR102394237B1 (en) | Multiple unmanned aerial vehicles navigation oprimizaition method and multiple unmanned aerial vehicles system using the same | |
| CN114020024B (en) | Unmanned aerial vehicle path planning method based on Monte Carlo tree search | |
| CN114690799A (en) | Air-space-ground integrated unmanned aerial vehicle Internet of things data acquisition method based on information age | |
| CN114025330B (en) | Air-ground cooperative self-organizing network data transmission method | |
| CN113359480B (en) | Multi-unmanned aerial vehicle and user cooperative communication optimization method based on MAPPO algorithm | |
| CN113660681A (en) | Multi-agent resource optimization method applied to unmanned aerial vehicle cluster auxiliary transmission | |
| CN113919483A (en) | Method and system for constructing and positioning radio map in wireless communication network | |
| Shi et al. | Age of information optimization with heterogeneous uavs based on deep reinforcement learning | |
| CN117873172A (en) | Multi-unmanned aerial vehicle track planning method and system | |
| CN112579290A (en) | Unmanned aerial vehicle-based calculation task migration method for ground terminal equipment | |
| CN116321237A (en) | Unmanned aerial vehicle auxiliary internet of vehicles data collection method based on deep reinforcement learning | |
| CN112867023B (en) | Method for minimizing perception data acquisition delay through dynamic scheduling of unmanned terminal | |
| CN112383893B (en) | Time-sharing-based wireless power transmission method for chargeable sensing network | |
| CN115119174A (en) | Unmanned aerial vehicle autonomous deployment method based on energy consumption optimization in irrigation area scene | |
| CN114727323A (en) | Unmanned aerial vehicle base station control method and device and model training method and device | |
| Song et al. | Personalized Federated Deep Reinforcement Learning-based Trajectory Optimization for Multi-UAV Assisted Edge Computing | |
| Cao et al. | Average transmission rate and energy efficiency optimization in uav-assisted IoT | |
| CN116017472B (en) | Unmanned aerial vehicle track planning and resource allocation method for emergency network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |