CN113993107A - Unmanned aerial vehicle relay network method for constructing obstacle crossing area based on multiple constraints - Google Patents

Unmanned aerial vehicle relay network method for constructing obstacle crossing area based on multiple constraints Download PDF

Info

Publication number
CN113993107A
CN113993107A CN202111251276.3A CN202111251276A CN113993107A CN 113993107 A CN113993107 A CN 113993107A CN 202111251276 A CN202111251276 A CN 202111251276A CN 113993107 A CN113993107 A CN 113993107A
Authority
CN
China
Prior art keywords
relay
area
relay network
obstacle
unmanned aerial
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111251276.3A
Other languages
Chinese (zh)
Inventor
陈鸣
王文韬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Aeronautics and Astronautics
Original Assignee
Nanjing University of Aeronautics and Astronautics
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Aeronautics and Astronautics filed Critical Nanjing University of Aeronautics and Astronautics
Priority to CN202111251276.3A priority Critical patent/CN113993107A/en
Publication of CN113993107A publication Critical patent/CN113993107A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/30—Services specially adapted for particular environments, situations or purposes
    • H04W4/40—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P]
    • H04W4/44—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P] for communication between vehicles and infrastructures, e.g. vehicle-to-cloud [V2C] or vehicle-to-home [V2H]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B7/00—Radio transmission systems, i.e. using radiation field
    • H04B7/14—Relay systems
    • H04B7/15—Active relay systems
    • H04B7/185—Space-based or airborne stations; Stations for satellite systems
    • H04B7/18502—Airborne stations
    • H04B7/18504—Aircraft used as relay or high altitude atmospheric platform
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W40/00—Communication routing or communication path finding
    • H04W40/02—Communication route or path selection, e.g. power-based or shortest path routing
    • H04W40/20—Communication route or path selection, e.g. power-based or shortest path routing based on geographic position or location
    • H04W40/205—Communication route or path selection, e.g. power-based or shortest path routing based on geographic position or location using topographical information, e.g. hills, high rise buildings
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W40/00—Communication routing or communication path finding
    • H04W40/02—Communication route or path selection, e.g. power-based or shortest path routing
    • H04W40/22—Communication route or path selection, e.g. power-based or shortest path routing using selective relaying for reaching a BTS [Base Transceiver Station] or an access point

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • Astronomy & Astrophysics (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • General Physics & Mathematics (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

A plurality of unmanned aerial vehicles carrying communication equipment are used as movable communication relay nodes in the air, and a relay network with flexibility, high efficiency and low cost can be constructed. However, in general situations, an obstacle area often exists between a mobile source node and a fixed sink node, the invention provides an unmanned aerial vehicle relay network method for crossing the obstacle area based on multiple constraints, the method is based on a model-free Q-learning idea, multiple conditions such as safety constraint, direct-view constraint, coverage constraint and uniqueness constraint are met, and the unmanned aerial vehicle relay network with lower cost can be flexibly and efficiently constructed without prior terrain knowledge.

Description

Unmanned aerial vehicle relay network method for constructing obstacle crossing area based on multiple constraints
Technical Field
The invention belongs to the field of network communication, and particularly relates to a method for constructing an unmanned aerial vehicle relay network under the condition that an obstacle area exists between a mobile source node and a fixed destination node.
Background
In emergency occasions such as reconnaissance and rescue, traffic evacuation, disaster assessment, personnel search and rescue and the like, the unmanned aerial vehicle is used as a movable communication relay node in the air, so that the constructed relay network has the advantages of flexibility, high efficiency, lower cost and the like, and an effective means can be provided for information interaction between a source for dynamically acquiring information and a destination for conducting command and scheduling. In such a situation, the source node can conveniently arrive at the event site to obtain effective information by using the advantages of the unmanned aerial vehicle, and the information is sent to the destination node (such as a ground control station). Due to the limited transmission distance of wireless signals, a series of unmanned aerial vehicle nodes need to be deployed between source and destination nodes to construct an unmanned aerial vehicle relay network, and information is sent from the source node to the destination nodes in a relay manner. The source node moves according to task requirements, and the unmanned aerial vehicle relay node is adjusted correspondingly according to the position of the source node.
The key point in designing such an unmanned aerial vehicle relay network is how to select the number and positions of relay nodes, so as to form the unmanned aerial vehicle relay network at the lowest cost, and the main factors include: firstly, in order to ensure the network communication quality, the distance between nodes is required to be not more than the transmission distance; secondly, the relationship between the nodes is kept as stable as possible; thirdly, in order to ensure the lowest cost, the number of relay nodes is required to be minimum; therefore, a method for constructing the unmanned aerial vehicle relay communication network through the obstacle area is needed, and an economical and effective unmanned aerial vehicle relay network is designed.
The information disclosed in this background section is only for enhancement of understanding of the general background of the patent, and should not be taken as an acknowledgement or any form of suggestion that this information forms the prior art already known to a person skilled in the art.
Disclosure of Invention
[ object of the invention ]: the invention provides a method for constructing an unmanned aerial vehicle relay communication network through an obstacle area, which is used for ensuring economic and efficient information interaction between source and destination nodes.
[ technical solution ]:
the technical scheme of the invention is as follows:
1. a relay network system passing through an obstacle area, characterized by comprising:
A. the relay network of the unmanned aerial vehicle is composed of a source node U0A plurality of relay nodes Ui(i 1, 2.. n, b1, b 2.. be) and a destination node Ground Control Station (GCS), the slave U being formed by a routing protocol0To an end-to-end path, wherebyThe unmanned aerial vehicle relay network forms an obstacle area relay network and an obstacle-free area relay network, and its composition is shown in fig. 1.
B.U0The unmanned aerial vehicle has sensing and wireless communication capabilities, can fly in a mission area and acquire information in real time, transmits the information to the GCS through a wireless relay network, and can also receive the information from the GCS.
The gcs is a node located at a fixed location on the ground.
D. Unmanned aerial vehicle node U deployed in obstacle areabi(i 1, 2.. e) to form an obstacle area relay network, while drone nodes U may be deployed in an obstacle-free areai(i ═ 1, 2.. n) forms a relay network.
E. All drone nodes may be considered as one particle and fly at the same altitude.
2. The obstacle area relay network of claim 1, comprising
A. The relay node is according to the coverage area, the terrain and the maximum communication distance d of the unmanned aerial vehicle of the obstacle areamaxThe number varies from one to another.
B. One or more obstacles exist in the obstacle area relay network, and a path for the unmanned aerial vehicle to fly exists between the obstacles.
C. The key point of constructing the relay network in the obstacle area is to find a relay path to minimize the path at the relay node, and the key point is to determine the endpoint U of the BRNet in the obstacle areab1I.e. by
Figure BSA0000255882580000021
Wherein n isBRNetNumber of relay nodes, n, representing relay network in obstacle areaBFRNetNumber of relay nodes, U, representing unobstructed areab1Can be considered to belong to a node common to both regions.
3. The obstacle area relay network as described in claim 1 is to satisfy the following constraints, which are characterized by including
A. Safety restraint
To ensureEnsuring that the unmanned aerial vehicle flies safely, deploying relay nodes should avoid terrain obstacle areas if the unmanned aerial vehicle is deployed
Figure BSA0000255882580000026
Indicating an obstacle area, then
Figure BSA0000255882580000022
B. Direct view constraint
To ensure the relay link to function normally, a direct-view path exists between any adjacent nodes of the link, and the order is { U }i，Ui+1Denotes a sight line segment between adjacent nodes, qiRepresents UiIn a position of
{Ui，Ui+1}＝{x|x∈R2，x＝λqi+(1-λ)qi+1，λ∈(0，1)} (3)
Is provided with
Figure BSA0000255882580000024
C. Coverage constraints
The maximum communication distance between the nodes is recorded as dmaxLet | Ui，Ui+1I represents node UiAnd node Ui+1The Euclidean distance between them is
||Ui，Ui+1||≤dmax，i＝0，...，n-1 (5)
D. Constraint of uniqueness
There is no same point of deployment in the relay network, i.e.
Figure BSA0000255882580000025
As shown in fig. 2, the deployment positions of the relay nodes are only limited to discrete points with equal spacing. Wherein, the dark region represents the coverage area of the barrier, the white point is a deployable point, the dotted region is a task area of the source node, and blackThe color point is then a non-deployable point. Wherein the deployment point spacing d0Should be much smaller than the maximum communication distance of the drone, i.e.
d0＜＜dmax. (7)
4. An algorithm for constructing a relay network in an obstructed area, comprising:
A. an unmanned aerial vehicle relay network is deployed in an obstacle area, and a Q-Learning algorithm of a Reinforcement Learning (RL) model without a model is adopted, wherein an unmanned aerial vehicle node is an agent. In the barrier area, in each step, an agent in the current state S ∈ S transitions to the state S' ∈ S (state set) after taking an action (action) a ∈ a (action set). When a certain node is found, so that no obstacle exists between the node and any position in the task area, the node is considered to cross the obstacle. Agent generates state space before taking next action
Figure BSA0000255882580000031
B. To guarantee communication of the relay link, the agent's state space is limited to a set of locations that satisfy the aforementioned constraints. As shown in fig. 3, a triangle represents the current position of an agent, a circle with the triangle as the center represents the wireless communication coverage, dark color points in the circle are all positions where relay nodes can be deployed, and the set of all selectable positions constitutes the action space of the current state (position).
C. Since the cost of the relay network increases due to the arrangement of more relay nodes, the reward function is defined as:
Figure BSA0000255882580000032
the number n of nodes in the barrier-free region is introduced in the formula (9)BFRNetThis is because when an agent searches a path passing through a barrier area, it needs to consider the connection with the relay network in the barrier-free area, and this location also needs to deploy a node, so the reward is(-nBFRNet-1). Otherwise, the reward is-1.
The goal of the RL is to learn the optimal strategy, i.e., maximize equation (10):
Figure BSA0000255882580000033
wherein, E [. C]Indicates expectation, skIndicating the k-th state. Considering the cost of the relay network, Q (s, a) can be derived by solving the Bellman equation:
Figure BSA0000255882580000034
where γ is 0. ltoreq. gamma. ltoreq.1 is a count factor, and s' represents a state where s assumes a. The optimal policy for state s is thus defined as:
Figure BSA0000255882580000035
the method is modeless, i.e. reward is the only feedback in agent's interaction with the environment. Thus, in this process, Q (s, a) can be estimated recursively, i.e.:
Figure BSA0000255882580000036
where 0 ≦ α ≦ 1 is the learning rate to weigh the gains before and after.
D. The multi-constraint based Q-Learning (CQL) algorithm describes a process of constructing a relay network of unmanned aerial vehicles in an obstacle area:
1: initializing Q-table, reward revenue function
2：for each episode then：
3：
Figure BSA0000255882580000037
4: selecting
Figure BSA0000255882580000038
Or randomly in A(s) (ε -green)
5：
Figure BSA0000255882580000039
6：s←s′
7：end while
8：end for
9: output Q (s, a)
In CQL, step 1 initializes Q-table, reward revenue function and initial position. And 2, searching a relay path by using a CQL algorithm in steps 2-5. Specifically, step 2 adds all nodes that satisfy the constraint to the action set based on the current location. And 3, in order to avoid trapping in a local optimal solution, selecting action according to an epsilon-green strategy so as to reach a new state. And 4, updating the Q-table and updating the position information based on the Bellman equation and a reward function. And 5, returning to the step 2 until the obstacle crossing area is searched. And finally, returning to the initial position, and performing the next iteration until all iterations are completed. According to Q (s, a) output by the algorithm 1, a relay path passing through an obstacle area, namely the deployment position of the relay node of the unmanned aerial vehicle can be obtained.
[ advantageous effects ]: the invention has the significance that the multi-constraint Q-learning-based method for constructing the unmanned aerial vehicle relay network through the obstacle area can flexibly and efficiently construct the unmanned aerial vehicle relay network with lower cost under the condition of no priori terrain knowledge.
Drawings
FIG. 1 is a composition of a relay network for unmanned aerial vehicles traversing obstructed areas
FIG. 2 shows that the deployment positions of the relay nodes are only limited to discrete points with equal spacing
FIG. 3 is an action space of an agent
FIG. 4 is a diagram illustrating an example of a relay network test for unmanned aerial vehicles constructed in an obstacle area
FIG. 5 is the impact of maximum communication distance of the drone on algorithm performance
FIG. 6 is the effect of deployment point spacing on algorithm performance
Detailed Description
The following detailed description of embodiments of the present invention will be described in conjunction with the accompanying drawings, but it should be understood that the scope of the present invention is not limited to the specific embodiments.
1. Test scenario
A system for realizing design based on the method provided by the invention is arranged on an OMNeT + + simulation platform, and results of implementing the unmanned aerial vehicle relay network in the obstacle area by adopting different algorithms are contrastively analyzed.
The moving range of the nodes of the unmanned aerial vehicle is 2000m by 1000m, the distance between adjacent deployment points is 10m, and the maximum flying speed of the unmanned aerial vehicle is 10 m/s; maximum wireless communication radius d between unmanned aerial vehiclesmaxIs 250 m; the task area of the source node is a square area with a side length of 400m and the center coordinate of the square is (1600,700). The obstacle parameters in the obstacle area are shown in table 1, and the rest are all the obstacle-free areas.
TABLE 1 obstacle parameter table
Figure BSA0000255882580000041
Figure BSA0000255882580000051
Fig. 4 shows an experimental example of a proprietary prototype system for validating the invention. The unmanned aerial vehicle relay network has the obstacle areas shown in table 1, and the unmanned aerial vehicle relay network is constructed based on a CQL algorithm, wherein '+' represents an unmanned aerial vehicle relay node; the destination node of the path is GCS, and also includes Ub4、Ub3、Ub2And Ub1And (4) nodes. In the barrier-free region, the source node U0Moving in task area, determining U according to equal division principle3、U2、U1The position of (a). Table 2 shows the position information of each drone relay node determined by the test system.
Table 3 relay drone deployment location
Figure BSA0000255882580000052
2. Performance testing
In order to evaluate the performance of the CQL algorithm under the condition without prior knowledge, a Search result of a Breadth-First Search algorithm (BFS) with prior knowledge is used as a reference for comparison. Under the condition of no prior knowledge, a CQL Algorithm is respectively compared with relay node deployment results obtained by running a Genetic Algorithm (GA) and an Ant Colony Optimization (ACO).
In order to reduce the adverse effect of various accidental factors on the test result, 100 different sets of tests are carried out based on the Monte-Carlo method, such as setting the obstacles as circular objects with the radius of 100m, and randomly giving the number of the obstacles, the task area, the position of the ground station and the task area.
Four algorithms were run for each trial: BFS algorithm, CQL algorithm, GA and ACO, and the BFS algorithm with prior knowledge is used as a reference for comparison. The iteration times of CQL and ACO algorithms are respectively set to 300; the GA initial population size is 100, the genetic algebra is 300, and sequencing selection, single-point crossing and single-point mutation operators are adopted. The average number of nodes required by the 3 algorithms to construct the relay network under the same conditions is compared respectively.
Defining Relative Network Cost (RNC) as a measure for evaluating algorithm performance:
Figure BSA0000255882580000053
wherein N represents the number of nodes required by the evaluated algorithm, and NBFSAnd the number of nodes obtained by the BFS algorithm is shown. Obviously, RNC is a positive number greater than 1, and the closer RNC is to 1, the better the algorithm performance is evaluated. The following analysis was made from the following factors, respectively.
1.dmaxImpact on the number of relay link nodes. The distance between deployment points is set to be 40m, as shown in fig. 5, wherein the horizontal axis is the maximum communication distance d of the unmanned aerial vehiclemaxAnd the vertical axis is RNC. It can be seen that with the enhancement of the communication capability of the drone, the performance of several algorithms is improved, while the CQL algorithm has better performance than the other two algorithms, and d ismaxThe advantage is more evident when smaller. The reason for this is that the CQL algorithm searches for a range d at a timemaxDetermine with dmaxThe search iteration times are reduced; and in the CQL algorithm updating process, the Bellman equation can enable the reward of each action to play a role.
2. The impact of deployment point spacing on the number of relay link nodes. Assuming that the maximum communication distance is kept constant at 100m, as shown in fig. 6, the horizontal axis represents the distance between deployment points and the vertical axis represents the RNC. Therefore, the performance of the algorithm can be improved by increasing the distance between the discrete deployment points. This is because the number of feasible deployment points is reduced due to the increase of the deployment point spacing, that is, the points selectable by the algorithm are reduced, so that the calculation amount of the algorithm is reduced, and the advantage of the CQL is more obvious when the deployment point spacing is smaller.
In summary, the test results show that the three algorithms have better performance when the problem size is not large. With the increase of the problem scale, the number of iterative search is also increased, and the performance of the CQL algorithm is obviously better than that of the GA and ACO algorithms.
The foregoing description of specific exemplary embodiments of the present patent is for the purpose of illustration and description. It is not intended to limit the invention to the precise form disclosed, and obviously many modifications and variations are possible in light of the above teaching. The exemplary embodiments were chosen and described in order to explain certain principles of the present patent disclosure and its practical application to enable one skilled in the art to make and use various exemplary embodiments of the present patent disclosure as well as various alternatives and modifications thereof. It is intended that the scope of the patent be defined by the claims and their equivalents.

Claims (4)

1. A relay network system passing through an obstacle area, characterized by comprising:
A. the relay network of the unmanned aerial vehicle is composed of a source node U0A plurality of relay nodes Ui(i 1, 2.. n, b1, b 2.. be) and a destination node Ground Control Station (GCS), the slave U being formed by a routing protocol0An end-to-end path to, whereby the unmanned relay network forms an obstacle area relay network and an obstacle-free area relay network, the composition of which is shown in fig. 1;
B.U0the unmanned aerial vehicle has sensing and wireless communication capabilities, can fly in a task area, acquire information in real time, transmit the information to the GCS through a wireless relay network, and also receive the information from the GCS;
the gcs is a node located at a ground fixed location;
D. unmanned aerial vehicle node U deployed in obstacle areabi(i 1, 2.. e) to form an obstacle area relay network, while drone nodes U may be deployed in an obstacle-free areai(i ═ 1, 2,. n) to form a relay network;
E. all drone nodes may be considered as one particle and fly at the same altitude.
2. An obstacle area relay network as claimed in claim 1, comprising
A. The relay node is according to the coverage area, the terrain and the maximum communication distance d of the unmanned aerial vehicle of the obstacle areamaxDifferent, the number of them is different;
B. one or more obstacles exist in the obstacle area relay network, and a path for the unmanned aerial vehicle to fly exists between the obstacles;
C. the key point of constructing the relay network in the obstacle area is to find a relay path to minimize the path at the relay node, and the key point is to determine the endpoint U of the BRNet in the obstacle areab1I.e. by
Figure FSA0000255882570000011
Wherein n isBRNetNumber of relay nodes, n, representing relay network in obstacle areaBFRNetNumber of relay nodes, U, representing unobstructed areab1Can be considered to belong to a node common to both regions.
3. The obstacle area relay network of claim 1, wherein the constraints comprise
A. Safety restraint
In order to ensure the safe flight of the unmanned aerial vehicle, the relay nodes are deployed to avoid the terrain obstacle area if the relay nodes are arranged
Figure FSA0000255882570000012
Indicating an obstacle area, then
Figure FSA0000255882570000013
B. Direct view constraint
To ensure the relay link to function normally, a direct-view path exists between any adjacent nodes of the link, and the order is { U }i，Ui+1Denotes a sight line segment between adjacent nodes, qiRepresents UiIn a position of
{Ui，Ui+1}＝{x|x∈R2，x＝λqi+(1-λ)qi+1，λ∈(0，1)} (3)
Is provided with
Figure FSA0000255882570000014
C. Coverage constraints
The maximum communication distance between the nodes is recorded as dmaxLet | Ui，Ui+1I represents node UiAnd node Ui+1The Euclidean distance between them is
||Ui，Ui+1||≤dmax，i＝0，...，n-1 (5)
D. Constraint of uniqueness
There is no same point of deployment in the relay network, i.e.
Figure FSA0000255882570000021
As shown in fig. 2, the deployment positions of the relay nodes are only limited to discrete points with equal spacing. The dark color area represents an obstacle coverage area, the white point is a deployable point, the dotted line area is a source node task area, and the black point is a non-deployable point. Wherein the deployment point spacing d0Should be much smaller than the maximum communication distance of the drone, i.e.
d0＜＜dmax. (7)。
4. An algorithm for constructing a relay network in an obstructed area, comprising:
A. an unmanned aerial vehicle relay network is deployed in an obstacle area, and a Q-Learning algorithm of a Reinforcement Learning (RL) model without a model is adopted, wherein an unmanned aerial vehicle node is an agent. In the barrier area, in each step, an agent in the current state S ∈ S transitions to the state S' ∈ S (state set) after taking an action (action) a ∈ a (action set). When a certain node is found, no obstacle exists between the node and any position in the task area, the node is considered to cross the obstacle; agent generates state space before taking next action
Figure FSA0000255882570000022
B. To guarantee communication of the relay link, the agent's state space is limited to a set of locations that satisfy the aforementioned constraints. As shown in fig. 3, a triangle represents the current position of an agent, a circle with the triangle as the center represents the wireless communication coverage, light color dots in the circle are all positions where relay nodes can be deployed, and the set of all selectable positions forms the action space of the current state (position);
C. since the cost of the relay network increases due to the arrangement of more relay nodes, the reward function is defined as:
Figure FSA0000255882570000023
the number n of nodes in the barrier-free region is introduced in the formula (9)BFRNetThis is because when an agent searches a path passing through an obstacle area, it needs to consider the connection with the relay network in the obstacle-free area, and this position also needs to deploy a node, so the reward is (-n)BFRNet-1). In addition, reward is-1;
the goal of the RL is to learn the optimal strategy, i.e., maximize equation (10):
Figure FSA0000255882570000024
wherein, E [. C]Indicates expectation, skIndicating the k-th state. Considering the cost of the relay network, Q (s, a) can be derived by solving the Bellman equation:
Figure FSA0000255882570000025
where γ is 0. ltoreq. gamma. ltoreq.1 is a count factor, and s' represents a state where s assumes a. The optimal policy for state s is thus defined as:
Figure FSA0000255882570000026
the method is modeless, i.e. reward is the only feedback in agent's interaction with the environment. Thus, in this process, Q (s, a) can be estimated recursively, i.e.:
Figure FSA0000255882570000027
where 0 ≦ α ≦ 1 is the learning rate to weigh the gains before and after.
D. The multi-constraint based Q-Learning (CQL) algorithm describes a process of constructing a relay network of unmanned aerial vehicles in an obstacle area:
1: initializing Q-table, reward revenue function
2：for each episode then：
3：while(
Figure FSA0000255882570000031
s.t.
Figure FSA0000255882570000032
)
4: selecting
Figure FSA0000255882570000033
Or randomly in A(s) (ε -green)
5：
Figure FSA0000255882570000034
6：s←s′
7：end while
8：end for
9: output Q (s, a)
In CQL, step 1 initializes Q-table, reward revenue function and initial position. And 2, searching a relay path by using a CQL algorithm in steps 2-5. Specifically, step 2 adds all nodes that satisfy the constraint to the action set based on the current location. And 3, in order to avoid trapping in a local optimal solution, selecting action according to an epsilon-green strategy so as to reach a new state. And 4, updating the Q-table and updating the position information based on the Bellman equation and a reward function. And 5, returning to the step 2 until the obstacle crossing area is searched. And finally, returning to the initial position, and performing the next iteration until all iterations are completed. According to Q (s, a) output by the algorithm 1, a relay path passing through an obstacle area, namely the deployment position of the relay node of the unmanned aerial vehicle can be obtained.
CN202111251276.3A 2021-10-26 2021-10-26 Unmanned aerial vehicle relay network method for constructing obstacle crossing area based on multiple constraints Pending CN113993107A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111251276.3A CN113993107A (en) 2021-10-26 2021-10-26 Unmanned aerial vehicle relay network method for constructing obstacle crossing area based on multiple constraints

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111251276.3A CN113993107A (en) 2021-10-26 2021-10-26 Unmanned aerial vehicle relay network method for constructing obstacle crossing area based on multiple constraints

Publications (1)

Publication Number Publication Date
CN113993107A true CN113993107A (en) 2022-01-28

Family

ID=79742063

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111251276.3A Pending CN113993107A (en) 2021-10-26 2021-10-26 Unmanned aerial vehicle relay network method for constructing obstacle crossing area based on multiple constraints

Country Status (1)

Country Link
CN (1) CN113993107A (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112636811A (en) * 2020-12-08 2021-04-09 北京邮电大学 Relay unmanned aerial vehicle deployment method and device
CN113162679A (en) * 2021-04-01 2021-07-23 南京邮电大学 DDPG algorithm-based IRS (inter-Range instrumentation System) auxiliary unmanned aerial vehicle communication joint optimization method
CN113258989A (en) * 2021-05-17 2021-08-13 东南大学 Method for obtaining relay track of unmanned aerial vehicle by using reinforcement learning

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112636811A (en) * 2020-12-08 2021-04-09 北京邮电大学 Relay unmanned aerial vehicle deployment method and device
CN113162679A (en) * 2021-04-01 2021-07-23 南京邮电大学 DDPG algorithm-based IRS (inter-Range instrumentation System) auxiliary unmanned aerial vehicle communication joint optimization method
CN113258989A (en) * 2021-05-17 2021-08-13 东南大学 Method for obtaining relay track of unmanned aerial vehicle by using reinforcement learning

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
MENG-CHUN HOU 等: "Optimum Aerial Base Station Deployment for UAV Networks: A Reinforcement Learning Approach", 2019 IEEE GLOBECOM WORKSHOPS (GC WKSHPS), 13 December 2019 (2019-12-13) *
屈鑫祺: "无人机中继的节能部署算法研究", 中国优秀硕士学位论文全文数据库, 15 February 2020 (2020-02-15) *

Similar Documents

Publication Publication Date Title
US20210348928A1 (en) Multi-police-officer collaborative round-up task allocation and path planning method under constraint of road network
US20120158280A1 (en) Computing route plans for routing around obstacles having spatial and temporal dimensions
KR102279956B1 (en) 3D optimal surveillance trajectory planning Method and Apparatus for multi-UAVs using particle swarm optimization with surveillance area priority
Yanmaz et al. A discrete stochastic process for coverage analysis of autonomous UAV networks
WO2009091431A1 (en) Computing flight plans for uavs while routing around obstacles having spatial and temporal dimensions
US11513515B2 (en) Unmanned vehicles and associated hub devices
Liu et al. Real-time UAV rerouting for traffic monitoring with decomposition based multi-objective optimization
Geng et al. Cooperative mission planning with multiple UAVs in realistic environments
Pang et al. A risk-based UAS traffic network model for adaptive urban airspace management
Sharma Evolution of unmanned aerial vehicles (UAVs) with machine learning
Labib et al. A multilayer low-altitude airspace model for UAV traffic management
CN111813144A (en) Multi-unmanned aerial vehicle collaborative route planning method based on improved flocks of sheep algorithm
Wang et al. Mission planning for heterogeneous tasks with heterogeneous UAVs
De Silva et al. Optimal desired trajectories of UAVs in private UAV networks
Jailton et al. Relay positioning strategy for traffic data collection of multiple unmanned aerial vehicles using hybrid optimization systems: A FANET-based case study
CN114399226A (en) Unmanned aerial vehicle region reconnaissance value evaluation method based on semantic cognition
Leu et al. Survivable networks via UAV swarms guided by decentralized real-time evolutionary computation
CN113993107A (en) Unmanned aerial vehicle relay network method for constructing obstacle crossing area based on multiple constraints
Kaya et al. A probabilistic risk assessment framework for the path planning of safe task-aware UAS operations
Kusyk et al. AI based flight control for autonomous uav swarms
Geng et al. Cooperative task planning for multiple autonomous UAVs with graph representation and genetic algorithm
Rahman et al. Relay vehicle formations for optimizing communication quality in robot networks
Ebenezer A mobility model for MANET in large scale disaster scenarios
Scozzaro et al. Noise abatement trajectories for a uav delivery fleet
Poudel et al. Priority-aware task assignment and path planning for efficient and load-balanced multi-UAV operation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination