CN115802313A - Air-ground mobile network energy-carrying fair communication method based on intelligent reflecting surface - Google Patents
Air-ground mobile network energy-carrying fair communication method based on intelligent reflecting surface Download PDFInfo
- Publication number
- CN115802313A CN115802313A CN202211472603.2A CN202211472603A CN115802313A CN 115802313 A CN115802313 A CN 115802313A CN 202211472603 A CN202211472603 A CN 202211472603A CN 115802313 A CN115802313 A CN 115802313A
- Authority
- CN
- China
- Prior art keywords
- unmanned aerial
- aerial vehicle
- representing
- reflecting surface
- intelligent
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000004891 communication Methods 0.000 title claims abstract description 92
- 238000000034 method Methods 0.000 title claims abstract description 60
- 230000005540 biological transmission Effects 0.000 claims abstract description 54
- 238000005265 energy consumption Methods 0.000 claims abstract description 39
- 239000011159 matrix material Substances 0.000 claims abstract description 28
- 238000005457 optimization Methods 0.000 claims abstract description 26
- 230000002787 reinforcement Effects 0.000 claims abstract description 13
- 230000007246 mechanism Effects 0.000 claims description 39
- 239000003795 chemical substances by application Substances 0.000 claims description 37
- 239000013598 vector Substances 0.000 claims description 36
- 230000006870 function Effects 0.000 claims description 28
- 230000009471 action Effects 0.000 claims description 25
- 238000013507 mapping Methods 0.000 claims description 21
- 238000004220 aggregation Methods 0.000 claims description 15
- 238000013528 artificial neural network Methods 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 13
- 230000002776 aggregation Effects 0.000 claims description 11
- 238000000354 decomposition reaction Methods 0.000 claims description 9
- 238000005516 engineering process Methods 0.000 claims description 9
- 238000013139 quantization Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000011156 evaluation Methods 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 230000009365 direct transmission Effects 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 239000006185 dispersion Substances 0.000 claims description 3
- 230000005251 gamma ray Effects 0.000 claims description 3
- 230000009349 indirect transmission Effects 0.000 claims description 3
- 239000000203 mixture Substances 0.000 claims description 3
- 230000000717 retained effect Effects 0.000 claims description 3
- 230000007704 transition Effects 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 2
- 230000001105 regulatory effect Effects 0.000 claims description 2
- 238000013461 design Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000005304 joining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Mobile Radio Communication Systems (AREA)
Abstract
The invention provides an air-ground mobile network energy-carrying fair communication method based on an intelligent reflecting surface, which comprises the following steps: establishing an air-ground mobile network architecture based on multiple unmanned aerial vehicles and an intelligent reflecting surface; establishing a wireless power transmission model; establishing an energy consumption model of the unmanned aerial vehicle according to the dynamic model and the communication model of the unmanned aerial vehicle; reconstructing a channel state between the unmanned aerial vehicle and a ground user by using the intelligent reflecting surface, and establishing a wireless communication model; establishing a fair communication model; constructing a judgment matrix about fair throughput and energy consumption, and determining weight coefficients of two sub-targets of fair weighted throughput and energy consumption; modeling is a multi-objective integer non-convex optimization problem of fair throughput and unmanned aerial vehicle residual energy maximization, and a complex multi-objective optimization problem is solved through multi-agent deep reinforcement learning. The invention optimizes the position of the unmanned aerial vehicle and the phase of the intelligent reflecting surface based on multi-agent deep reinforcement learning, provides fair communication for ground users and wirelessly charges the unmanned aerial vehicle.
Description
Technical Field
The invention relates to the technical field of air-ground cooperative auxiliary communication, in particular to an air-ground mobile network energy-carrying fair communication method based on an intelligent reflecting surface, which is used for carrying out fair energy-carrying communication based on an unmanned aerial vehicle and the intelligent reflecting surface under limited communication resources.
Background
At present, the establishment and implementation of communication networks mainly rely on terrestrial base stations or other fixed communication devices, and the flexibility thereof is greatly limited. To address this problem, drone-assisted wireless communication has gained widespread attention as a new communication means in both academic and industrial sectors. The unmanned aerial vehicle has the advantages of high maneuverability, strong universality, rapid deployment and the like, and is widely applied to the fields of intelligent traffic, post-disaster reconstruction, emergency communication, remote area communication range expansion and the like.
In the unmanned aerial vehicle auxiliary communication, the unmanned aerial vehicle mainly serves as a mobile base station to provide communication service for ground users. The high mobility and flexibility of the drone can quickly establish a communication connection and significantly improve data transmission efficiency, for example, when a ground infrastructure communication facility is damaged by a natural disaster or the like, the drone can be used as a temporary base station to provide a temporary emergency communication service for a ground user. Drone-assisted communication still presents the following challenges: the position optimization of the unmanned aerial vehicle mobile base station is a typical optimal sequence decision problem, the problem is usually provided with a plurality of decision variables and is non-convex, and the problem is difficult to directly solve by adopting a traditional convex optimization method. In addition, when the traditional method solves the problem of trajectory optimization, the calculation complexity exponentially increases along with the number of unmanned aerial vehicles and ground users. The channel state between the drone and the user is susceptible to the external environment and does not take into account the impact of fairness between users on system performance in optimizing system communication efficiency.
Intelligent reflective surfaces have been extensively studied with the benefit of improved propagation environment and increased signal strength, and generally consist of energy-efficient, cost-effective reconfigurable passive components. Each of which may be phase shifted by an intelligent controller with respect to the incident signal. Thus, with the help of the intelligent reflective surface, signals from different communication links can be superimposed at the desired receivers to increase the energy of the received signal, or can be destructively added at the undesired receivers to avoid information leakage. Due to the fact that urban environments are complex and changeable and various buildings are shielded, the intelligent reflecting surface is used for reconstructing the transmission link to play a very important role in future smart cities.
Disclosure of Invention
Aiming at the technical problems that the existing unmanned aerial vehicle auxiliary communication method is easily influenced by the external environment and fairness among users is not considered, the invention provides an air-ground mobile network energy-carrying fair communication method based on an intelligent reflector, the position of the unmanned aerial vehicle and the phase of the intelligent reflector are optimized based on a multi-agent deep reinforcement learning optimization algorithm, and the wireless charging of the unmanned aerial vehicle is realized while fair communication is provided for ground users.
In order to achieve the purpose, the technical scheme of the invention is realized as follows: an air-ground mobile network energy-carrying fair communication method based on an intelligent reflecting surface comprises the following steps:
s1: establishing an air-ground mobile network architecture based on a plurality of unmanned aerial vehicles and an intelligent reflecting surface, wherein the air-ground mobile network architecture comprises K ground mobile users and D unmanned aerial vehicles;
s2: establishing a wireless power transmission model according to a wireless power transmission technology: the intelligent lamp pole is used as an energy source, and two transmission paths of the intelligent lamp-the direct transmission of the unmanned aerial vehicle and the intelligent lamp pole-the intelligent reflecting surface-the indirect transmission of the unmanned aerial vehicle are adopted to realize the wireless charging of the unmanned aerial vehicle;
s3: establishing an energy consumption model of the unmanned aerial vehicle according to the dynamic model and the communication model of the unmanned aerial vehicle;
s4: reconstructing a channel state between the unmanned aerial vehicle and a ground user by using the intelligent reflecting surface, and establishing a wireless communication model;
s5: establishing a fair communication model: a fair communication model is established by considering communication efficiency and fairness among users, and system throughput is maximized on the premise of guaranteeing user fairness;
s6: constructing a judgment matrix about fair throughput and energy consumption according to the user service quality grade, solving and normalizing the eigenvalue and the eigenvector of the judgment matrix, and determining the weight coefficients of two sub-targets of fair weighted throughput and energy consumption;
s7: the unmanned aerial vehicle energy carrying communication problem is modeled into a multi-target integer non-convex optimization problem with fair throughput and maximized unmanned aerial vehicle residual energy, the problem is described as a Markov game process again, the complex multi-target optimization problem is solved through multi-agent deep reinforcement learning, and the position of the unmanned aerial vehicle and the phase of an intelligent reflecting surface are updated.
Preferably, the set of terrestrial mobile users is represented asThe set of drones is denoted asThe intelligent reflecting surface is a reflecting surface responsible for charging, the unmanned aerial vehicle serves as a mobile base station to provide communication service for ground mobile users, and the intelligent lamp pole serves as an energy source to provide energy transmission for the unmanned aerial vehicle;
the method for constructing the wireless power transmission model in the step S2 comprises the following steps: an intelligent reflecting surface is adopted to reconstruct an energy transmission path, an energy beam emitted by a power emitter of the intelligent lamp pole reaches a light receiver of the unmanned aerial vehicle through two transmission paths of the intelligent lamp pole-the unmanned aerial vehicle and the intelligent lamp pole-the intelligent reflecting surface-the unmanned aerial vehicle, and the energy harvested by the unmanned aerial vehicle is as follows:
wherein,representing the direct channel gain, H, between the drone and the terrestrial mobile users s (t) represents the channel gain between the energy source and the drone,the channel gain between the intelligent reflecting surface and the unmanned aerial vehicle is represented, theta (t) represents the phase of the intelligent reflecting surface, and eta represents the energy conversion coefficient.
Preferably, the method for establishing the energy consumption model of the unmanned aerial vehicle in S3 includes:
unmanned aerial vehicle has the energy resource consumption that removes consumption, communication consumption and internal circuit produced at the in-process of carrying out the auxiliary communication task, then the total energy consumption of unmanned aerial vehicle is:
wherein, P t (t) denotes the transmitted power of the drone, P c (t) constant circuit power, P, for the drone d (t) represents the propulsion power of the drone at time t, and the propulsion power P d (t) is expressed as:
wherein,representing the flying speed, V, of the drone at time t max Represents the maximum flying speed of the unmanned aerial vehicle;respectively representing the positions of the unmanned aerial vehicle at t +1 moment and t moment and the time difference between the two moments; p o Representing blade section power, P i Indicating hovering power, v 0 Denotes the average induced speed of the rotor, d 0 Expressing the fuselage drag coefficient, ρ air density, s motor volume, A rotor area, U tip Representing the linear speed of the blade tip;
the remaining energy of the drone is:
wherein E is max Represents the maximum energy value after the unmanned aerial vehicle is fully charged, E d (t) represents the battery energy remaining at the drone at time t,representing the energy harvested by the drone.
Preferably, the method for establishing the wireless communication model in step S4 includes: adjusting the phase and amplitude reconstruction channel conditions of the intelligent reflecting surface to improve the transmission rate between the unmanned aerial vehicle and the user, wherein the transmission rate received by the ground mobile user is as follows:
wherein,a phase coefficient matrix representing the anti-jamming intelligent reflecting surface,indicating the phase, M, of an intelligent reflecting surface in wireless information transmission r 、M c Respectively representing the number of reflecting elements on the rows and columns of the intelligent reflecting surface,(m) th representing intelligent reflecting surface r ,m c ) The phase of the reflective element; h is a total of UG (t) denotes the channel gain of the transmission link for the unmanned aerial vehicle and the terrestrial mobile subscriber, B k (t) bandwidth allocated to the ground mobile user k by the drone, α d,k (t) indicates whether unmanned plane d is serving ground mobile user k, P t Denotes the transmission power, h RG (t) represents the channel gain between the intelligent reflecting surface and the land mobile user, h UR (t) represents the channel gain, σ, between the drone and the smart reflecting surface 2 Representing gaussian white noise.
Preferably, the unmanned plane U-intelligent reflector R, the intelligent reflector R-ground mobile user G and the unmanned plane U-ground mobile user G are arranged to have channel gain h between the unmanned plane U-intelligent reflector R and the unmanned plane U-ground mobile user G without a line of sight link between the unmanned plane and the ground mobile user UR (t)、h RG (t) and h UG (t) are respectively:
wherein, beta 0 Denotes the channel power gain per unit distance, D UR (t)、D RG (t) and D UG (t) respectively representing the distances between the unmanned aerial vehicle U-intelligent reflector R, the intelligent reflector R-ground mobile user G and the unmanned aerial vehicle U-ground mobile user G at time t, and respectively representing the distances; 2. alpha and beta represent unmanned plane U-intelligence respectivelyThe path loss indexes on the links of the reflective surface R, the intelligent reflective surface R-ground mobile user G and the unmanned aerial vehicle U-ground mobile user G;the portion representing the line of sight in the U-intelligent reflector R link of the drone, depending on the flight trajectory of the drone at time slot n, is represented by:
wherein,
and isAndcosine and sine respectively representing the horizontal arrival angle of the signal on the intelligent reflecting surface;a sine representing a vertical arrival angle of the signal at the intelligent reflecting surface; λ represents the wavelength of the carrier wave, (x) r ,y r ,z r ) Indicating the position of the intelligent reflecting surface, x d (t)、y d (t)、z d (t) represents the horizontal coordinate and the flying height of the drone respectively,represents the kronecker product;
part of the line of sight of the intelligent reflector R-ground mobile subscriber GComprises the following steps:
wherein
Wherein,andcosine and sine representing the horizontal departure angle of the signal to the kth user;a sine representing the vertical departure angle of the signal to the kth user; x is the number of k (t)、y k (t) horizontal coordinates representing the ground users, respectively;representing the non line-of-sight portion of the R-G link,representing the random scattering index.
Preferably, the method for establishing the fair communication model in step S5 includes: fair index representation maximization system throughput and fairness balance based on throughput ratio, and ground mobile user definitionThroughput ratio f k (t) measure the importance of terrestrial mobile users:
wherein,representing the terrestrial mobile user k during a time period [0, t ]]The throughput of the network element(s) is,represents the throughput of all terrestrial mobile users;
measuring fairness among users by using Jain's fairness index, and the new evaluation index for balancing communication efficiency and fairness is as follows:
preferably, the method for determining the weight coefficient in step S6 is: and performing grade quantization on the energy consumption sub-target and the fair throughput by taking the user service quality as a standard according to the attributes of the tasks to obtain a grade quantization table as follows:
applications of | Energy consumption | Throughput capacity |
Real-time data | y 2 | y 3 |
Image data | y 1 | y 2 |
Audio data | y 2 | y 4 |
Non-compressed video | y 4 | y 3 |
Compressing video | y 1 | y 2 |
Wherein the rank [ y 1 ,y 2 ,y 3 ,y 4 ]Represents the level of importance; constructing a judgment matrix about energy consumption and fair throughput according to a grade quantization table:
solving the eigenvalue and eigenvector of the judgment matrix by a Jacobi method and normalizing the eigenvalue and eigenvector to obtain a weight coefficient [ w ] corresponding to the two sub-targets 1 ,w 2 ]。
Preferably, the multi-objective integer non-convex optimization problem in step S7 is:
s.t.C1:E d (0)=E max ,E d (T t )=E min ,
C4:R GU (t)≥γ dk ,
wherein u is d (t) represents a position of the drone;a utility function representing the weighted throughput and the residual energy composition; t is t Representing a task execution time; e t (0) The electric quantity of the unmanned aerial vehicle at the initial time is represented; e max Representing the maximum battery capacity of the unmanned aerial vehicle when the unmanned aerial vehicle is fully charged; e d (T t ) Representing the remaining electric quantity when the unmanned aerial vehicle task is finished; e min Representing the minimum electric quantity required by safe return after the unmanned aerial vehicle executes the task; gamma ray dk Represents a transmission rate minimum threshold; u. of i (t) and u j (t) respectively representing the positions of the unmanned planes i and j at the time t; x is the number of d (t)、x k (t)、y d (t)、y k (t) coordinates, X, representing the unmanned aerial vehicle and the ground mobile user, respectively min 、X max 、Y min 、Y max The boundary value of the whole rectangular task area is obtained;
dividing the whole task execution time into N t A time slot, each time slot having a length ofWill continue to questionConversion to discrete problems:
s.t.C1~C7
solve the problem of dispersionThe method is described as a Markov game process of a multi-agent < S, A, P, R, gamma >, wherein S is a state set, A is an action set, R is a reward function, P is a state transition probability function, and Gamma is a reward discount factor;
the method for the multi-agent deep reinforcement learning comprises the following steps:
in time slot n ∈ [0, N t ]Internal stateWherein,the coordinates of the drone at time slot n are indicated,the coordinates representing the terrestrial mobile user at time slot n,representing the residual energy consumption of the unmanned aerial vehicle, and theta (n) representing the phase of the intelligent reflecting surface;
acting within a time slot nWherein, dist d (n)∈[0,V d (t)δ t ]Indicating that the drone base station is on timeThe distance of flight within the gap n;representing the flight direction of the unmanned aerial vehicle base station in time slot n; delta theta is the variation of the phase of the intelligent reflecting surface; v d (t) the flight speed of the drone;
the reward function is r = r 1 +r 2 -ξ 1 p 1 -ξ 2 p 2 -ξ 3 p 3 ;
Wherein the fair throughputCoverage rewardse d,k =1 indicates that user k can be covered by drone d, whereas e d,k =0;
Punishing: when the following conditions are satisfied, the drone base station will be punished: (1) Drone flight mission boundary zones, e.g.Wherein X min 、X max 、Y min 、Y max The values of the abscissa and the ordinate of the task area range are represented; (2) Collision of unmanned aerial vehicle i with unmanned aerial vehicle j, i.e. | | u i (n)-u j (n)|| 2 ≥d min In which d is min Represents a safe distance threshold; (3) When the energy consumption of the unmanned aerial vehicle is lower than a set value, namely E d (t)≤E min (ii) a By defining a binary variable ξ l E {0,1} indicates whether the above condition l is violated; if xi l The condition of violation l is represented by =1,l epsilon {1,2,3}, and the unmanned plane is given a fixed penalty p l ,l∈{1,2,3};
In the Markov game process, the intelligent agent maximizes the reward function and disperses the problem through the optimal self strategy piIs described again as
s.t.C1~C7
Wherein,representing the desired operation, and s and a are the concatenation of the state space and the action space of all agents.
Updating the state of the unmanned aerial vehicle based on an information sharing mechanism of a gate control unit, and inputting a strategy network to obtain the action to be executed by the unmanned aerial vehicle; and constructing an Actor network of state decomposition-expansion-aggregation to decompose and reduce the dimension of state information, and then performing state aggregation on the processed sub-states according to different correlation degrees by using a multi-head attention mechanism.
Preferably, the implementation method of the information sharing mechanism based on the gate control unit is as follows:
by means of a memory with a memory capacity MEstablishing state information sharing, wherein the memory is used for storing collective state information m of the unmanned aerial vehicle to form an element R M (ii) a The strategy of each unmanned aerial vehicle becomes
Each unmanned aerial vehicle sends its own state s d Mapping to an embedded vector representing the current state:wherein,is a network parameter ofA neural network of (a);
unmanned aerial vehicle carries out reading and operatingAs fetched for storage in memoryBy generating a context vector h d To capture an embedded vector e d The spatio-temporal information of (c):
wherein,representing parameters of a linear mapping network, H and E respectively representing a context vector H d And embedding vector e d Dimension (d);
embedded vector e of joint agent observed value d Context vector h d And the current memoryAs input, learn a gating mechanism:
where σ (·) is a sigmoid function, [ e ] d ,h d ,m]Representing the concatenation of three vectors, k d As a weighting factor;
regulating the reading of information r from memory by gating mechanism d =m⊙k d (ii) a Wherein, l represents a hadamard product.
The agent generates a candidate memory content through nonlinear mapping according to the coding of the state value of the agent and the information of the current shared memory:
wherein,is a network parameter; input gate g d For adjusting the content of the candidate memory, f d Determining information that needs to be retained and discarded, and:
where σ represents the sigmode activation function,Respectively representing neural network parameters needing to be trained;
then, the unmanned aerial vehicle d generates new updated information by weighted combination of the new and old information: m' = g d ⊙c d +f d ⊙m;
The unmanned aerial vehicle takes the code of the current self state and the information read from the memory as the input of the strategy network, and the strategy network outputs the action to be executed by the unmanned aerial vehicle
Wherein r is d Which represents the information read from the memory device,representing a policy function;
the state decomposition of the Actor network decomposes different types of state information and adopts a dimension expansion technology to expand all the state information to the same dimension; the aggregation is to carry out aggregation and linear mapping on each sub-state after decomposition into a low-dimensional input vector according to different correlation degrees;
selecting state information based on state information selection strategy of self-attention mechanism, and dividing the passing state intoPosition state information, remaining capacity state information, and state information read from a memory after the solution, dimension expansion, and linear mapping processes are treated as three vectors a 1 、a 2 And a 3 The calculation method of the self-attention mechanism comprises the following steps:
q i =W q I,Q=(q 1 ,q 2 ,q 3 ),I=[s 1 ,s 2 ,m d ]
k i =W k I,K=(k 1 ,k 2 ,k 3 ),I=[s 1 ,s 2 ,m d ]
v i =W v I,V=(v 1 ,v 2 ,v 3 ),I=[s 1 ,s 2 ,m d ]
wherein, W q 、W k And W v Respectively representing weight parameters of the full connection layer neural network; q. q of i 、k i And v i Representing queries, keys, and values in the attention mechanism, respectively; q, K and V respectively represent a query matrix, a key matrix and a value matrix; i is formed by 1 ,a 2 And a 3 A matrix composed of three vectors;
the attention score is expressed as: alpha is alpha score =Softmax(K T Q);
The output of the attention mechanism is: b = a score ·V,B={b 1 ,b 2 ,b 3 };
The output of the attention mechanism is processed through a linear mapping: s. the input =FC(B);
Wherein S is input Representing the inputs to the policy network and FC representing the linear mapping implemented by the fully-connected layer neural network.
The invention has the beneficial effects that: an air-ground cooperative network composed of an unmanned aerial vehicle and an intelligent reflecting surface provides communication service for a ground mobile network, the system communication efficiency and the fairness among ground mobile users are comprehensively considered, and an evaluation index based on the throughput priority and the fairness index is designed to balance the communication efficiency and the user fairness; the method is characterized in that the method takes the maximization of the system fair throughput as a target, considers the energy consumption of the unmanned aerial vehicle and provides energy support for the unmanned aerial vehicle by adopting a wireless power transmission technology, and optimizes the flight trajectory of the unmanned aerial vehicle and the phase of intelligent reflection by utilizing a multi-agent deep reinforcement learning optimization algorithm, so that the unmanned aerial vehicle is in the optimal position at each time slot to maximize the system fair throughput.
The invention provides fair communication service for ground mobile users and completes wireless charging of the unmanned aerial vehicles by utilizing the plurality of unmanned aerial vehicles and the intelligent reflecting surfaces fixed on the surface of the building, and provides fair communication service for the ground mobile users under the condition of meeting energy consumption and network connectivity.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flow chart of the auxiliary communication of the drone according to the present invention.
Fig. 2 is an architecture diagram of the auxiliary communication of the unmanned aerial vehicle according to the present invention.
Fig. 3 is a schematic diagram of the information sharing mechanism based on the gating function according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive effort based on the embodiments of the present invention, are within the scope of the present invention.
An energy-carrying fair communication method of an air-to-ground mobile network based on an intelligent reflecting surface is mainly divided into three parts as shown in fig. 2: the method specifically comprises the following steps as shown in figure 1, and the implementation method comprises the following steps:
s1: establishing an air-ground mobile network architecture based on multiple unmanned aerial vehicles and intelligent reflecting surfaces, wherein the air-ground mobile network architecture comprises K ground mobile users, and the set of the K ground mobile users is represented asDeploying D drones and whose set is represented asThe air-ground mobile network architecture is a scene of wireless power transmission and fair communication, the intelligent reflecting surface is a reflecting surface responsible for charging, the unmanned aerial vehicle serves as a mobile base station to provide communication service for ground mobile users (mobile vehicles, mobile terminals, mobile robots and the like), and the intelligent lamp pole serves as an energy source to provide energy transmission for the unmanned aerial vehicle.
S2: a wireless power transfer model is established according to a wireless power transfer technique. The intelligent lamp pole is used as an energy source, two transmission paths of the intelligent lamp-unmanned aerial vehicle direct transmission and the intelligent lamp pole-intelligent reflecting surface-unmanned aerial vehicle indirect transmission are adopted, wireless charging of the unmanned aerial vehicle is achieved by means of a wireless power transmission technology and a path reconstruction technology of the intelligent reflecting surface, and the cruising ability of the unmanned aerial vehicle is prolonged.
In the step S2, the wireless power transmission technology uses an urban infrastructure intelligent lamp pole as an energy source, and a power transmitter of the intelligent lamp pole transmits energy beams to a receiving end of the unmanned aerial vehicle through a transmission medium. Because certain loss is generated in the transmission process and the influence of buildings is easily received, the intelligent reflecting surface is adopted to reconstruct the energy transmission path, and the energy beam emitted by the power transmitter can reach the optical receiver of the unmanned aerial vehicle through two transmission paths of the intelligent lamp post-unmanned aerial vehicle and the intelligent lamp post-intelligent reflecting surface-unmanned aerial vehicle. Thus, the energy harvested at the unmanned end can be expressed as:
wherein,representing the direct channel gain, H, between the drone and the terrestrial mobile users s (t) represents the channel gain between the energy source and the drone,the channel gain between the smart reflective surface and the drone is represented, Θ (t) represents the phase of the smart reflective surface, and η represents the energy conversion factor. Energy ofThe first term of express wisdom lamp to unmanned aerial vehicle's direct transmission, the second term expresses the transmission path through the reconstruction of intelligent plane of reflection, can effectively avoid the influence of shelter from thing to power transmission. After the intelligent reflecting surface is added to reconstruct a transmission path, the receiving power of the receiving end of the unmanned aerial vehicle can be improved on the one hand, and the influence of obstacles on transmission can be avoided on the other hand.
S3: and establishing an energy consumption model of the unmanned aerial vehicle according to the dynamic model and the communication model of the unmanned aerial vehicle. Unmanned aerial vehicle energy consumption mainly divide into at the flight in-process and removes the consumption and provide communication service's communication consumption for the user, in order to guarantee the normal execution of communication task, needs satisfy unmanned aerial vehicle residual energy restraint.
The unmanned aerial vehicle mainly has the energy consumption that removal, communication and internal circuit produced in the process of carrying out the auxiliary communication task, therefore the total energy consumption of unmanned aerial vehicle can be expressed as:
wherein, P t (t) denotes the transmit power of the drone, P c (t) constant circuit power, P, for the drone d (t) represents the propulsion power of the drone at time t, and the propulsion power P d (t) can be represented as:
Wherein,representing the flying speed of the drone at time t, where V max Representing the maximum flying speed of the drone.Respectively representing the drone position at t +1 and t and the time difference between the two moments. P o Representing blade section power, P i Indicating hovering power, v 0 Denotes the average induced speed of the rotor, d 0 Expressing the fuselage drag coefficient, ρ air density, s motor volume, A rotor area, U tip Indicating the linear tip speed.
The remaining energy of the drone is expressed as
Wherein E is max Represents the maximum energy value after the unmanned aerial vehicle is fully charged, E d (t) represents the battery energy remaining at time t for the drone. E c (t) represents the total energy consumption of the drone.
S4: and establishing a wireless communication model. And the intelligent reflecting surface is utilized to reconstruct the channel state between the unmanned aerial vehicle and the ground user, so that the communication quality between the unmanned aerial vehicle and the ground user is improved.
In step S4, the quality of the wireless channel between the unmanned aerial vehicle and the ground mobile user is poor due to the influence of obstacles or other environments, and the transmission rate between the unmanned aerial vehicle and the ground mobile user is increased by reconstructing the channel condition by adjusting the phase and amplitude of the intelligent reflecting surface, so that the transmission rate received by the ground mobile user is represented as:
wherein,a matrix of phase coefficients representing the tamper resistant intelligent reflective surfaces,indicating the phase, M, of an intelligent reflecting surface in wireless information transmission r 、M c 、Respectively representing the number of rectangular reflecting surface rows and reflective elements on columns and the (m) th r ,m c ) The phase of the reflective element. h is a total of UG (t) denotes the channel gain of the transmission link for the unmanned aerial vehicle and the terrestrial mobile subscriber, B k (t) bandwidth allocated to the ground mobile user k by the drone, α d,k (t) indicates whether unmanned plane d is serving ground mobile user k, P t Denotes the transmission power, h RG (t) represents the channel gain between the intelligent reflecting surface and the land mobile user, h UR (t) represents the channel gain between the drone and the intelligent reflecting surface. Sigma 2 Representing gaussian white noise.
Assuming that there is no line-of-sight link between the drone and the ground mobile user, the channel gain h between the drone U-intelligent reflector R, the intelligent reflector R-ground mobile user G and the drone U-ground mobile user G UR (t)、h RG (t) and h UG (t) can be expressed as:
wherein beta is 0 Represents the channel power gain per unit distance, and the distances of U-R, R-G and U-G at time t are respectively represented as D UR (t),D RG (t) and D UG (t) of (d). 2. Alpha and beta represent path loss indexes on the U-R, R-G and U-G links respectively,the portion representing the line of sight in the U-R link, depending on the flight trajectory of the drone at time slot n, may be represented as:
wherein,
and is provided withAndrespectively representing the cosine and sine of the horizontal Angle of Arrival (AoA) of the signal at the intelligent reflecting surface;represents the sine of the signal perpendicular to the AoA at the intelligent reflective surface; λ represents the wavelength of the carrier wave, (x) r ,y r ,z r ) Indicating the position of the intelligent reflecting surface, x d (t)、y d (t)、z d (t) respectively representing the horizontal coordinates and the flying height of the drone,representing the kronecker product.
wherein
Wherein,andcosine and sine of an Angle of horizontal Departure (AoD) representing the signal to the kth user;representing the sine of the vertical AoD of the signal to the kth user. x is the number of k (t)、y k (t) respectively represent horizontal coordinates of the land users.Representing the non line-of-sight portion of the R-G link,representing the random scattering index.
It can be seen from the transmission rate expression received by the ground mobile user that when no intelligent reflection surface is added, the channel between the unmanned aerial vehicle and the user only comprises a direct channel. After the reflecting surface is added, the channel is divided into a direct channel and a part of channels formed by the beams of the intelligent reflecting surface, the two channels are superposed at a user, and the channel propagation environment between the original unmanned aerial vehicle and the user can be improved, so that the transmission rate is improved.
S5: establishing a fair communication model: a fair communication model is established by considering communication efficiency (maximizing system throughput) and fairness among users, and the system throughput is maximized on the premise of guaranteeing the user fairness.
In the step S5, for the system maximization target, the unmanned aerial vehicle adjusts the flight trajectory to enable the unmanned aerial vehicle to be in a position with good channel quality. However, throughput maximization and user fairness are contradictory. The invention provides a fairness index based on a throughput ratio to represent the balance of system throughput and fairness. Defining usersThroughput ratio f of k (t) to measure the importance of the user:
wherein,indicating that user k is in time period [0, t ]]The throughput of the network element(s) is,representing the throughput of all users.
Modeling according to the importance degree of the users can achieve higher throughput, but unfairness of the ground mobile users can be caused, so that the fairness among the users is measured by using Jain's fairness index, and the maximum throughput is achieved on the premise of ensuring the fairness among the users. Selecting users of a service according to priority may improve communication efficiency but may result in unfairness among users. A new evaluation index is designed by balancing communication efficiency and fairness:
in order to measure the importance degree of the ground users, priority based on throughput rate is designed, unfairness is caused if only the priority is considered, and then an evaluation index which simultaneously considers the priority and fairness is obtained according to the priority and fairness index.
S6: and constructing a judgment matrix about fair throughput and energy consumption according to the user service quality grade, solving and normalizing the eigenvalue and the eigenvector of the judgment matrix, and determining the weight coefficients of two sub-targets of fair weighted throughput and energy consumption.
In step S6, the total optimization goal is a weighted sum of two sub-goals regarding the remaining energy of the drone and the system fair throughput, which are mutually opposite. In order to reasonably determine the weight coefficient of the sub-targets, the invention designs a weight coefficient determination method based on the user service quality. First, in order to establish a multi-objective optimization problem for communication and energy transmission, energy consumption sub-objectives and fair throughput are hierarchically quantized on the basis of user qos criteria according to attributes of tasks (task volume, transmission delay, computation power, etc.) as shown in table 1, where the hierarchy y 1 ,y 2 ,y 3 ,y 4 ]The importance is represented by 1,2,3 and 4 grades, and the higher grade represents the higher quality of service of the user. According to the grade quantization table, taking a real-time data task as an example, a judgment matrix about energy consumption and fair throughput is constructed:
the eigenvalue and the eigenvector of the judgment matrix are solved and normalized by a Jacobi method, and the weight system corresponding to the two sub-targets can be obtainedNumber [ w ] 1 ,w 2 ]。
TABLE 1 QOS class quantization table for different tasks with respect to energy consumption and throughput
S7: constructing a multi-objective optimization problem: modeling the unmanned aerial vehicle energy-carrying communication problem into a multi-target integer non-convex optimization problem with fair throughput and maximized unmanned aerial vehicle residual energy, and re-describing the problem into a Markov game process, and solving the complex multi-target optimization problem through multi-agent deep reinforcement learning.
In the step S7, the unmanned aerial vehicle energy-carrying fair communication is modeled into a multi-objective optimization problem:
s.t.C1:E d (0)=E max ,E d (T t )=E min ,
C4:R GU (t)≥γ dk ,
wherein u is d (t) represents a position of the drone;a utility function representing the weighted throughput and the residual energy composition; t is a unit of t Representing a task execution time; e t (0) Representing the initial electric quantity of the unmanned aerial vehicle; e max Representing a maximum battery capacity when the drone is fully charged; e d (T t ) Representing the remaining electric quantity when the unmanned aerial vehicle task is finished; e min Representing the minimum electric quantity required by safe return after the unmanned aerial vehicle executes the task; gamma ray dk Represents a transmission rate minimum threshold; u. of i (t) and u j (t) respectively representing the positions of the unmanned planes i and j at the time t; x is the number of d (t)、x k (t)、y d (t)、y k (t)、X min 、X max 、Y min 、Y max Coordinates representing the drone and the user, respectively, and boundary values for the entire rectangular task area.
Since the position of the drone is continuously changing, its optimization variables are continuous and there is a non-linear coupling. To make a continuous problemBecomes easy to solve, the invention divides the whole task execution time into N t A time slot, each time slot having a length ofThus continuing the problemCan be converted into discrete problems
s.t.C1~C7
The above problem is still a non-convex integer optimization problem, and the conventional optimization algorithm has high computational complexity and is not easy to solve. Thus, the problem of dispersionRe-described as a multi-agent Markov game process < S, A, P, R, γ >. The Markov game process comprises five parts, namely a state set S, an action set A, a reward function R, a state transition probability function P and a reward discount factor gamma. Its state, actions and rewards are defined as follows:
in time slot n ∈ [0, N t ]Internal stateThe device consists of four parts, wherein,the coordinates of the drone base station at time slot n are indicated.The coordinates of the terrestrial mobile subscriber in time slot n are indicated.Representing the remaining energy consumption of the drone, and Θ (n) representing the phase of the intelligent reflecting surface.
Acting within a time slot nMainly consists of two parts, dist d (n)∈[0,V d (t)δ t ]Indicating the distance the drone base station is flying during time slot n.Indicates that there is noAnd the direction of the flight of the man-machine base station in the time slot n. Δ Θ is the amount of change in the phase of the intelligent reflecting surface. V d (t) the flight speed of the drone.
The goal of the action taken by the agent is to maximize the system reward, so the setting of the reward function plays an important role in multi-agent reinforcement learning, and the reward of the unmanned aerial vehicle base station mainly comprises the following parts:
fair throughputIn the multi-drone base station assisted fair communication problem, each drone base station has the same objective, namely maximizing the global fair throughput and the remaining energy of the drones.
Coverage rewardsIn order to accelerate the convergence speed of the algorithm, the coverage reward of the unmanned aerial vehicle is designed in a reward function, namely the coverage reward e d,k Is in direct proportion to the number of users covered by the unmanned aerial vehicle. Wherein e d,k =1 indicates that user k can be covered by drone d, whereas e d,k =0。
Punishing: when satisfying following condition, the unmanned aerial vehicle basic station will receive punishment. (1) Unmanned aerial vehicle departure mission boundary regions, e.g.Wherein X min 、X max 、Y min 、Y max And the values of the abscissa and the ordinate of the task area range are represented. (2) Collision of drone i with drone j, e.g. | | u i (n)-u j (n)|| 2 ≥d min In which d is min Representing a safe distance threshold. (3) When the energy consumption of the drone is below a set value, e.g. E d (t)≤E min . By defining a binary variable ξ l E {0,1} indicates whether the above is violated. If xi l That =1,l ∈ {1,2,3} indicates that in violation of the above, the drone is given a fixed penalty p l ,l∈{1,2,3}。
In summary, the reward function can be expressed as
r=r 1 +r 2 -ξ 1 p 1 -ξ 2 p 2 -ξ 3 p 3
In the Markov game process, the agent aims to maximize the reward function through the optimal self strategy pi, so the discrete problemCan be re-described as
s.t.C1~C7
Wherein,s and a respectively represent the calculation of the expected operation, the state space and the action space; s and a are the concatenation of the state space and the action space of all agents described above.
The goal of the agent in the deep reinforcement learning is to maximize the value of the reward obtained by the agent, and the problem can be converted into a maximization problem. In addition, the intelligent agent maximizes the reward value through the strategy of the intelligent agent, namely the executed action, so that the optimization target of the intelligent agent can be used as the reward of the intelligent agent, the optimization variable can be used as the action of the intelligent agent, and the unmanned aerial vehicle and the environment interact to search the optimal strategy of the intelligent agent through the state space with reasonable design, so that the reward value is maximized.
The energy quantum target in fig. 1 is to maximize the remaining energy consumption of the drone, and the throughput quantum target is to maximize fair throughput. The optimization variables of the whole problem mainly comprise the phase of the intelligent reflecting surface and the position of the unmanned aerial vehicle. The positions of the unmanned aerial vehicles and the phases of the reflecting surfaces are updated according to the magnitude of the reward values in reinforcement learning.
S8: and updating the state of the unmanned aerial vehicle based on an information sharing mechanism of the gate control unit, and inputting a strategy network to obtain the action to be executed by the unmanned aerial vehicle. And storing the state information of all the unmanned planes into a memory, wherein each unmanned plane can access the memory to read the state information of other intelligent bodies when executing actions.
In the step S8, an information sharing mechanism based on a gate control unit is designed to solve the problem of uncertainty of the policy due to partial observability, as shown in fig. 3. The mechanism is implemented by a central memory with a memory capacity of MTo establish state information sharing, the memory is used to store the collective state information m of the unmanned aerial vehicle M . After joining the information sharing mechanism, the policy of each drone becomesThat is, the strategy of the unmanned aerial vehicle does not depend on the observation value s of the unmanned aerial vehicle at the moment d And also to the information in the memory. The information sharing mechanism mainly comprises an encoding operation, a reading operation, a writing operation and an action selection part. Each unmanned aerial vehicle sends its own state s d Mapping to an embedded vector representing the current state:
After encoding the current information, the drone performs a read operation extraction stored in the central memoryThe related information in (1). By generating a context vector h d To capture an embedded vector e d Temporal and spatial information of
Wherein,network parameters representing linear mapping, H and E respectively representing context vector H d And embedding vector e d Of (c) is calculated. Embedded vector e of agent observations d Context vector h d And a current central storageRespectively, contains different information, which are combined as input to learn a gating mechanism:
where σ (·) is a sigmoid function, [ e ] d ,h d ,m]Representing the concatenation of three vectors and M representing the memory size. k is a radical of d Adjusting information read from a central memory as a weighting factor
r d =m⊙k d
Wherein, l represents a hadamard product.
The agent generates a candidate memory content through nonlinear mapping according to the coding of the state value of the agent and the information of the current shared memory:
wherein,is a network parameter. Input gate g d For adjusting the contents of the candidate memory, f d Deciding which information to useWhich information needs to be retained and discarded, these operations can be expressed as:
where σ represents the sigmode activation function,Respectively representing the neural network parameters to be trained.
Then, the unmanned aerial vehicle d finally generates new updated information through the weighted combination of the new and old information:
m'=g d ⊙c d +f d ⊙m
after the read-write operation is finished, the unmanned aerial vehicle takes the code of the current self state and the information read from the memory as the input of a strategy network, and the strategy network outputs the action to be executed by the unmanned aerial vehicle
Wherein e is d Represents an observed value s d A mapping vector of states. r is d Which represents the information read from the memory device,representing a policy function. The policy network is part of the Actor network.
S9: and constructing an Actor network of state decomposition-expansion-aggregation to decompose and reduce the dimension of the state information. Decoupling the state of the unmanned aerial vehicle according to different categories, carrying out dimension expansion on the state with smaller dimension, and then carrying out state aggregation on the processed sub-states according to different correlation degrees by using a multi-head attention mechanism.
In step S9, the final action of the drone depends on the position information of the drone and the ground mobile user, the energy information of the drone, and the phase of the intelligent reflective surface. If all the status information(s) is directly used d The position of the drone, the position of the user, and the remaining capacity of the drone) and shared information (contents read from a memory by the agent through a read operation) are input to the Actor network, and it is difficult to output an ideal policy due to the unbalanced dimensionality of the status information. Therefore, the invention designs a new Actor network architecture of state decomposition-extension-aggregation. The network structure mainly comprises three parts, namely state decomposition and expansion, state linear mapping dimension space reduction and multi-head attention mechanism state aggregation. The state decomposition refers to decomposing different types of state information (position information, energy information and phase information), and since the dimensions of the different types of state information are different, all the state information is expanded to the same dimension by adopting a dimension expansion technology. The aggregation is to aggregate and linearly map the sub-states after the decomposition into a low-dimensional input vector according to different degrees of correlation. And (3) an attention mechanism is adopted to enable the unmanned aerial vehicle to learn the strategy, and then the strategy is spliced into an integral vector according to the importance degree (essentially, a weight coefficient) of different sub-states.
In addition, the output action of the unmanned aerial vehicle is closely related to the state information, for example, when the remaining capacity of the unmanned aerial vehicle is lower than a safety threshold, the action output by the unmanned aerial vehicle is biased to ensure that the charging capacity of the unmanned aerial vehicle is maximized rather than the throughput is maximized. Therefore, the influence of different types of state information on the action of the unmanned aerial vehicle is different, and a state information selection strategy based on a self-attention mechanism is designed based on the method. Position state information, remaining capacity state information and state information read from a memory after state decomposition, dimension expansion and linear mapping processing are taken as three vectors a 1 、a 2 And a 3 . The calculation formula of the self-attention mechanism is as follows
q i =W q I,Q=(q 1 ,q 2 ,q 3 ),I=[s 1 ,s 2 ,m d ]
k i =W k I,K=(k 1 ,k 2 ,k 3 ),I=[s 1 ,s 2 ,m d ]
v i =W v I,V=(v 1 ,v 2 ,v 3 ),I=[s 1 ,s 2 ,m d ]
Wherein, W q 、W k And W v Respectively, representing the weight parameters of the fully-connected layer neural network. q. q of i 、k i And v i Representing queries, keys and values in the attention mechanism, respectively. Q, K, V represent the query matrix, key matrix, and value matrix, respectively. I is formed by 1 ,a 2 And a 3 A matrix of three vectors. Thus, the attention score may be expressed as
α score =Softmax(K T Q)
The attention score may reflect the correlation between any two vectors. The output of the attention mechanism can be expressed as:
B=α score ·V,B={b 1 ,b 2 ,b 3 }
for better input strategy network (full connection layer neural network), the invention processes the output of attention mechanism through linear mapping
S input =FC(B)
Wherein S is input Representing the inputs to the policy network and FC representing the linear mapping implemented by the fully-connected layer neural network. Therefore, the convergence difficulty caused by dimension imbalance and dimension explosion can be solved by carrying out the aggregation of the decoupling-expanding-attention mechanism on the input of the strategy network, and in addition, the correlation among the sub-states can be well represented by adopting the attention mechanism. Then, the drone focuses on the sub-state with stronger relevance when performing the action, so as to help the drone find the optimal strategy.
Firstly, designing an air-ground mobile network architecture based on an unmanned aerial vehicle and an intelligent reflecting surface, and establishing an energy transmission model and a communication model based on the intelligent reflecting surface. In order to balance communication efficiency and user fairness, a fairness index based on a throughput ratio is designed to describe user-level fairness. And finally, providing a multi-agent deep reinforcement learning optimization algorithm, and enabling the total fair throughput of the system and the residual energy of the unmanned aerial vehicle to be maximum by optimizing the position of the unmanned aerial vehicle and the phase of an intelligent reflecting surface. In addition, in order to solve the problems of strategy uncertainty and algorithm training difficulty caused by partial observability and state dimension imbalance, the invention also designs an information sharing mechanism based on a gating function and a novel Actor network architecture of state decoupling, state expansion and state aggregation, and the unmanned aerial vehicle can pay attention to more important state information by self by adopting a self-attention mechanism during aggregation.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and should not be taken as limiting the scope of the present invention, which is intended to cover any modifications, equivalents, improvements, etc. within the spirit and scope of the present invention.
Claims (10)
1. An air-ground mobile network energy-carrying fair communication method based on an intelligent reflecting surface is characterized by comprising the following steps:
s1: establishing an air-ground mobile network architecture based on a plurality of unmanned aerial vehicles and an intelligent reflecting surface, wherein the air-ground mobile network architecture comprises K ground mobile users and D unmanned aerial vehicles;
s2: establishing a wireless power transmission model according to a wireless power transmission technology: the intelligent lamp post is used as an energy source, and two transmission paths of direct transmission of the intelligent lamp-the unmanned aerial vehicle and indirect transmission of the intelligent lamp post-the intelligent reflecting surface-the unmanned aerial vehicle are adopted, so that the unmanned aerial vehicle is wirelessly charged;
s3: establishing an energy consumption model of the unmanned aerial vehicle according to the dynamic model and the communication model of the unmanned aerial vehicle;
s4: reconstructing a channel state between the unmanned aerial vehicle and a ground user by using the intelligent reflecting surface, and establishing a wireless communication model;
s5: establishing a fair communication model: a fair communication model is established by considering communication efficiency and fairness among users, and system throughput is maximized on the premise of guaranteeing user fairness;
s6: constructing a judgment matrix about fair throughput and energy consumption according to the user service quality grade, solving and normalizing eigenvalues and eigenvectors of the judgment matrix, and determining weight coefficients of two sub-targets of fair weighted throughput and energy consumption;
s7: modeling the unmanned aerial vehicle energy-carrying communication problem into a multi-target integer non-convex optimization problem with fair throughput and maximized unmanned aerial vehicle residual energy, and re-describing the problem into a Markov game process, solving the complex multi-target optimization problem through multi-agent deep reinforcement learning, and updating the position of the unmanned aerial vehicle and the phase of an intelligent reflecting surface.
2. The air-ground mobile network energy-carrying fair communication method based on the intelligent reflecting surface as claimed in claim 1, wherein the set of ground mobile users is represented asThe set of drones is denoted asThe intelligent reflecting surface is a reflecting surface responsible for charging, the unmanned aerial vehicle serves as a mobile base station to provide communication service for ground mobile users, and the intelligent lamp pole serves as an energy source to provide energy transmission for the unmanned aerial vehicle;
the method for constructing the wireless power transmission model in the step S2 comprises the following steps: an intelligent reflecting surface is adopted to reconstruct an energy transmission path, an energy beam emitted by a power emitter of the intelligent lamp pole reaches a light receiver of the unmanned aerial vehicle through two transmission paths of the intelligent lamp pole-the unmanned aerial vehicle and the intelligent lamp pole-the intelligent reflecting surface-the unmanned aerial vehicle, and the energy harvested by the unmanned aerial vehicle is as follows:
wherein,representing the direct channel gain, H, between the drone and the terrestrial mobile users s (t) represents the channel gain between the energy source and the drone,the channel gain between the intelligent reflecting surface and the unmanned aerial vehicle is represented, theta (t) represents the phase of the intelligent reflecting surface, and eta represents the energy conversion coefficient.
3. The air-ground mobile network energy-carrying fair communication method based on the intelligent reflecting surface as claimed in claim 1, wherein the method for establishing the energy consumption model of the unmanned aerial vehicle in S3 is as follows:
unmanned aerial vehicle has the energy resource consumption that removes consumption, communication consumption and internal circuit produced at the in-process of carrying out the auxiliary communication task, then the total energy consumption of unmanned aerial vehicle is:
wherein, P t (t) denotes the transmit power of the drone, P c (t) constant circuit power, P, for the drone d (t) represents the propulsion power of the drone at time t, and the propulsion power P d (t) is expressed as:
wherein,representing the flying speed, V, of the drone at time t max Representing the maximum flight speed of the drone;individual watchShowing the positions of the unmanned aerial vehicle at the t +1 moment and the t moment and the time difference between the two moments; p o Representing blade section power, P i Indicating hovering power, v 0 Denotes the average induced speed of the rotor, d 0 Expressing the fuselage drag coefficient, ρ air density, s motor volume, A rotor area, U tip Expressing the linear speed of a blade tip;
the remaining energy of the drone is:
4. The air-ground mobile network energy-carrying fair communication method based on the intelligent reflecting surface as claimed in claim 2 or 3, wherein the method for establishing the wireless communication model in the step S4 is as follows: the phase position and the amplitude value of the intelligent reflecting surface are adjusted to reconstruct the channel condition, so that the transmission rate between the unmanned aerial vehicle and the user is improved, and the transmission rate received by the ground mobile user is as follows:
wherein,a phase coefficient matrix representing the anti-jamming intelligent reflecting surface,indicating the phase, M, of an intelligent reflecting surface in wireless information transmission r 、M c Respectively representing the number of reflecting elements on the rows and columns of the intelligent reflecting surface,(m) th representing an intelligent reflecting surface r ,m c ) The phase of the reflective element; h is a total of UG (t) denotes the channel gain of the transmission link for the unmanned aerial vehicle and the terrestrial mobile subscriber, B k (t) bandwidth allocated to the ground mobile user k by the drone, α d,k (t) indicates whether unmanned plane d is serving ground mobile user k, P t Denotes the transmission power, h RG (t) represents the channel gain between the intelligent reflecting surface and the land mobile user, h UR (t) represents the channel gain, σ, between the drone and the smart reflecting surface 2 Representing gaussian white noise.
5. The air-ground mobile network energy-carrying fair communication method based on the intelligent reflector as claimed in claim 4, wherein a channel gain h between the unmanned plane U and the intelligent reflector R, the intelligent reflector R and the ground mobile subscriber G is set to be equal to a channel gain h between the unmanned plane U and the intelligent reflector R and between the unmanned plane U and the ground mobile subscriber G, provided that no line-of-sight link exists between the unmanned plane and the ground mobile subscriber UR (t)、h RG (t) and h UG (t) are respectively:
wherein beta is 0 Denotes the channel power gain per unit distance, D UR (t)、D RG (t) and D UG (t) respectively representing unmanned plane U-intelligent reflector R and intelligent reflector R-ground movementThe distance between the user G and the unmanned aerial vehicle U-ground mobile user G at the time t is respectively expressed as; 2. alpha and beta represent path loss indexes on links of an unmanned plane U-intelligent reflecting surface R, an intelligent reflecting surface R-ground mobile user G and an unmanned plane U-ground mobile user G respectively;the part representing the line of sight in the U-intelligent reflector R link of the drone, depending on the flight trajectory of the drone in time slot n, is represented as:
wherein,
and isAndrespectively representing cosine and sine of a horizontal arrival angle of the signal on the intelligent reflecting surface;a sine representing the perpendicular angle of arrival of the signal at the intelligent reflecting surface; λ represents the wavelength of the carrier wave, (x) r ,y r ,z r ) Indicating the position of the intelligent reflecting surface, x d (t)、y d (t)、z d (t) represents the horizontal coordinate and the flying height of the drone respectively,represents the kronecker product;
part of line-of-sight in the link of the intelligent reflector R-ground mobile subscriber GComprises the following steps:
wherein
Wherein,andcosine and sine representing the horizontal departure angle of the signal to the kth user;a sine representing the vertical departure angle of the signal to the kth user; x is the number of k (t)、y k (t) horizontal coordinates representing the ground users, respectively;representing the non line-of-sight portion of the R-G link,is represented byThe machine scattering index.
6. The intelligent reflecting surface-based air-ground mobile network energy-carrying fair communication method according to any one of claims 2,3 and 5, wherein the method for establishing the fair communication model in the step S5 comprises the following steps: fair index representation maximization system throughput and fairness balance based on throughput ratio, and ground mobile user definitionThroughput ratio f k (t) measure the importance of terrestrial mobile users:
wherein,representing the terrestrial mobile user k during the time period [0, t]The throughput of the network element(s) is,represents the throughput of all terrestrial mobile users;
measuring fairness among users by using Jain's fairness index, and the new evaluation index for balancing communication efficiency and fairness is as follows:
7. the air-ground mobile network energy-carrying fair communication method based on the intelligent reflecting surface as claimed in claim 6, wherein the method for determining the weight coefficient in the step S6 is as follows: and performing grade quantization on the energy consumption sub-target and the fair throughput by taking the user service quality as a standard according to the attributes of the tasks to obtain a grade quantization table as follows:
Wherein the rank [ y 1 ,y 2 ,y 3 ,y 4 ]Represents the level of importance; constructing a table of energy consumption and energy consumption according to the grade quantization tableJudgment matrix of fair throughput:
solving the eigenvalue and the eigenvector of the judgment matrix by a Jacobi method and normalizing the eigenvalue and the eigenvector to obtain a weight coefficient [ w ] corresponding to two sub-targets 1 ,w 2 ]。
8. The air-ground mobile network energy-carrying fair communication method based on the intelligent reflecting surface as claimed in claim 7, wherein the multi-objective integer non-convex optimization problem in the step S7 is as follows:
s.t.C1:E d (0)=E max ,E d (T t )=E min ,
C4:R GU (t)≥γ dk ,
wherein u is d (t) represents a position of the drone;a utility function representing the weighted throughput and the residual energy composition; t is t Representing the task execution time; e t (0) The electric quantity of the unmanned aerial vehicle at the initial time is represented; e max Representing the maximum battery capacity of the unmanned aerial vehicle when the unmanned aerial vehicle is fully charged; e d (T t ) Representing the remaining electric quantity when the unmanned aerial vehicle task is finished; e min Representing the minimum electric quantity required by safe return after the unmanned aerial vehicle executes the task; gamma ray dk Represents a transmission rate minimum threshold; u. of i (t) and u j (t) respectively representing the positions of the unmanned planes i and j at the time t; x is the number of d (t)、x k (t)、y d (t)、y k (t) coordinates, X, representing the unmanned aerial vehicle and the ground mobile user, respectively min 、X max 、Y min 、Y max The boundary value of the whole rectangular task area is obtained;
dividing the whole task execution time into N t A time slot, each time slot having a length ofWill continue to questionConversion to the discrete problem:
s.t.C1~C7
solve the problem of dispersionRe-describing the Markov game process of a multi-agent < S, A, P, R, gamma >, wherein S is a state set, A is an action set, R is a reward function, P is a state transition probability function, and gamma is a reward discount factor;
the multi-agent deep reinforcement learning method comprises the following steps:
in time slot n ∈ [0, N t ]Internal stateWherein,the coordinates of the drone at time slot n are indicated,the coordinates of the terrestrial mobile subscriber in time slot n are represented,representing the residual energy consumption of the unmanned aerial vehicle, and theta (n) representing the phase of the intelligent reflecting surface;
within a time slot n an action a d ={dist d (n),Δ Θ }, wherein dist d (n)∈[0,V d (t)δ t ]The flying distance of the unmanned aerial vehicle base station in the time slot n is represented;representing the flight direction of the unmanned aerial vehicle base station in the time slot n; delta theta is the variation of the phase of the intelligent reflecting surface; v d (t) the flight speed of the drone;
the reward function is r = r 1 +r 2 -ξ 1 p 1 -ξ 2 p 2 -ξ 3 p 3 ;
Wherein the fair throughputCoverage rewardse d,k =1 indicates that user k can be covered by drone d, whereas e d,k =0;
Punishing: when satisfying following condition, the unmanned aerial vehicle basic station will receive punishment: (1) Unmanned aerial vehicle departure mission boundary regions, e.g.Wherein X min 、X max 、Y min 、Y max The values of the abscissa and the ordinate of the task area range are represented; (2) Collision of unmanned aerial vehicle i with unmanned aerial vehicle j, i.e. | | u i (n)-u j (n)|| 2 ≥d min In which d is min Represents a safe distance threshold; (3) When the energy consumption of the unmanned aerial vehicle is lower than a set value, namely E d (t)≤E min (ii) a By defining a binary variable ξ l E {0,1} indicates whether the above condition l is violated; if xi l The condition of violation l is represented by =1,l epsilon {1,2,3}, and the unmanned plane is given a fixed penalty p l ,l∈{1,2,3};
In the Markov game process, the intelligent agent maximizes the reward function through the optimal self strategy pi, and the problem is dispersedIs described again as
s.t.C1~C7
9. The intelligent reflector-based air-ground mobile network energy-carrying fair communication method as claimed in claim 8, wherein the state of the unmanned aerial vehicle is updated based on an information sharing mechanism of a gate control unit, and a policy network is input to obtain actions required to be executed by the unmanned aerial vehicle; and constructing an Actor network of state decomposition-expansion-aggregation to decompose and reduce the dimension of state information, and then performing state aggregation on the processed sub-states according to different correlation degrees by using a multi-head attention mechanism.
10. The air-ground mobile network energy-carrying fair communication method based on the intelligent reflecting surface as claimed in claim 9, wherein the gating cell-based information sharing mechanism is implemented by:
by means of a memory with a memory capacity of MEstablishing state information sharing, wherein a memory is used for storing collective state information m of the unmanned aerial vehicle belongs to R M (ii) a The strategy of each unmanned aerial vehicle becomes
Each unmanned aerial vehicle sends its own state s d Mapping to an embedded vector representing the current state:wherein,is a network parameter ofA neural network of (a);
the unmanned aerial vehicle executes the read operation to be extracted and stored in the memoryBy generating a context vector h d To capture an embedded vector e d The spatio-temporal information of (a):
wherein,parameters representing a linear mapping network, H, E representing a context vector H, respectively d And embedding vector e d Dimension (d);
embedded vector e of joint agent observed value d Context vector h d And the current memoryAs input, learn a gating mechanism:
where σ (·) is a sigmoid function, [ e ] d ,h d ,m]Representing the concatenation of three vectors, k d As a weighting factor;
regulating the reading of information r from memory by gating mechanism d =m⊙k d (ii) a Wherein, l represents a hadamard product.
The agent generates a candidate memory content through nonlinear mapping according to the coding of the state value of the agent and the information of the current shared memory:
wherein,is a network parameter; input gate g d For adjusting the content of the candidate memory, f d Deciding information that needs to be retained and discarded, and:
where σ represents the sigmode activation function,Respectively representing neural network parameters needing to be trained;
then, the unmanned aerial vehicle d generates new updated information by weighted combination of the new and old information: m' = g d ⊙c d +f d ⊙m;
The unmanned aerial vehicle takes the code of the current self state and the information read from the memory as the input of the strategy network, and the strategy network outputs the action to be executed by the unmanned aerial vehicle
Wherein r is d Which represents the information read from the memory device,representing a policy function;
the state decomposition of the Actor network decomposes different types of state information and adopts a dimension expansion technology to expand all the state information to the same dimension; the aggregation is to aggregate and linearly map each sub-state after decomposition into a low-dimensional input vector according to different correlation degrees;
selecting the state information based on a state information selection strategy of a self-attention mechanism, and taking the position state information, the residual capacity state information and the state information read from the memory after the state decomposition, the dimension expansion and the linear mapping processing as three vectors a 1 、a 2 And a 3 The calculation method of the self-attention mechanism comprises the following steps:
q i =W q I,Q=(q 1 ,q 2 ,q 3 ),I=[s 1 ,s 2 ,m d ]
k i =W k I,K=(k 1 ,k 2 ,k 3 ),I=[s 1 ,s 2 ,m d ]
v i =W v I,V=(v 1 ,v 2 ,v 3 ),I=[s 1 ,s 2 ,m d ]
wherein, W q 、W k And W v Respectively representing weight parameters of the full connection layer neural network; q. q.s i 、k i And v i Representing queries, keys, and values in the attention mechanism, respectively; q, K and V respectively represent a query matrix, a key matrix and a value matrix; i is formed by 1 ,a 2 And a 3 A matrix composed of three vectors;
the attention score is expressed as: alpha is alpha score =Softmax(K T Q);
The output of the attention mechanism is: b = a score ·V,B={b 1 ,b 2 ,b 3 };
The output of the attention mechanism is processed through linear mapping: s. the input =FC(B);
Wherein S is input Representing the inputs to the policy network and FC representing the linear mapping implemented by the fully-connected layer neural network.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211472603.2A CN115802313B (en) | 2022-11-16 | 2022-11-16 | Air-ground mobile network energy-carrying fair communication method based on intelligent reflecting surface |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211472603.2A CN115802313B (en) | 2022-11-16 | 2022-11-16 | Air-ground mobile network energy-carrying fair communication method based on intelligent reflecting surface |
Publications (2)
Publication Number | Publication Date |
---|---|
CN115802313A true CN115802313A (en) | 2023-03-14 |
CN115802313B CN115802313B (en) | 2024-06-28 |
Family
ID=85440416
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202211472603.2A Active CN115802313B (en) | 2022-11-16 | 2022-11-16 | Air-ground mobile network energy-carrying fair communication method based on intelligent reflecting surface |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN115802313B (en) |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111786713A (en) * | 2020-06-04 | 2020-10-16 | 大连理工大学 | Unmanned aerial vehicle network hovering position optimization method based on multi-agent deep reinforcement learning |
WO2021143052A1 (en) * | 2020-01-13 | 2021-07-22 | 电子科技大学中山学院 | Multi-dimensional resource joint scheduling optimization method for wireless energy transmission network |
US20210255641A1 (en) * | 2019-09-30 | 2021-08-19 | South China University Of Technology | Method for designing three-dimensional trajectory of unmanned aerial vehicle based on wireless power transfer network |
CN113645635A (en) * | 2021-08-12 | 2021-11-12 | 大连理工大学 | Design method of intelligent reflector-assisted high-energy-efficiency unmanned aerial vehicle communication system |
CN114124705A (en) * | 2021-11-26 | 2022-03-01 | 重庆邮电大学 | Resource allocation method based on max-min fairness for unmanned aerial vehicle-assisted backscatter communication system |
CN114422056A (en) * | 2021-12-03 | 2022-04-29 | 北京航空航天大学 | Air-ground non-orthogonal multiple access uplink transmission method based on intelligent reflecting surface |
CN114826380A (en) * | 2022-04-22 | 2022-07-29 | 昆明理工大学 | Unmanned aerial vehicle-assisted air-ground communication optimization algorithm based on deep reinforcement learning algorithm |
CN115226255A (en) * | 2022-07-15 | 2022-10-21 | 南京邮电大学 | Unmanned aerial vehicle auxiliary communication working mode adjusting method based on intelligent reflecting surface |
-
2022
- 2022-11-16 CN CN202211472603.2A patent/CN115802313B/en active Active
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20210255641A1 (en) * | 2019-09-30 | 2021-08-19 | South China University Of Technology | Method for designing three-dimensional trajectory of unmanned aerial vehicle based on wireless power transfer network |
WO2021143052A1 (en) * | 2020-01-13 | 2021-07-22 | 电子科技大学中山学院 | Multi-dimensional resource joint scheduling optimization method for wireless energy transmission network |
CN111786713A (en) * | 2020-06-04 | 2020-10-16 | 大连理工大学 | Unmanned aerial vehicle network hovering position optimization method based on multi-agent deep reinforcement learning |
CN113645635A (en) * | 2021-08-12 | 2021-11-12 | 大连理工大学 | Design method of intelligent reflector-assisted high-energy-efficiency unmanned aerial vehicle communication system |
CN114124705A (en) * | 2021-11-26 | 2022-03-01 | 重庆邮电大学 | Resource allocation method based on max-min fairness for unmanned aerial vehicle-assisted backscatter communication system |
CN114422056A (en) * | 2021-12-03 | 2022-04-29 | 北京航空航天大学 | Air-ground non-orthogonal multiple access uplink transmission method based on intelligent reflecting surface |
CN114826380A (en) * | 2022-04-22 | 2022-07-29 | 昆明理工大学 | Unmanned aerial vehicle-assisted air-ground communication optimization algorithm based on deep reinforcement learning algorithm |
CN115226255A (en) * | 2022-07-15 | 2022-10-21 | 南京邮电大学 | Unmanned aerial vehicle auxiliary communication working mode adjusting method based on intelligent reflecting surface |
Non-Patent Citations (6)
Title |
---|
HUNG NGUYEN-KHA等: "Joint UAV Placement and IRS Phase Shift Optimization in Downlink Networks", 《 IEEE ACCESS》, 13 October 2022 (2022-10-13) * |
YI ZHOU: "Latency Minimization for Secure Intelligent Reflecting Surface Enhanced Virtual Reality Delivery Systems", 《IEEE WIRELESS COMMUNICATIONS LETTERS》, 15 March 2022 (2022-03-15) * |
周毅等: "基于深度强化学习的无人机自主部署及能效优化策略", 《物联网学报》, 30 June 2019 (2019-06-30) * |
李思贤: "基于智能反射面的无人机通信研究", 《信息科技》, 15 January 2022 (2022-01-15) * |
白艳宇;吴颖;: "无线激光通信网络的上行接入带宽动态分配", 激光杂志, no. 08, 25 August 2017 (2017-08-25) * |
陈前斌;管令进;李子煜;王兆堃;杨恒;唐伦;: "基于深度强化学习的异构云无线接入网自适应无线资源分配算法", 电子与信息学报, no. 06, 15 June 2020 (2020-06-15) * |
Also Published As
Publication number | Publication date |
---|---|
CN115802313B (en) | 2024-06-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Huang et al. | Deep reinforcement learning for UAV navigation through massive MIMO technique | |
CN113162679B (en) | DDPG algorithm-based IRS (intelligent resilient software) assisted unmanned aerial vehicle communication joint optimization method | |
Yan et al. | Task allocation and route planning of multiple UAVs in a marine environment based on an improved particle swarm optimization algorithm | |
CN114690799A (en) | Air-space-ground integrated unmanned aerial vehicle Internet of things data acquisition method based on information age | |
CN110049566B (en) | Downlink power distribution method based on multi-unmanned-aerial-vehicle auxiliary communication network | |
CN110166154A (en) | A kind of software radio spectrum monitoring knowledge method for distinguishing neural network based | |
Dong et al. | Joint optimization of deployment and trajectory in UAV and IRS-assisted IoT data collection system | |
CN114142908B (en) | Multi-unmanned aerial vehicle communication resource allocation method for coverage reconnaissance task | |
CN115659803A (en) | Intelligent unloading method for computing tasks under unmanned aerial vehicle twin network mapping error condition | |
CN114980169A (en) | Unmanned aerial vehicle auxiliary ground communication method based on combined optimization of track and phase | |
Liao et al. | Energy minimization for UAV swarm-enabled wireless inland ship MEC network with time windows | |
CN115866574A (en) | Disaster area rescue-oriented multi-unmanned aerial vehicle flight search and rescue trajectory optimization method | |
CN114158010B (en) | Unmanned aerial vehicle communication system and resource allocation strategy prediction method based on neural network | |
Li et al. | DNN partition and offloading strategy with improved particle swarm genetic algorithm in VEC | |
Li et al. | TaskPOI priority-based energy balanced multi-UAVs cooperative trajectory planning algorithm in 6G networks | |
CN115802313A (en) | Air-ground mobile network energy-carrying fair communication method based on intelligent reflecting surface | |
CN117053790A (en) | Single-antenna unmanned aerial vehicle auxiliary communication flight route-oriented planning method | |
Sun et al. | Semantic-driven computation offloading and resource allocation for uav-assisted monitoring system in vehicular networks | |
CN116896777A (en) | Unmanned aerial vehicle group general sense one-body energy optimization method based on reinforcement learning | |
Zhang et al. | Offloading strategy for UAV-assisted mobile edge computing based on reinforcement learning | |
Tarekegn et al. | Channel Quality Estimation in 3D Drone Base Station for Future Wireless Network | |
Xu et al. | Energy optimization in multi-UAV-assisted edge data collection system | |
Xu et al. | Learning-Based Energy Minimization Optimization for IRS-Assisted Master-Auxiliary-UAV-Enabled Wireless-Powered IoT Networks | |
Zhang et al. | Learning-based trajectory design and time allocation in UAV-supported wireless powered NOMA-IoT networks | |
Wang et al. | Safety constrained trajectory optimization for completion time minimization for uav communications |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |