CN116684925B - Unmanned aerial vehicle-mounted intelligent reflecting surface safe movement edge calculation method - Google Patents
Unmanned aerial vehicle-mounted intelligent reflecting surface safe movement edge calculation method Download PDFInfo
- Publication number
- CN116684925B CN116684925B CN202310904821.7A CN202310904821A CN116684925B CN 116684925 B CN116684925 B CN 116684925B CN 202310904821 A CN202310904821 A CN 202310904821A CN 116684925 B CN116684925 B CN 116684925B
- Authority
- CN
- China
- Prior art keywords
- aerial vehicle
- unmanned aerial
- irs
- relay
- expressed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004364 calculation method Methods 0.000 title claims abstract description 24
- 230000033001 locomotion Effects 0.000 title claims description 8
- 238000000034 method Methods 0.000 claims abstract description 59
- 238000005265 energy consumption Methods 0.000 claims abstract description 40
- 230000005540 biological transmission Effects 0.000 claims abstract description 39
- 238000005457 optimization Methods 0.000 claims abstract description 37
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 29
- 230000006870 function Effects 0.000 claims description 44
- 230000009471 action Effects 0.000 claims description 32
- 238000004891 communication Methods 0.000 claims description 25
- 230000002787 reinforcement Effects 0.000 claims description 20
- 238000012549 training Methods 0.000 claims description 11
- 230000007704 transition Effects 0.000 claims description 9
- 238000013468 resource allocation Methods 0.000 claims description 7
- 238000013461 design Methods 0.000 claims description 6
- 230000003993 interaction Effects 0.000 claims description 5
- 230000008054 signal transmission Effects 0.000 claims description 3
- XOFYZVNMUHMLCC-ZPOLXVRWSA-N prednisone Chemical compound O=C1C=C[C@]2(C)[C@H]3C(=O)C[C@](C)([C@@](CC4)(O)C(=O)CO)[C@@H]4[C@@H]3CCC2=C1 XOFYZVNMUHMLCC-ZPOLXVRWSA-N 0.000 claims 1
- 238000004088 simulation Methods 0.000 abstract description 10
- 238000007430 reference method Methods 0.000 abstract description 7
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000003111 delayed effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 230000002045 lasting effect Effects 0.000 description 2
- 230000000116 mitigating effect Effects 0.000 description 2
- 210000002569 neuron Anatomy 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 241001515997 Eristalis tenax Species 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000001210 attenuated total reflectance infrared spectroscopy Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000033228 biological regulation Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000010363 phase shift Effects 0.000 description 1
- 230000035755 proliferation Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/08—Load balancing or load distribution
- H04W28/09—Management thereof
- H04W28/0958—Management thereof based on metrics or performance parameters
- H04W28/0967—Quality of Service [QoS] parameters
- H04W28/0975—Quality of Service [QoS] parameters for reducing delays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B17/00—Monitoring; Testing
- H04B17/30—Monitoring; Testing of propagation channels
- H04B17/391—Modelling the propagation channel
- H04B17/3912—Simulation models, e.g. distribution of spectral power density or received signal strength indicator [RSSI] for a given geographic region
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B7/00—Radio transmission systems, i.e. using radiation field
- H04B7/14—Relay systems
- H04B7/15—Active relay systems
- H04B7/185—Space-based or airborne stations; Stations for satellite systems
- H04B7/18502—Airborne stations
- H04B7/18506—Communications with or from aircraft, i.e. aeronautical mobile service
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W12/00—Security arrangements; Authentication; Protecting privacy or anonymity
- H04W12/02—Protecting privacy or anonymity, e.g. protecting personally identifiable information [PII]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/0215—Traffic management, e.g. flow control or congestion control based on user or device properties, e.g. MTC-capable devices
- H04W28/0221—Traffic management, e.g. flow control or congestion control based on user or device properties, e.g. MTC-capable devices power availability or consumption
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/0226—Traffic management, e.g. flow control or congestion control based on location or mobility
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/08—Load balancing or load distribution
- H04W28/09—Management thereof
- H04W28/0917—Management thereof based on the energy state of entities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/08—Load balancing or load distribution
- H04W28/09—Management thereof
- H04W28/0925—Management thereof using policies
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Abstract
The invention discloses a safe mobile edge computing method for an intelligent reflection surface carried by an unmanned aerial vehicle. By modeling a joint relay unmanned aerial vehicle track, an IRS unmanned aerial vehicle track, IRS reflection element coefficients and unloading to an edge calculation facility data proportion problem, a D3QN-DDPG hybrid algorithm is used in an intelligent optimization method, the D3QN optimization sub relay unmanned aerial vehicle track and the IRS unmanned aerial vehicle track, and the SAC optimization IRS reflection element coefficients and unloading to the edge calculation facility data proportion. Simulation results show that compared with other reference methods, the unmanned aerial vehicle IRS auxiliary safety MEC network provided by the invention can obviously improve the safety transmission rate, the system delay and the energy consumption, and the intelligent mixing based on D3QN-DDPG can efficiently find an optimal scheme.
Description
Technical Field
The invention relates to a safe moving edge computing method for an intelligent reflection surface carried by an unmanned aerial vehicle, and belongs to the technical field of communication.
Background
Unmanned aerial vehicles (unmanned aerial vehicle, UAVs) are used as aerial communication platforms, and due to the high mobility and high cost performance of deployment, the coverage range, capacity and energy efficiency of a wireless communication network can be effectively improved, and the unmanned aerial vehicles are widely applied. Furthermore, with the prevalence of computationally intensive tasks in the everything interconnect (Internet of Everything, ioE), the computing and battery resources of many smart devices and user devices may be limited, resulting in lower quality of service (Quality of Service, qoS) for the user. Thus, supporting mobile edge computing (mobile edge computing, MEC) of unmanned aerial vehicles has received a lot of attention, as it brings the computing power and storage power of cloud computing closer to the user equipment. Compared to traditional fixed-location MEC infrastructure, enabling the drone's MEC may provide more flexible, more convenient, and faster computing services due to the rapid mobility of the drone.
With the increasing proliferation of data traffic and connected equipment, there is an increasing concern about the scarcity of available spectrum resources in UAV-supported MEC networks. Furthermore, the broadcast and open nature of wireless communications has led to internet of things devices being vulnerable to security threats during task offloading. In recent years, intelligent reflective surfaces (intelligent reflecting surfaces, IRS) have emerged as a promising technology for future communications, as they can increase the signal strength of legitimate user devices and decrease Eavesdropper (Eve) signal strength in a spectrally and energy efficient manner. IRS is a planar structure consisting of a large number of passive reflective elements that can be electronically reconfigured to adjust the phase shift of the impact transformation.
While traditional mathematical optimization methods based on convex optimization are efficient in many applications, the methods are difficult to adapt to dynamic scenarios where large numbers of IRS reflective array elements are adjusted in real time, and intelligent reflective surface assisted secure transmission systems require a more efficient optimization method such as deep reinforcement learning (deep reinforcement learning, DRL). Due to the powerful computing power of DRL, the best solution to the non-convex optimization problem can be quickly found, making it suitable for large systems with real-time performance requirements. The decision problem of offloading and system design in secure MEC systems with unmanned aerial vehicle eavesdroppers, which can intercept secure computing tasks from users to computing access points, is studied in the paper "Intelligent secure mobile edge computing for beyond 5G wireless networks (Physical Communication)" by s.lai, r.zhao et al. To solve the non-convex optimization problem, authors propose a DRL-based adaptive offloading strategy that jointly optimizes the wireless bandwidth and transmit power allocation among users. However, IRS has not been applied in the above work to further improve the system security transmission capability. In the L.Zhang, S.Lai et al paper "Deep reinforcement learning based IRS-assisted mobile edge computing under physical-layer security (Physical Communication)", the authors describe an IRS-assisted MEC network that considers physical layer security. To minimize system delay and energy consumption, an intelligent DRL algorithm is proposed to jointly optimize joint allocation of user offloading rate, wireless bandwidth and computing power. However, unmanned aerial vehicles are not used in the above work and unmanned aerial vehicles carry IRSs and unmanned aerial vehicle-assisted mobile computing networks require further research. In the paper "Task Offloading and Trajectory Control for UAV-Assisted Mobile Edge Computing Using Deep Reinforcement Learning (IEEE Access)" by l, zhang, z, zhang et al, the authors propose a DRL-based method to optimize control of unmanned aerial vehicle trajectories and scheduling of user offload task ratios, improving system stability while minimizing energy consumption and computational delay. However, the information security transmission problem is not considered in the above work.
Disclosure of Invention
The invention aims at overcoming the defects and shortcomings of the prior art, and provides a safe moving edge computing method for an intelligent reflection surface carried by an unmanned aerial vehicle, which is based on deep reinforcement learning and utilizes a mixed DRL algorithm based on a duel-bucket double-depth Q network (dueling double deep Q networks, D3 QN) -depth determination strategy gradient (deep deterministic policy gradient, DDPG) to minimize the overhead of an EMC system. The invention can efficiently solve the problem of non-convex optimization including coupling variables, and can achieve the trade-off between safe transmission and system overhead.
The technical scheme adopted for solving the technical problems is as follows: the method for calculating the safe moving edge of the intelligent reflection surface carried by the unmanned aerial vehicle comprises the following steps:
step 1: and establishing a safe mobile edge system model of the intelligent reflection surface carried by the unmanned aerial vehicle.
Step 1-1: and establishing a transmission model, and establishing a three-dimensional Cartesian coordinate system for all communication nodes. Setting the positions of the base station and the user. Setting stopping point set of relay unmanned aerial vehicleAnd corresponding stopping point position->. Setting a stopping point set for carrying an IRS unmanned aerial vehicle>Corresponding to the stopping point +.>. An eavesdropper movement range is set. The channel between the relay drone to the on-board IRS drone, eavesdropper, user and edge computing resources is modeled. The channel from the IRS drone to the user, eavesdropper and edge computing resources is modeled. Obtaining channel gain according to channel modeling;
Step 1-2: and establishing a delay model, and jointly considering the calculated delay and the flight delay.
Step 1-3: and (3) establishing an energy consumption model, and jointly considering the calculated energy consumption and the flight energy consumption.
Step 2: optimizing the relay unmanned aerial vehicle track, the IRS reflecting element coefficient and the unloading to edge calculation facility data proportion;
step 3: the target problem is converted into a Markov problem, the unmanned plane control center is set as an intelligent body, and a state space is designedAction space->Reporting functionAnd state transition probability->Establishing a reinforcement learning model;
step 4: and the mixed algorithm based on the D3QN-DDPG is utilized to jointly optimize the track of the relay unmanned aerial vehicle, the track of the IRS unmanned aerial vehicle, the coefficients of the IRS reflection element and the data proportion of the data to be unloaded to the edge computing facility, so that the system overhead is minimized. The method specifically comprises the following steps:
step 4-1: an intelligent resource allocation algorithm based on D3QN-DDPG is designed.
Step 4-2: the intelligent agent continuously interacts with the system to obtain the reinforcement learning training experience. And the intelligent agent decision relay unmanned aerial vehicle track, the IRS reflecting element coefficient and the data proportion of the offloaded edge calculation facility are input into the system to obtain and enter a new environment, and a reward value is obtained through a reward function. The intelligent agent stores the new and old environmental states, action selection and rewarding values generated by each step of iteration into an experience pool;
Step 4-3: and updating algorithm parameters. After the interaction between the intelligent agent and the environment meets the set iteration times, the intelligent agent learns by taking the minimized loss function as a training target through replaying the experience stored in the experience replay pool, updates the network parameters through the reverse gradient, and returns to the step 4-2 until the training is finished;
step 4-4: and obtaining a convergence algorithm, and storing each network parameter in a local file.
Step 5: and solving a sub-optimal solution of the relay unmanned aerial vehicle track, the IRS reflection element coefficient and the data proportion of the unloading edge calculation facility by using an intelligent resource allocation method based on the D3 QN-DDPG.
Further, the step 1 of the present invention includes: modeling a channel existing in a network, and obtaining channel gains. The safe transmission rate, the system delay and the system energy consumption are calculated, and are defined as follows:
and establishing a transmission model, and establishing a three-dimensional Cartesian coordinate system for all communication nodes. A three-dimensional cartesian coordinate system is established for all communication nodes. Setting an edge calculation positionAnd the position of the single antenna user->. Setting stopping point set of relay unmanned plane +.>And corresponding stopping point position->. Setting a stopping point set for carrying an IRS unmanned aerial vehicle >Corresponding to the stopping point +.>. The single antenna eavesdropper movement range is set. The channel between the relay drone to the on-board IRS drone, eavesdropper, user and edge computing resources is modeled. The channel from the IRS drone to the user, eavesdropper and edge computing resources is modeled. Channel gains are obtained from channel modeling. IRS is provided with->The reflection units are used for unloading tasks to the relay unmanned aerial vehicle by a user and unloading tasks to the edge calculation server by the relay unmanned aerial vehicle, and the phase of each reflection unit can be dynamically adjusted by the unmanned aerial vehicle control center. Definitions->Representing the user to hover at the stop point +.>Relay none of (2)Distance between man and machine>Representing the distance of the user from the eavesdropper, can be written separately as:
。
definition of the definitionRepresenting hovering at stop point->Is to hover at stopping point +.>Distance of IRS unmanned aerial vehicle, +.>Representing the distance of the relay drone hovering at the stopping point to the eavesdropper, +.>Representing hovering at stop point->The distance from the relay unmanned aerial vehicle to the edge calculation server can be written as:
。
defining hover at stop pointThe distance of the IRS unmanned aerial vehicle to the edge server is +.>Hovering at stop point->The IRS drone to eavesdropper is a distance +. >Can be expressed as:
。
the signal transmission is assumed to operate in millimeter wave communication, and the communication link is regarded as line-of-sight communication. Definition of the definitionRepresentative hover at->Channel gain of IRS drone to edge server,/>Representative hover at->The channel gain of the IRS drone to the eavesdropper can be written as:
wherein,representing the path loss at a reference distance of 1 meter. />And->Respectively represent from hovering at stopping point +>Is the approximate cosine of the angle of arrival of the IRS drone to the user and to the eavesdropper. Define hover at stop point->Is to hover at stopping point +.>The IRS drone has a channel gain to eavesdroppers of the order of magnitude, and an edge server channel gain of the order of magnitude, which can be written as:
wherein the method comprises the steps ofRepresenting the path loss factor. Define the user to hover at the stop point +.>Relay none of (2)Man-machine to channel gain is +.>The channel gain to eavesdropper is +.>Can be written separately as:
。
defining a two-stage track of a relay unmanned plane asIRS unmanned aerial vehicle end track is +.>. Define the two-phase IRS array element offset as +.>Wherein->For the first phase IRS array element offset, < >>For the second phase IRS array element offset, +.>And->For->Is true of (I)>In imaginary units. Thus, the first stage task off-load rate can be expressed as:
，
Wherein the method comprises the steps ofRepresenting the user bandwidth>Representing the user uplink,/>Representing noise power. The second stage partial task offload rate may be expressed as:
，
wherein the method comprises the steps ofRepresenting the bandwidth of the relay drone,/->Representing the relay drone uplink. The two-stage eavesdropping rate can be expressed as:
，
。
the two-stage security task offload rates may be expressed separately as:
，
。
the time of flight of the relay drone is expressed as:
，
wherein,representing the speed of flight of the drone. The time of flight of the IRS drone is expressed as:
。
the drone latency is expressed as:
。
the transmission delay is expressed as:
，
wherein the method comprises the steps ofSize of data representing user tasks, +.>Representing task offload to edge computing resource proportion, +.>. The drone computation delay may be expressed as:
，
wherein,represents the CPU cycle number of the relay unmanned plane, +.>Representing the relay drone computing power. The edge server computation delay can be expressed as:
，
wherein,representing the number of CPU cycles of the edge server, +.>Representing edge server computing power. Thus, the total delay of the EMC system can be expressed as:
。
the flight cost of the relay unmanned aerial vehicle and the IRS unmanned aerial vehicle is expressed as follows:
，
wherein the method comprises the steps ofRepresenting the unmanned aerial vehicle flight power. The hover cost can be expressed as:
，
Wherein the method comprises the steps ofRepresenting unmanned hover power. Total energy consumption is expressed as:
。
further, the step 2 of the present invention includes: and (5) jointly considering the system delay and the energy consumption, and establishing an objective function and constraint conditions. The optimization problem is expressed as follows:
，
wherein the method comprises the steps ofGiven a weight factor. The system delay and the energy consumption are minimized by jointly optimizing the relay unmanned aerial vehicle track, the unmanned aerial vehicle track for building the IRS, the IRS reflection element coefficient and the data proportion unloaded to the edge computing facility.
Optimization problem constraints: the first constraint is to offload the transmission rate constraint for the security task, expressed asWherein->And->Representing the minimum safe rate requirements of the first and second phases, respectively. The second constraint is the IRS reflection factor constraint, i.e. +.>. The third constraint is the constraint of task offloading rate, +.>. The fourth constraint is the position constraint of the relay unmanned aerial vehicle and the IRS unmanned aerial vehicle, namelyAnd->。
Further, the step 3 of the present invention includes: converting a target optimization problem into a Markov problem, extracting key elements in the optimization problem, establishing a reinforcement learning model comprising an intelligent body, an action space, a state space, a return value model and a state transition probability, and setting the unmanned aerial vehicle control The heart is the intelligent agent, and the relay unmanned aerial vehicle channel, the IRS unmanned aerial vehicle channel, the action selection and the rewarding value are the state space. At the position ofThe time of day, the action space can be expressed as:
，
wherein the method comprises the steps of、/>、/>And->Respectively indicate->The relay unmanned aerial vehicle track, the IRS reflection array element factors and the task unloading to the edge computer proportion at moment. The invention comprehensively considers the system delay, the energy consumption and the safe transmission efficiency in the design of the return value function.
At the position ofThe state space can be expressed as:
，
wherein,and->Representing the channel conditions of the relay drone and the IRS drone, respectively, < >>Action selected, ++>Representing the prize value. Setting a relay unmanned aerial vehicle track, an IRS unmanned aerial vehicle track, IRS reflection array element factors and a task unloading to an edge computer proportion to form a state space.
Thus, the return value function is expressed as:
，
wherein,given a weight factor. />Rewards that represent the system achieving a secure task offload may be expressed as:
。
the invention usesRepresenting status->Execution of action down->Enter state->State transition probabilities of (a).
The beneficial effects are that:
1. the invention contemplates IRS-assisted secure mobile edge communication networks, including dynamic relay drones and IRS drones. In view of the secure transmission of task offloading, system delay and energy consumption are minimized by jointly optimizing the relay drone trajectories, IRS reflective element coefficients, and the proportion of data offloaded to the edge computing facility.
2. The D3QN is used for solving the track selection problem of the relay unmanned aerial vehicle and the IRS unmanned aerial vehicle, the DDPG is used for learning the optimal IRS reflection coefficient and unloading the optimal IRS reflection coefficient to the data percentage of the computing facility, and experimental results show that the intelligent optimization method provided by the invention has good convergence and can efficiently treat the non-convex problem and the coupling variable.
3. According to the invention, the dynamic relay unmanned aerial vehicle and the unmanned aerial vehicle carrying the IRS are deployed to achieve the required safe transmission of task unloading, and the optimization problem of the joint relay unmanned aerial vehicle track, the IRS reflection element coefficient and the unloading to the edge calculation facility data proportion is modeled, so that the minimization of the system overhead can be realized under the condition of ensuring the safe transmission, and the intelligent D3QN-DDPG hybrid algorithm is used for solving the target problem. Simulation results show that compared with other reference methods, the intelligent algorithm provided by the method can achieve good balance between system consumption and confidentiality rate. In addition, the unmanned aerial vehicle assisted safety MEC network with IRS can remarkably reduce system consumption and improve confidentiality rate.
Drawings
FIG. 1 is a system architecture diagram of the present invention.
Fig. 2 is a convergence performance chart of the D3QN-DDPG mixing algorithm proposed by the present invention.
Fig. 3 is a diagram of a trajectory of a drone under the intelligent algorithm of the present invention.
Fig. 4 is a graph comparing the safe transmission rates of the intelligent optimization method and the reference method under different numbers of intelligent reflection plane array elements.
Fig. 5 is a graph comparing the system transmission delay of the intelligent optimization method and the reference method under different numbers of intelligent reflection plane array elements.
Detailed Description
The invention is described in further detail below with reference to the drawings.
Example 1
As shown in fig. 1, the invention provides a method for calculating a safe moving edge of an intelligent reflection surface carried by an unmanned aerial vehicle, which comprises the following steps:
step 1: and establishing a safe mobile edge system model of the intelligent reflection surface carried by the unmanned aerial vehicle.
Step 1-1: and establishing a transmission model, and establishing a three-dimensional Cartesian coordinate system for all communication nodes. A three-dimensional cartesian coordinate system is established for all communication nodes. Setting an edge calculation positionAnd the position of the single antenna user->. Setting stopping point set of relay unmanned plane +.>And corresponding stopping point position->. Setting a stopping point set for carrying an IRS unmanned aerial vehicle>Corresponding to the stopping point +.>. The single antenna eavesdropper movement range is set. The channel between the relay drone to the on-board IRS drone, eavesdropper, user and edge computing resources is modeled. The channel from the IRS drone to the user, eavesdropper and edge computing resources is modeled. Channel gains are obtained from channel modeling. IRS is provided with- >The reflection units are used for unloading tasks to the relay unmanned aerial vehicle by a user and unloading tasks to the edge calculation server by the relay unmanned aerial vehicle, and the phase of each reflection unit can be dynamically adjusted by the unmanned aerial vehicle control center. Definitions->Representing user hovering to a stopping point/>Distance of relay unmanned aerial vehicle, +.>Representing the distance of the user from the eavesdropper, can be written separately as:
。
definition of the definitionRepresenting hovering at stop point->Is to hover at stopping point +.>Distance of IRS unmanned aerial vehicle, +.>Representing the distance of the relay drone hovering at the stopping point to the eavesdropper, +.>Representing hovering at stop point->The distance from the relay unmanned aerial vehicle to the edge calculation server can be written as:
。
defining hover at stop pointThe distance of the IRS unmanned aerial vehicle to the edge server is +.>Hovering at stop point->The IRS drone to eavesdropper is a distance +.>Can be expressed as:
。
the signal transmission is assumed to operate in millimeter wave communication, and the communication link is regarded as line-of-sight communication. Definition of the definitionRepresentative hover at->Channel gain of IRS drone to edge server,/>Representative hover at->The channel gain of the IRS drone to the eavesdropper can be written as:
wherein,representing the path loss at a reference distance of 1 meter. / >And->Respectively represent from hovering at stopping point +>Is the approximate cosine of the angle of arrival of the IRS drone to the user and to the eavesdropper. Define hover at stop point->Is to hover at stopping point +.>The channel gain of the IRS unmanned aerial vehicle is +.>The channel gain to eavesdropper is +.>Channel gain to edge server is +.>Can be written as:
wherein the method comprises the steps ofRepresenting the path loss factor. Define the user to hover at the stop point +.>Is the relay drone to channel gain +.>The channel gain to eavesdropper is +.>Can be written separately as:
。
IRS-assisted task offloading can be divided into two phases. The first phase is to offload all tasks from the user to the relay drone, and the second phase is to offload some tasks from the relay drone to the compute server. Defining a two-stage track of a relay unmanned plane asIRS unmanned aerial vehicle end track is +.>. Define the two-phase IRS array element offset as +.>Wherein->For first-stage IRS array element offset，/>For the second phase IRS array element offset, +.>And->For->Is true of (I)>In imaginary units. Thus, the first stage task off-load rate can be expressed as:
，
wherein the method comprises the steps ofRepresenting the user bandwidth>Representing the user uplink,/>Representing noise power. The second stage partial task offload rate may be expressed as:
，
Wherein the method comprises the steps ofRepresenting the bandwidth of the relay drone,/->Representing the relay drone uplink. The two-stage eavesdropping rate can be expressed as:
，
。
thus, the two-stage security task offload rates may be expressed separately as:
，
。
step 1-2: and establishing a delay model, and jointly considering the calculated delay and the flight delay.
The time of flight of the relay drone may be expressed as:
，
wherein,representing the speed of flight of the drone. The time of flight of an IRS drone may be expressed as:
。
the drone latency may be expressed as:
。
the transmission delay can be expressed as:
，
wherein the method comprises the steps ofSize of data representing user tasks, +.>Representing task offload to edge computing resource proportion, +.>. The drone computation delay may be expressed as:
，
wherein,represents the CPU cycle number of the relay unmanned plane, +.>Representing the relay drone computing power. The edge server computation latency can be expressed as:
，
wherein,representing the number of CPU cycles of the edge server, +.>Representing edge server computing power. Thus, the total delay of the EMC system can be expressed as:
。
step 1-3: and (3) establishing an energy consumption model, and jointly considering the calculated energy consumption and the flight energy consumption.
Because the flight cost of the unmanned aerial vehicle is directly related to the flight time, the flight costs of the relay unmanned aerial vehicle and the IRS unmanned aerial vehicle are expressed as follows:
，
Wherein the method comprises the steps ofRepresenting the unmanned aerial vehicle flight power. The hover time includes first and second stage relay drones and IRS drone task offload times, and a waiting time of the drone that first reaches a stopping point. Thus, the hover cost can be expressed as:
，
wherein the method comprises the steps ofRepresenting unmanned hover power. The total energy consumption can be expressed as:
。
step 2: and establishing a resource allocation optimization problem according to the system model.
The invention provides a method for jointly optimizing system delay and energy expenditure under secure communication based on a relay unmanned aerial vehicle, an IRS unmanned aerial vehicle and an edge computing server, wherein edge computing resources are limited. The optimization problem can be expressed as follows:
，
wherein the method comprises the steps ofGiven a weight factor. The system delay and the energy consumption are minimized by jointly optimizing the relay unmanned aerial vehicle track, the unmanned aerial vehicle track for building the IRS, the IRS reflection element coefficient and the data proportion unloaded to the edge computing facility.
Optimization problem constraints: the first constraint is that the security task uninstalls the transmission rate constraint, which is safe for two phasesTransmission rate constraints, expressed asWherein->And->Representing the minimum safe rate requirements of the first and second phases, respectively. The second constraint is the IRS reflection factor constraint, i.e. +. >. The third constraint is the constraint of task offloading rate, +.>. The fourth constraint is the position constraint of the relay unmanned aerial vehicle and the IRS unmanned aerial vehicle, namelyAnd->。
Step 3: the target problem is converted into a Markov problem, and a reinforcement learning model is built.
Considering the IRS unmanned aerial vehicle assisted secure mobile edge computing network in a dynamic context, a model-free reinforcement learning model may be applied to solve the decision problem. The computational delay optimization problem is converted into a markov decision process. The unmanned aerial vehicle-IRS assisted secure mobile edge computing network is considered a dynamic environment and the unmanned aerial vehicle control center is considered an intelligent agent. The markov problem consists of a state space, an action space, a transition probability function, and a reward function. Markov key elements are modeled as follows. At the position ofThe state space can be expressed as:
，
wherein the method comprises the steps of、/>、/>And->Respectively indicate->The relay unmanned aerial vehicle track, the IRS reflection array element factors and the task unloading to the edge computer proportion at moment. The invention comprehensively considers the system delay, the energy consumption and the safe transmission efficiency in the design of the return value function. At->The time of day, the action space can be expressed as:
，
wherein,and->Representing the channel conditions of the relay drone and the IRS drone, respectively, < > >Action selected, ++>Representing the prize value. Setting a relay unmanned aerial vehicle track, an IRS unmanned aerial vehicle track, IRS reflection array element factors and a task unloading to an edge computer proportion to form a state space. Thus, the return value function is expressed as:
，
wherein,given a weight factor. />Rewards that represent the system achieving a secure task offload may be expressed as:
。
the invention usesRepresenting status->Execution of action down->Enter state->State transition probabilities of (a).
Step 4: and the mixed algorithm based on the D3QN-DDPG is utilized to jointly optimize the relay unmanned aerial vehicle track, the IRS reflection array element factors and the task unloading to the edge computer proportion, so that the system delay and the energy consumption are minimized.
Step 4-1: an intelligent optimization method based on D3QN-DDPG is designed.
Based on the reinforcement learning modeling in the step 3, the invention provides a hybrid intelligent optimization method based on D3 QN-DDPG. The accurate Q value estimating capability of the D3QN and the continuous action space processing capability of the DDPG are fully utilized, the relay unmanned aerial vehicle track, the IRS reflection array element factors and the task unloading to an edge computer are jointly optimized, and the system delay and the energy consumption are minimized.
D3QN is a hybrid reinforcement learning algorithm, which combines with Doubl e Q-Learning and lasting Q-Networks and Deep Q-Networks to improve the stability and convergence of the Learning process. In D3QN, D3QN divides Q network into slave statesStatus network of value obtained>And from the actions->Mobile network of value obtained in->The estimation of the state value is separated from the dominant estimation of the action. Thus, D3QN is used to solve the trajectory planning problem of the drone.
The target Q function of D3QN can be expressed as:
，
wherein the method comprises the steps ofParameters representing hidden layer->And->The value function parameter and the action function parameter are respectively represented.
DDPG has been demonstrated to effectively improve learning efficiency and algorithm stability using empirical playback and delayed update techniques, thereby mitigating the risk of local optimization. In addition, DDPG uses deep neural networks to approximate strategies and cost functions, which can effectively solve high-dimensional, nonlinear, and continuous space problems. The Q function can be expressed as:
，
wherein the method comprises the steps ofFor average action network parameters +.>For the target action network parameters, +.>For average critical network parameters +.>And judging the network parameters for the target. />
Step 4-2: the intelligent agent interacts with the system to obtain training experience.
The intelligent agent continuously interacts with the system to obtain the reinforcement learning training experience. And the intelligent agent decision relay unmanned aerial vehicle track, the IRS reflecting element coefficient and the data proportion of the offloaded edge calculation facility are input into the system to obtain and enter a new environment, and a reward value is obtained through a reward function. The intelligent agent stores the new and old environmental states, action selections and rewards generated by each step of iteration into an experience pool.
Step 4-3: and updating algorithm parameters. After the interaction between the intelligent agent and the environment meets the set iteration times, the intelligent agent learns by taking the minimized loss function as a training target through replaying the experience stored in the experience replay pool, and updates the network parameters through the reverse gradient.
And after the interaction between the intelligent agent and the environment meets the set times, entering an algorithm parameter updating stage. The intelligent agent is randomly taken out from the experience pool to be as followsIs->. D3QN and DDPG target minimum loss functions, network parameters are adjusted by reverse gradients.
In D3QN, the network loss function can be expressed as:
，
d3QN minimizes the loss function by inverting the gradient to the loss function, updating the parameters.
In DDPG, the target Q function is expressed as:
，
the loss function is minimized by calculating the inverse gradient of the loss function, i.e. updating the network parameters.
If the training times do not reach the set times, returning to the step 4-2 until the training is finished.
Step 4-5: and (5) the algorithm converges, and each network parameter is locally stored.
Step 5: and solving a suboptimal solution of the relay unmanned aerial vehicle track, the IRS reflection element coefficient and the data proportion unloaded to the edge computing facility by using a D3QN-DDPG hybrid intelligent optimization method.
The effects of the invention are further described in detail below in connection with simulation experiments, and specifically include:
1. emulating hardware conditions
The simulation experiment of the invention is carried out on a simulation platform of Python3.7 and TensorFlow 2.70. The CPU model of the computer is E5-2680 v4, the number of the CPU models is 6, and the GPU model is Injeida Geforce RTX 3060. The video memory is 12.6GB.
2. Simulation system parameters
The number of users is 1, the number of IRS unmanned aerial vehicles is 1, the number of relay unmanned aerial vehicles is 1, the number of base stations is 1, and the number of eavesdroppers is 1. The edge computing server is deployed at (0, 0), the initial position of the relay unmanned aerial vehicle is (0, 50), and the initial position of the IRS unmanned aerial vehicle is (10,10,50). The eavesdropper moving range is (-100, 0) to (100, 0), and the user, the eavesdropper, the relay unmanned aerial vehicle, the IRS unmanned aerial vehicle and the edge computing server are all single antennas. IRS passive reflection array element number. The stopping points of the relay unmanned aerial vehicle are (520,70,50), (480,30,50), (520,30,50), (480,70,50), (-10, -20, 50), (10, 20, 50), (-10, 30, 50), (20, -30, 50), and the stopping points of the IRS unmanned aerial vehicle are (500,30,50), (500,70,50), (470,40,50), (480,60,50), (0, -20, 50), (0,30,50), (-20,0,50), (60,0,50). User maximum transmit power +. >10dbm, maximum transmitting power of relay unmanned plane +.>For 10dbm, the flying speed of the relay unmanned aerial vehicle and the IRS unmanned aerial vehicle is 20 m/s, and the available bandwidth of the user and the relay unmanned aerial vehicle is +.>Is 1MHz, unmanned plane flying power +.>1000W, unmanned aerial vehicle hover power +.>1000W, path loss reference->-20dB, path loss factor->2. The D3QN consists of 2 target Q networks and 2 evaluation Q networks, each network having 3 hidden layers, each layer having 256 neurons, the learning rate of the evaluation Q network being set to 0.0025. The DDPG consists of two action networks, two criticizing networks, each network having 3 hidden layers, each layer having 256 neurons. The learning rate of the criticizing network is set to 0.002, and the learning rate of the action network is set to 0.002.
3. Emulation content
FIG. 2 shows the number of array elements in an intelligent reflection plane according to the intelligent resource allocation algorithm of the present inventionConvergence case in the case. As shown in the figure, the D3QN-DDPG hybrid intelligent optimization method provided by the invention can achieve convergence in a shorter iteration period, and the high efficiency of the method is proved. The prize value eventually exceeds 0, indicating that the method proposed by the invention can always achieve safe transmission while balancing system delay and energy consumption. In addition, the reward function coefficient set by the invention has proved to have better regulation effect on realizing balance among safe transmission, system delay and energy consumption.
Fig. 3 shows trajectory planning for a relay drone and an IRS drone. In the first stage, the relay drone and IRS drone fly to the user's equipment in the nearest place to the start point while taking into account flight delays and energy consumption, while avoiding eavesdroppers. In the second stage, in view of the existence of an eavesdropper around the edge computing facility, the relay drone selects a stopping point furthest from the eavesdropper's trajectory to improve security, while the IRS drone selects the fastest arrival point around the edge computing server to reduce energy consumption and delay.
Fig. 4 shows that the safety task unloading rate of the hybrid intelligent optimization method and the reference method provided by the invention changes along with the number of IRS passive reflecting elements. The "ground IRS" scheme fixes IRS at (50, 10, 0). The result proves that the unmanned aerial vehicle IRS auxiliary safety transmission scheme provided by the invention has the advantages of better safety performance compared with the ground IRS scheme. The invention uses the mobile IRS, has better flexibility, can improve the safety of the system, and confirms the high efficiency of the IRS unmanned aerial vehicle. Furthermore, simulation results indicate that MEC networks without IRS are difficult to implement for secure transmission.
Fig. 5 shows that the system delay of the hybrid intelligent optimization method and the reference method proposed by the present invention varies with the number of IRS passive reflecting elements. The first 'no IRS' benchmark scheme moves the IRS unmanned aerial vehicle out of the MEC network supported by the unmanned aerial vehicle, and optimizes the track of the relay unmanned aerial vehicle by using the method proposed by us. The second "random method" benchmark approach allows the drone to always select a random stopping point around the target in two phases. The results demonstrate the superiority of the intelligent scheme we propose, which effectively reduces transmission delay while ensuring secure transmission, and performance is superior to the baseline scheme. The IRS reflection array element can enhance the signal intensity of legal equipment, inhibit eavesdropper eavesdropping intensity and reduce system delay.
In summary, the simulation result and analysis are carried out, compared with other reference methods, the unmanned aerial vehicle IRS auxiliary safety MEC network provided by the invention can obviously improve the safety transmission rate, the system delay and the energy consumption, and the intelligent hybrid optimization method based on D3QN-DDPG can efficiently find the optimal relay unmanned aerial vehicle track, the IRS reflection array element and the task unloading to the edge server proportion scheme. In addition, simulation results also show that the IRS unmanned aerial vehicle is applied to the safe MEC network, so that the safe transmission rate can be effectively enhanced.
Example two
Aiming at the problem of a safety EMC network, the invention provides an IRS-carried safety mobile edge computing method for a UAV, which utilizes a D3 QN-DDPG-based hybrid DRL algorithm to minimize the overhead of an EMC system. Simulation results show that the invention can efficiently solve the problem of non-convex optimization comprising coupling variables, and can achieve the trade-off among safe transmission, system delay and energy consumption.
The invention considers an IRS unmanned aerial vehicle auxiliary safety EMC network model, which comprises the following specific steps:
(2a) Setting an edge calculation positionAnd the position of the single antenna user->. Setting stopping point set of relay unmanned plane +.>And corresponding stopping point position->. Setting a stopping point set for carrying an IRS unmanned aerial vehicle>Corresponding to the stopping point +.>. The single antenna eavesdropper movement range is set. The channel between the relay drone to the on-board IRS drone, eavesdropper, user and edge computing resources is modeled.
(2b) And establishing a safe transmission model.
，
。
(2c) And (3) establishing a system delay model, and jointly considering the calculated delay and the flight delay.
，
，
。
(2d) And (3) establishing an energy consumption model, and jointly considering the calculated energy consumption and the flight energy consumption.
，
，
。
The invention considers the relay unmanned aerial vehicle track, the unmanned aerial vehicle track for constructing the IRS, the IRS reflecting element coefficient and the data proportion E unloaded to the edge computing facility by joint optimization, and comprises the following specific steps:
(3a) The system aims at jointly optimizing the track of the relay unmanned aerial vehicle, the track of the unmanned aerial vehicle for building the IRS, the IRS reflection element coefficient and the data proportion E unloaded to the edge computing facility, and simultaneously meets the safety task unloading transmission rate constraint, the IRS reflection factor constraint, the task unloading rate constraint and the position constraint of the relay unmanned aerial vehicle and the IRS unmanned aerial vehicle. The targets are defined as follows:
。
optimization problem constraints: the first constraint is to offload the transmission rate constraint for the security task, expressed asWherein->And->Representing the minimum safe rate requirements of the first and second phases, respectively. The second constraint is the IRS reflection factor constraint, i.e. +.>. The third constraint is the task offloading rate constraint, +.>. The fourth constraint is the position constraint of the relay unmanned aerial vehicle and the IRS unmanned aerial vehicle, namelyAnd->。
And the mixed algorithm based on the D3QN-DDPG is utilized to jointly optimize the relay unmanned aerial vehicle track, the IRS reflection array element factors and the task unloading to the edge computer proportion, so that the system delay and the energy consumption are minimized. The specific design steps are as follows:
(4a) The target problem is converted into a Markov problem, and a reinforcement learning model is built.
At the position ofThe time of day, the action space can be expressed as:
，
Wherein the method comprises the steps of、/>、/>And->Respectively indicate->The relay unmanned aerial vehicle track, the IRS reflection array element factors and the task unloading to the edge computer proportion at moment. The invention comprehensively considers the system delay, the energy consumption and the safe transmission efficiency in the design of the return value function. />
At the position ofThe state space can be expressed as:
，
wherein,and->Representing the channel conditions of the relay drone and the IRS drone, respectively, < >>Action selected, ++>Representing the prize value. Setting a relay unmanned aerial vehicle track, an IRS unmanned aerial vehicle track, IRS reflection array element factors and a task unloading to an edge computer proportion to form a state space.
Thus, the return value function is expressed as:
，
wherein,given a weight factor. />Rewards that represent the system achieving a secure task offload may be expressed as:
。
the invention usesRepresenting status->Execution of action down->Enter state->State transition probabilities of (a).
(4b) Based on reinforcement learning modeling, the invention provides an intelligent resource allocation algorithm based on D3 QN-DDPG.
The invention fully utilizes the accurate Q value estimating capability of D3QN and the continuous motion space processing capability of DDPG, and jointly optimizes the relay unmanned aerial vehicle track, the IRS reflection array element factors and the task unloading to the edge computer, thereby minimizing the system delay and the energy consumption.
D3QN is a hybrid reinforcement Learning algorithm, which combines the ideas of Double Q-Learning and lasting Q-Networks with Deep Q-Networks to improve the stability and convergence of the Learning process. In D3QN, D3QN divides Q network into slave statesStatus network of value obtained>And from the actions->Obtain value action network->The estimation of the state value is separated from the dominant estimation of the action. Thus, D3QN is used to solve the trajectory planning problem of the drone.
The target Q function of D3QN can be expressed as:
，
wherein the method comprises the steps ofParameters representing hidden layer->And->The value function parameter and the action function parameter are respectively represented.
DDPG has been demonstrated to effectively improve learning efficiency and algorithm stability using empirical playback and delayed update techniques, thereby mitigating the risk of local optimization. In addition, DDPG uses deep neural networks to approximate strategies and cost functions, which can effectively solve high-dimensional, nonlinear, and continuous space problems. The Q function can be expressed as:
，
wherein the method comprises the steps ofFor average action network parameters +.>For the target action network parameters, +.>For average critical network parameters +.>And judging the network parameters for the target.
(4c) And after the interaction between the intelligent agent and the environment meets the set times, entering an algorithm parameter updating stage. The intelligent agent is randomly taken out from the experience pool to be as follows Is->. D3QN and DDPG target minimum loss functions, network parameters are adjusted by reverse gradients.
In D3QN, the network loss function can be expressed as:
，
d3QN minimizes the loss function by inverting the gradient to the loss function, updating the parameters.
In DDPG, the target Q function is expressed as:
，
the loss function is minimized by calculating the inverse gradient of the loss function, i.e. updating the network parameters.
(4d) And (5) the algorithm converges, and each network parameter is locally stored.
In the art, the above description is only a preferred example of the present invention and is not intended to limit the present invention. Any modification, substitution, improvement and color rendering made by those skilled in the art is intended to be within the scope of the present technology.
Claims (4)
1. The method for calculating the safe moving edge of the intelligent reflection surface carried by the unmanned aerial vehicle is characterized by comprising the following steps of:
step 1: establishing a safe mobile edge system model of the intelligent reflection surface carried by the unmanned aerial vehicle;
step 1-1: establishing a transmission model, establishing a three-dimensional Cartesian coordinate system for all communication nodes, setting positions of a base station and a user, and setting a stopping point set S of the relay unmanned aerial vehicle U ＝{1，...，s u ，...，s U Sum of corresponding stopping point positionsSetting a stopping point set S of an IRS unmanned aerial vehicle I ＝{1，...，s i ，…，s I Corresponding position of stop point ∈ }>Setting the moving range of an eavesdropper, modeling a channel between the relay unmanned aerial vehicle and the IRS unmanned aerial vehicle, the eavesdropper, the user and the edge computing resource, modeling a channel between the IRS unmanned aerial vehicle and the user, the eavesdropper and the edge computing resource, and obtaining channel gain according to the channel modeling;
step 1-2: establishing a delay model, and jointly considering the calculated delay and the flight delay;
step 1-3: establishing an energy consumption model, and jointly considering the calculated energy consumption and the flight energy consumption;
step 2: optimizing the relay unmanned aerial vehicle track, the IRS reflecting element coefficient and the unloading to edge calculation facility data proportion;
step 3: the target problem is converted into a Markov problem, the unmanned plane control center is set as an intelligent body, and a state space is designedAction space->Return function r= -T- α E E+α R R Sec And state transition probabilities τ(s) t+1 |s t ，a t ) Establishing a reinforcement learning model;
step 4: the method utilizes a hybrid algorithm based on D3QN-DDPG to jointly optimize the track of the relay unmanned aerial vehicle, the track of the IRS unmanned aerial vehicle, the coefficients of the IRS reflecting element and the data proportion of the data to be unloaded to the edge computing facility, and minimizes the system overhead, and specifically comprises the following steps:
Step 4-1: designing an intelligent resource allocation algorithm based on D3 QN-DDPG;
step 4-2: continuously interacting the intelligent agent with the system to obtain reinforcement learning training experience;
the intelligent agent decides a relay unmanned plane track, an IRS reflecting element coefficient and a data proportion of a facility calculated by unloading to the edge, inputs the data proportion into a system, obtains and enters a new environment, obtains a reward value through a reward function, and stores the new and old environment states, action selections and the reward value generated by each step of iteration into an experience pool;
step 4-3: updating algorithm parameters;
after the interaction between the intelligent agent and the environment meets the set iteration times, the intelligent agent learns by taking the minimized loss function as a training target through replaying the experience stored in the experience replay pool, updates the network parameters through the reverse gradient, and returns to the step 4-2 until the training is finished;
step 4-4: obtaining a convergence algorithm, and storing all network parameters in a local file;
step 5: and solving a sub-optimal solution of the relay unmanned aerial vehicle track, the IRS reflection element coefficient and the data proportion of the unloading edge calculation facility by using an intelligent resource allocation method based on the D3 QN-DDPG.
2. The method for calculating the safe moving edge of the intelligent reflection surface carried by the unmanned aerial vehicle according to claim 1, wherein the step 1 comprises: modeling a channel existing in a network, obtaining a channel gain, and calculating a safe transmission rate, a system delay and a system energy consumption, including:
Establishing a transmission model, establishing a three-dimensional Cartesian coordinate system for all communication nodes, and setting an edge calculation position [ X ] C ，Y C ，0]And the location of a single antenna user [ X ] K ，X K ，0]Setting a stopping point set S of the relay unmanned aerial vehicle U ＝{1，…，s u ，…，s U Sum of corresponding stopping point positionsSetting a stopping point set S of an IRS unmanned aerial vehicle I ＝{1，...，s i ，...，s I Corresponding position of stop point ∈ }>Setting a single antenna eavesdropper movement range, modeling a channel between an intermediate unmanned aerial vehicle and an IRS-carrying unmanned aerial vehicle, an eavesdropper, a user and edge computing resources, modeling a channel between the IRS unmanned aerial vehicle and the user, the eavesdropper and the edge computing resources, obtaining a channel gain according to the channel modeling, and the IRS is provided with>The reflecting units are used for unloading tasks to the relay unmanned aerial vehicle by a user and unloading the relay unmanned aerial vehicle to the edge computing server, and the phase of each reflecting unit is dynamically adjusted by the unmanned aerial vehicle control center to define +.>Representing the user hovering over stop point s u Distance d of relay unmanned aerial vehicle K，E Representing the distance of the user from the eavesdropper, can be written separately as:
definition of the definitionRepresenting hovering over stopping point s u Is hovered at stopping point s i Is a distance of the IRS unmanned aerial vehicle,representing the distance of the relay drone hovering at the stopping point to the eavesdropper, +. >Representing hovering over stopping point s u The distance from the relay unmanned aerial vehicle to the edge calculation server can be written as:
defining hover at stop point s i The distance from the IRS unmanned aerial vehicle to the edge server isHovering at stopping point s i The IRS drone to eavesdropper is a distance +.>Can be expressed as:
the signal transmission is assumed to operate in millimeter wave communication, the communication link is regarded as line of sight communication, definitionRepresenting hovering over s i Channel gain of IRS drone to edge server,/>Representing hovering over s i The channel gain of the IRS drone to the eavesdropper can be written as:
where a represents the path loss at a reference distance of 1 meter,and->Respectively representFrom hovering over stopping point s i The approximate cosine of the angle of arrival of the IRS unmanned aerial vehicle to the user and to the eavesdropper defines hovering at the stopping point s u Is hovered at stopping point s i The IRS drone has a channel gain to eavesdroppers of the order of magnitude, and an edge server of the order of magnitude, which can be written as:
where β represents a path loss factor defining the user to hover at stopping point s u Is the gain of the relay unmanned aerial vehicle to the channelChannel gain to eavesdropper of h K，E Can be written as:
Defining a two-stage track of a relay unmanned plane asIRS unmanned aerial vehicle end track isDefine two-phase IRS array element offset as Θ= { Θ 1 ，Θ 2 }, where Θ 1 For the first phase IRS array element offset, Θ 2 For the second phase IRS array element offset, θ 1，n E [0,2 pi) and θ 2，n E [0,2 pi) pair ]>For example, j is an imaginary unit, so the first stage task offload rate can be expressed as:
wherein B is K Representing user bandwidth, P K Representing the user uplink, sigma 2 Representing noise power, the second stage partial task offload rate can be expressed as:
wherein B is U Representing the bandwidth of the relay unmanned plane, P U On behalf of the relay drone uplink, the two-stage eavesdropping rates can be expressed separately as:
the two-stage security task offload rates can be expressed as:
the time of flight of the relay drone is expressed as:
wherein, V represents unmanned aerial vehicle's flight speed, and IRS unmanned aerial vehicle's flight time represents as:
the drone latency is expressed as:
the transmission delay is expressed as:
wherein D represents the size of user task data, delta represents the task offloading to edge computing resource ratio, delta epsilon [0,1], and unmanned aerial vehicle computing delay can be expressed as:
wherein f U Representing the CPU cycle number of the relay unmanned aerial vehicle, F U On behalf of the relay drone computing capability, the edge server computing delay can be expressed as:
Wherein f C Representing the CPU cycle number of the edge server, F C Representing edge server computing power, the total delay of an EMC system can therefore be expressed as:
the flight cost of the relay unmanned aerial vehicle and the IRS unmanned aerial vehicle is expressed as follows:
E Fly ＝P Fly (T Fly，U +T Fly，I )，
wherein P is Fly Representing unmanned aerial vehicle flight power, hover costs can be expressed as:
E Hov ＝P Hov (2*T Tran，U +T Comp，U +T Comp，C +T Wait )，
wherein P is Hov Representing unmanned hover power, total energy consumption is expressed as:
E＝E Fly +E Hov 。
3. the method for calculating the safe moving edge of the intelligent reflection surface carried by the unmanned aerial vehicle according to claim 1, wherein the step 2 comprises: the system delay and the energy consumption are jointly considered, an objective function and constraint conditions are established, and the optimization problem is expressed as follows:
wherein alpha is E For giving weight factors, minimizing system delay and energy consumption by jointly optimizing a relay unmanned aerial vehicle track, an unmanned aerial vehicle track for building an IRS, IRS reflecting element coefficients and data proportion unloaded to an edge computing facility;
optimization problem constraints: the first constraint is to offload the transmission rate constraint for the security task, expressed asWherein R is th，1 And R is th，2 Representing the minimum safe rate requirements of the first and second phases, respectively, the second constraint being the IRS reflection factor constraint, i.e. +. >The third constraint is the constraint of task offloading rate, delta E [0,1 ]]The fourth constraint is the position constraint of the relay unmanned aerial vehicle and the IRS unmanned aerial vehicle, namelyAnd->
4. The method for calculating the safe moving edge of the intelligent reflection surface carried by the unmanned aerial vehicle according to claim 1, wherein the step 3 comprises: converting the target optimization problem into a Markov problem, extracting key elements in the optimization problem, establishing a reinforcement learning model, wherein the reinforcement learning model comprises an intelligent body, an action space, a state space, a return value model and a state transition probability, setting an unmanned plane control center as the intelligent body, and setting a relay unmanned plane channel, an IRS unmanned plane channel, action selection and a rewarding value as the state space, wherein the action space can be expressed as:
wherein the method comprises the steps ofΘ i And delta i Respectively representing the relay unmanned aerial vehicle track, the IRS reflection array element factors and the task unloading to an edge computer proportion at the moment i, and comprehensively considering system delay, energy consumption and safe transmission efficiency in the return value function design;
at time i, the state space can be expressed as:
wherein,and->Respectively representing channel conditions of the relay unmanned aerial vehicle and the IRS unmanned aerial vehicle, a i-1 Selected action, r i Representing a reward value, setting a relay unmanned aerial vehicle track, an IRS unmanned aerial vehicle track, IRS reflection array element factors and a task unloading to an edge computer to form a state space;
Thus, the return value function is expressed as:
r＝-T-α E E+α R R Sec ，
wherein alpha is R To give weight factors, R Sec Rewards that represent the system achieving a secure task offload can be expressed as:
R Sec ＝(R Sec，1 -R th，1 )+(R Sec，2 -R th，2 )，
the method uses τ(s) i+1 |s i ，a i ) Representative state s i Lower execution action a i Enter state s i+1 State transition probabilities of (a).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310904821.7A CN116684925B (en) | 2023-07-24 | 2023-07-24 | Unmanned aerial vehicle-mounted intelligent reflecting surface safe movement edge calculation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310904821.7A CN116684925B (en) | 2023-07-24 | 2023-07-24 | Unmanned aerial vehicle-mounted intelligent reflecting surface safe movement edge calculation method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116684925A CN116684925A (en) | 2023-09-01 |
| CN116684925B true CN116684925B (en) | 2023-11-14 |
Family
ID=87789409
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310904821.7A Active CN116684925B (en) | 2023-07-24 | 2023-07-24 | Unmanned aerial vehicle-mounted intelligent reflecting surface safe movement edge calculation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116684925B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117519995B (en) * | 2024-01-05 | 2024-03-22 | 中国人民解放军陆军指挥学院 | Star network mobile edge computing resource allocation method |
| CN117715117A (en) * | 2024-02-06 | 2024-03-15 | 南京信息工程大学 | Task unloading method for multi-UAV-RIS auxiliary mobile edge calculation |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114666803A (en) * | 2022-03-01 | 2022-06-24 | 北京邮电大学 | Deployment and control method and system of mobile edge computing system |
| CN115664486A (en) * | 2022-12-29 | 2023-01-31 | 南京邮电大学 | Energy efficiency optimization method for wireless energy supply in RIS (RIS) assisted UAV (unmanned aerial vehicle) edge computing system |
| WO2023048510A1 (en) * | 2021-09-23 | 2023-03-30 | Samsung Electronics Co., Ltd. | Method and wireless network for managing aerial subscriptions information of uav |
| CN116017303A (en) * | 2022-12-22 | 2023-04-25 | 昆明理工大学 | Combined resource allocation and track design method for safety IRS auxiliary UA V-MEC system |
| CN116390125A (en) * | 2023-04-03 | 2023-07-04 | 南京邮电大学 | Industrial Internet of things cloud edge cooperative unloading and resource allocation method based on DDPG-D3QN |
| CN116418808A (en) * | 2021-12-29 | 2023-07-11 | 中国电信股份有限公司 | Combined computing unloading and resource allocation method and device for MEC |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023059036A1 (en) * | 2021-10-08 | 2023-04-13 | Samsung Electronics Co., Ltd. | Communication method and device in wireless communication system supporting unmanned aerial system service |
| KR20230057219A (en) * | 2021-10-21 | 2023-04-28 | 삼성전자주식회사 | Method and apparatus for RIS automatic setup |
-
2023
- 2023-07-24 CN CN202310904821.7A patent/CN116684925B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023048510A1 (en) * | 2021-09-23 | 2023-03-30 | Samsung Electronics Co., Ltd. | Method and wireless network for managing aerial subscriptions information of uav |
| CN116418808A (en) * | 2021-12-29 | 2023-07-11 | 中国电信股份有限公司 | Combined computing unloading and resource allocation method and device for MEC |
| CN114666803A (en) * | 2022-03-01 | 2022-06-24 | 北京邮电大学 | Deployment and control method and system of mobile edge computing system |
| CN116017303A (en) * | 2022-12-22 | 2023-04-25 | 昆明理工大学 | Combined resource allocation and track design method for safety IRS auxiliary UA V-MEC system |
| CN115664486A (en) * | 2022-12-29 | 2023-01-31 | 南京邮电大学 | Energy efficiency optimization method for wireless energy supply in RIS (RIS) assisted UAV (unmanned aerial vehicle) edge computing system |
| CN116390125A (en) * | 2023-04-03 | 2023-07-04 | 南京邮电大学 | Industrial Internet of things cloud edge cooperative unloading and resource allocation method based on DDPG-D3QN |
Non-Patent Citations (5)
| Title |
|---|
| Dynamic Task Offloading in MEC-Enabled IoT Networks: A Hybrid DDPG-D3QN Approach;H. Hu等;《2021 IEEE Global Communications Conference (GLOBECOM)》;全文 * |
| Intelligent Resource Allocation for IRS-Enhanced OFDM Communication Systems: A Hybrid Deep Reinforcement Learning Approach;W. Wu等;《IEEE Transactions on Wireless Communications》;全文 * |
| Intelligent resource allocation for transmission security on IRS-assisted spectrum sharing systems with OFDM;Lingyi Wang等;《Physical Communication》;全文 * |
| 无人机辅助边缘计算的能量效率最大化算法设计;吴启晖;吴伟;;通信学报(第10期);全文 * |
| 智能无线通信技术研究概况;梁应敞;谭俊杰;Dusit Niyato;;通信学报(第07期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116684925A (en) | 2023-09-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116684925B (en) | Unmanned aerial vehicle-mounted intelligent reflecting surface safe movement edge calculation method | |
| Wang et al. | Joint trajectory and passive beamforming design for intelligent reflecting surface-aided UAV communications: A deep reinforcement learning approach | |
| CN111800828B (en) | Mobile edge computing resource allocation method for ultra-dense network | |
| CN113162679A (en) | DDPG algorithm-based IRS (inter-Range instrumentation System) auxiliary unmanned aerial vehicle communication joint optimization method | |
| Gao et al. | Game combined multi-agent reinforcement learning approach for UAV assisted offloading | |
| CN113472419B (en) | Safe transmission method and system based on space-based reconfigurable intelligent surface | |
| CN114422363B (en) | Capacity optimization method and device for unmanned aerial vehicle-mounted RIS auxiliary communication system | |
| CN113115344B (en) | Unmanned aerial vehicle base station communication resource allocation strategy prediction method based on noise optimization | |
| CN113316169B (en) | UAV auxiliary communication energy efficiency optimization method and device for smart port | |
| CN113377533A (en) | Dynamic computation unloading and server deployment method in unmanned aerial vehicle assisted mobile edge computation | |
| CN116436512A (en) | Multi-objective optimization method, system and equipment for RIS auxiliary communication | |
| WO2022242468A1 (en) | Task offloading method and apparatus, scheduling optimization method and apparatus, electronic device, and storage medium | |
| CN113282352A (en) | Energy-saving unloading method based on multi-unmanned aerial vehicle cooperative auxiliary edge calculation | |
| CN115827108A (en) | Unmanned aerial vehicle edge calculation unloading method based on multi-target depth reinforcement learning | |
| CN113254188A (en) | Scheduling optimization method and device, electronic equipment and storage medium | |
| CN116208619A (en) | Intelligent reflection surface-assisted Internet of vehicles safety calculation unloading method, system, equipment and medium | |
| Park et al. | Joint trajectory and resource optimization of MEC-assisted UAVs in sub-THz networks: A resources-based multi-agent proximal policy optimization DRL with attention mechanism | |
| CN117221951A (en) | Task unloading method based on deep reinforcement learning in vehicle-mounted edge environment | |
| CN116963034A (en) | Emergency scene-oriented air-ground network distributed resource scheduling method | |
| CN116684851A (en) | MAPPO-based multi-RIS auxiliary Internet of vehicles throughput improving method | |
| CN116578354A (en) | Method and device for unloading edge calculation tasks of electric power inspection unmanned aerial vehicle | |
| CN115734238A (en) | Intelligent reflecting surface assisted communication network coverage optimization method | |
| CN115915069A (en) | Unmanned aerial vehicle RIS-carrying auxiliary vehicle network communication method and system | |
| CN114928826A (en) | Two-stage optimization method, controller and decision method for software-defined vehicle-mounted task unloading and resource allocation | |
| CN117376985B (en) | Energy efficiency optimization method for multi-unmanned aerial vehicle auxiliary MEC task unloading under rice channel |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |