CN116600344A - Multi-layer MEC resource unloading method with power cost difference - Google Patents

Multi-layer MEC resource unloading method with power cost difference Download PDF

Info

Publication number
CN116600344A
CN116600344A CN202310443772.1A CN202310443772A CN116600344A CN 116600344 A CN116600344 A CN 116600344A CN 202310443772 A CN202310443772 A CN 202310443772A CN 116600344 A CN116600344 A CN 116600344A
Authority
CN
China
Prior art keywords
satellite
user
mec
transmission
calculation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310443772.1A
Other languages
Chinese (zh)
Inventor
王波
郭城瑞
黄冬艳
王旭
谢心颖
吕佳齐
卢泽林
方宇航
谢杰成
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guilin University of Electronic Technology
Original Assignee
Guilin University of Electronic Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guilin University of Electronic Technology filed Critical Guilin University of Electronic Technology
Priority to CN202310443772.1A priority Critical patent/CN116600344A/en
Publication of CN116600344A publication Critical patent/CN116600344A/en
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W28/00—Network traffic management; Network resource management
    • H04W28/02—Traffic management, e.g. flow control or congestion control
    • H04W28/08—Load balancing or load distribution
    • H04W28/09—Management thereof
    • H04W28/0958—Management thereof based on metrics or performance parameters
    • H04W28/0967—Quality of Service [QoS] parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W28/00—Network traffic management; Network resource management
    • H04W28/02—Traffic management, e.g. flow control or congestion control
    • H04W28/08—Load balancing or load distribution
    • H04W28/09—Management thereof
    • H04W28/0958—Management thereof based on metrics or performance parameters
    • H04W28/0967—Quality of Service [QoS] parameters
    • H04W28/0975—Quality of Service [QoS] parameters for reducing delays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W84/00—Network topologies
    • H04W84/02—Hierarchically pre-organised networks, e.g. paging networks, cellular networks, WLAN [Wireless Local Area Network] or WLL [Wireless Local Loop]
    • H04W84/04—Large scale networks; Deep hierarchical networks
    • H04W84/06—Airborne or Satellite Networks
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Radio Relay Systems (AREA)

Abstract

The application discloses a multi-layer MEC resource unloading method with electric power cost difference, which comprises the steps of firstly establishing a network model of the multi-layer MEC resource with the electric power cost difference; then establishing a communication model and a calculation model under different resource levels; allocating channel resources by using a sub-channel user recombination algorithm based on NOMA; and finally, converting the optimization problem into an equivalent reinforcement Learning problem by using a Q-Learning-based calculation unloading and resource allocation algorithm, and converging a Q table through training of the intelligent agent so as to guide the unloading decision of the base station intelligent agent. The problem is expressed as a mixed integer programming problem by combining unloading decision and resource allocation with the weighted sum of the time cost and unloading cost of all users as an optimization target, and a solution scheme for optimizing transmission and unloading based on NOMA and Q-Learning is provided. Simulation results show that the multi-layer MEC architecture is superior to the traditional single-layer MEC architecture, and meanwhile, the algorithm is superior to other basic algorithms in solving.

Description

Multi-layer MEC resource unloading method with power cost difference
Technical Field
The application relates to the field of Internet of things and resource allocation, in particular to a multi-layer MEC resource unloading method with power cost difference.
Background
In recent years, with the explosive growth of internet of things services, many emerging business types, such as 3D games, telemedicine, AI training, and the like, are being induced. Since a User Equipment (UE) is limited by computing power and battery capacity, it is difficult to perform a computing task requiring a large amount of computing power. To solve this problem, multiple access edge computing (Multi-Access Edge Computing, MEC) has been proposed to expand the computing power of the user equipment. By offloading the computing tasks to the MEC server with a higher computing power, the computing latency and energy consumption of the user device can be significantly reduced. However, as the user access increases, existing local MEC facilities have not been able to carry such large-scale computing demands. Therefore, actively expanding more MEC resources to provide services for users becomes a problem that we need to solve.
Low Earth Orbit (LEO) satellites maintain their own power demand by solar and chemical energy generation, with Low power costs.
With the access of LEO-MEC and W-CS, the partial calculation power demands of the eastern users are migrated, so that the calculation power burden of the local MEC can be effectively relieved. This presents a significant challenge to problem modeling because of the large variance in response latency and power costs of computing resources between different levels.
The Architecture of MECs under satellite-ground fusion was studied in the literature "Xie R, tang Q, wang Q, et al, satellite-terrestrial integrated edge computing networks: architecture, changes, and open issues [ J ]. Ieee networks, 2020,34 (3): 224-231", which placed satellite computing resources into multi-tier heterogeneous edge computing clusters and presented promising technical challenges including collaborative computing offloading, multi-node task scheduling, and the like. The literature "Tang Z, zhou H, ma T, et al, learning LEO Assisted Cloud-Edge Collaboration for Energy Efficient Computation Offloading [ C ]//2021IEEE Global Communications Conference.IEEE,2021:1-6 ], and" Tang N, lyu F, quan W, et al, space/area-assisted computing offloading for IoT applications: learning-based approach [ J ]. IEEE Journal on Selected Areas in Communications,2019,37 (5): 1117-1129", respectively, show a MEC integrated network containing LEO satellites, offloading computing tasks to cloud servers by unmanned aerial vehicles or LEO satellites relay, but without considering on-board processing power, resulting in wasted space resources, while offloading to cloud servers in a relayed manner would introduce additional propagation delay, affecting the quality of experience of users. The literature, "Song Z, hao Y, liu Y, et al energy-efficient multiaccess edge computing for terrestrial-satellite Internet of Things [ J ]. IEEE Internet of Things Journal,2021,8 (18): 14202-14218" offloads local partial computing tasks to satellite MEC server processing via a Terrestrial Satellite Terminal (TST), and proposes an energy-efficient computing offloading and resource allocation algorithm that minimizes the weighted energy sum of local devices without violating task-tolerant delays. The literature, "Tang Q, fei Z, li B, et al, computation offloading in LEO satellite networks with hybrid cloud and edge computing [ J ]. IEEE Internet of Things Journal,2021,8 (11): 9164-9176" shows a MEC network of hybrid cloud and LEO satellites with a three-layer computing architecture, and proposes a distributed algorithm that approximates the optimal solution using a multiplier-alternating direction method to minimize the total energy consumption of ground user computing offloading. It can be seen that the students can offload part of tasks to the satellite or the ground cloud through some optimization strategies so as to reduce the energy consumption of the local equipment, and the energy consumption of the calculation processing of the satellite and the ground cloud server is not considered.
The literature, "Wu J, jia M, zhang L, et al, dnns Based Computation Offloading for LEO Satellite Edge Computing [ J ]. Electronics,2022,11 (24): 4108" considers the computational power consumption of LEO-MEC in the system, proposes a deep learning-based LEO satellite edge computation network offloading algorithm that optimizes the weighted sum of system power consumption and time delay. The literature 'Cao X, yang B, shen Y, et al edge-Assisted Multi-Layer Offloading Optimization of LEO Satellite-Terrestrial Integrated Networks [ J ]. IEEE Journal on Selected Areas in Communications, 2022' researches a Multi-layer Multi-access MEC system, expands the MEC concept to the edge of an LEO satellite, formulates the problem of joint optimization of computing and communication resources, and verifies the idea of low computing delay and low energy consumption of the scheme by adopting a classical alternating optimization method to decompose and solve the original problem.
The literature "Hossain M D, sulta T, hossain M A, et al, fuzzy decision-based efficient task offloading management scheme in multi-tier MEC-enabled networks [ J ]. Sensors,2021,21 (4): 1484" proposes a cloud-MEC collaborative task offloading scheme based on fuzzy decision, and a user selects an optimal task offloading target node according to server capacity, delay sensitivity and network conditions, so that the successful execution rate of task offloading is remarkably improved, and the completion time of tasks is reduced. The literature "Tong M, wang X, li S, et al, joint Offloading Decision and Resource Allocation in Mobile Edge Computing-Enabled Satellite-Terrestrial Network [ J ]. Symmetry,2022,14 (3): 564.) proposes a joint offloading decision and resource allocation scheme for an satellite-to-ground MEC network that minimizes the completion delay of all terminal tasks, ensuring the latency requirements of most users.
Disclosure of Invention
In order to fully utilize the low-cost power advantages of LEO satellites and western regions, more calculation unloading opportunities are provided for user equipment, and meanwhile, the unloading selection problem caused by multi-level calculation resource unloading difference is solved.
The technical scheme for realizing the aim of the application is as follows:
a multi-tier MEC resource offloading method with power cost differences, comprising the steps of:
(1) Establishing a network model of a multi-layer MEC resource with power cost difference;
(2) Establishing a communication model and a calculation model under different resource levels;
(3) Allocating channel resources by using a sub-channel user recombination algorithm based on NOMA;
(4) And converting the optimization problem into an equivalent reinforcement Learning problem by using a Q-Learning-based calculation unloading and resource allocation algorithm, and converging a Q table by training the intelligent agent so as to guide the unloading decision of the intelligent agent of the base station.
Further:
the establishing a network model of the multi-tier MEC resource with power cost differential of step (1) includes:
1) The user equipment downloads a calculation task to a target server through a Base Station for processing, wherein the Base Station (BS) is used as an access point of the user equipment, two devices of a traditional cellular Base Station and a ground satellite terminal are integrated, and a C wave band and a Ka wave band are respectively adopted for communication with the ground equipment and a low-orbit satellite;
2) The coverage area of the base station is provided with M users, and each user has a calculation task d i (i.epsilon.M) need to unload processing and computing task is not partitionable, user's task data set is noted asThe three layers of computing resources provide computing services for users;
3) The user equipment uploads the calculation task to the base station by adopting a NOMA technology, and the base station controller performs unified scheduling on the user task according to the task attribute of the user and the real-time state of each level of resources; recording a=1 when the computing task is offloaded at the local MEC server, whereas a=0; recording b=1 when the computing task is offloaded at the satellite MEC server, whereas b=0; record c=1, reverse when computing tasks are offloaded at the western cloud serverC=0. Marking offloading decisions as
The three layers of computing resources for providing computing services for users are respectively:
(1) a local MEC server at the base station side;
(2) a satellite MEC server mounted on the low orbit satellite;
(3) and a western cloud server built in a western region.
The establishing a communication model under different resource levels in the step (2) comprises the following steps:
1) With NOMA transmission scheme, it is assumed that each sub-channel can be occupied by two users at the same time, and the users in the sub-channels are orthogonal to each other, so that M users are divided into N pairs, i.e. N sub-channels, denoted as
2) At a receiving end, decoding is carried out according to a principle of NOMA uplink channel gain descending decoding; the second user is considered as interference when decoding the first user, so the transmission rate of the first decoded user is:
the second user is decoded while interfering with the user signal at the following transmission rate:
wherein B is the subcarrier bandwidth; p represents the transmit power of the user; n is n 0 Representing the noise power.
3) For user equipment in the coverage area of the base station, the base station controller is uniformly adopted to carry out batch processing on user tasks, and the transmission rate of the ground satellite communication is set as a constant R because the distance between the base station and the satellite is a fixed value GS The transmission rate of the satellite-to-ground communication is constant R SG 。
The establishing of the calculation model under different resource levels in the step (2) comprises the following steps:
1) The total cost of user task offloading is divided into time cost and offloading cost, and its specific gravity to the total cost is represented by weight factors ω and v.
Wherein the time cost consists of transmission, propagation and processing delays, and is related to the distance of the user to the target server; the unloading cost consists of transmission cost and calculation cost, and is related to energy consumption and local unit electricity price;
2) The local MEC server allocates a computational power f to UEi i L The number of CPU Cycles required per bit of data is β (Cycles/bit), so the local offload time cost is:
the first term in the formula is the transmission delay from the UE to the BS; the second term is the processing time delay of the calculation task;
the transmission power of the UEi is P i The transmission energy consumption of the UE to the BS is:
dynamic power consumption P and V by dynamic voltage and frequency scaling 2 f is proportional, and the calculation frequency f of the CPU chip and the power supply voltage V are approximately in linear relation under the limit of low voltage, namely V=af; modeling power consumption of CPU as p=epsilonf 3 Where f is the calculation frequency of the CPU, epsilon is the coefficient of the chip architecture, and the calculation energy consumption of the user i at the local MEC server l is:
E i,l C ＝ε(f i L ) 2 d i β (6)
local unit electricity price p L The cost function available for local offloading is:
U i L (d,f)＝ωT i L +υ(E i T p L +E i,l C p L ) (7)
3) The distance from the BS to the access satellite is H, and the computing power allocated to the UEi by the satellite MEC server is f i S Therefore, the time cost of satellite unloading is:
the first term in the formula is the transmission delay from the UE to the BS; the second and third items are uplink transmission and propagation delay of the ground satellite communication, and the fourth item is processing delay of a calculation task;
the transmission power of the BS is P GS The transmission energy consumption of BS to satellite s is:
the calculation energy consumption of the UEi at the satellite MEC server s is:
E i,s C ＝ε(f i S ) 2 d i β (10)
the unit electricity price of the satellite server is p S The cost function of the available satellite offloads is:
U i S (d,f)＝ωT i S +υ(E i T p L +E gs T p L +E i,s C p S ) (11)
4) The computing power allocated to UEi by the western cloud server is f i W The western unloading time costs are therefore:
the first term in the formula is the transmission delay from the UE to the BS; the second and third items are transmission delays of satellite-to-satellite communication, the fourth item is transmission delay of BS (base station) to the western cloud server through satellite relay, and the fifth item is processing delay of calculation tasks in the western cloud server.
The transmission power of the satellite is P SG The transmission energy consumption from the satellite s to the western cloud server w is:
the calculation energy consumption of the UEi at the western cloud server w is as follows:
E i,w C ＝ε(f i W ) 2 d i β (13)
the unit electricity price of the western cloud server is p W The cost function available for western offloading is:
U i W (d,f)＝ωT i W +υ(E i T p L +E gs T p L +E sg T p S +E i,w C p W ) (14)。
step (3) of allocating channel resources by using a sub-channel user reorganization algorithm based on NOMA, including:
1) By usingRepresenting a set of users;Representing a set of subchannels, wherein each subchannel comprises 2 users;Representing the sum of the rates of the users in each sub-channel, wherein +.>
2) Selecting one user from any two sub-channels to perform position exchange to generate a new sub-channel user combination n' i And n' j (i≠j)And calculate q' i And q' j User reorganization is successful if the following inequality is satisfied:
the Q-Learning-based computing unloading and resource allocation algorithm in the step (4) converts the optimization problem into an equivalent reinforcement Learning problem for solving, and the method comprises the following steps:
1) Converting the optimization problem into an equivalent reinforcement learning problem, namely:
state space: the state space is the set of available resources of the target server, denoted s= { F L ，F S ，F W }；
Action space: the action space is determined by unloading the decision vectorAnd resource allocation vectorTwo-part composition, denoted as a= { af 1 …af M ,bf 1 …bf M ,cf 1 …cf M }；
Rewarding: state s k Lower execution action a k Is defined as R(s) k ,a k )＝-U(s k ,a k )。
2) The base station controller selects the action with the maximum known action value according to the probability of epsilon (0 < epsilon < 1); the probability of 1-epsilon randomly selects an action to execute:
3) In the process of utilizing or exploring, the Q table is required to be updated through a formula (17) every time an action is executed; through repeated training, the intelligent agent can be guided to select the optimal strategy until the Q table converges:
wherein R represents a state s k Lower execution action a k The obtained instant rewards;representing the maximum Q value of all actions that can be taken after the state transition; alpha is learning rate, which means the proportion of new Q value to the whole; gamma is the discount rate.
The application has the advantages that: the multi-layer MEC resource with the power cost difference is oriented, the weighted sum of the time cost and the unloading cost of all users is minimized as an optimization target, the problem is expressed as a mixed integer programming problem through joint unloading decision and resource allocation, and a solution scheme for optimizing transmission and unloading based on NOMA and Q-Learning is provided. Simulation results show that the multi-layer MEC architecture is superior to the traditional single-layer MEC architecture, and meanwhile, the algorithm is superior to other basic algorithms in solving.
Drawings
FIG. 1 is a schematic diagram of a network model according to an embodiment of the present application;
FIG. 2 is a graph of an iterative process for optimizing average transmission rate in an embodiment of the application;
FIG. 3 is a graph illustrating the overall cost change of an algorithm training process according to an embodiment of the present application;
FIG. 4 is a schematic diagram illustrating cost comparison under different transmission schemes according to an embodiment of the present application;
FIG. 5 is a diagram illustrating cost comparisons under different offloading strategies according to an embodiment of the present application.
Detailed Description
The application is further illustrated in the following figures and examples.
Examples:
as shown in fig. 1, a network model with three layers of (edge, star, cloud) computing resources is shown, under which a user device can offload computing tasks to a target server for processing by a base station. The base station is used as an access point of user equipment, integrates two devices of a traditional cellular base station and a ground satellite terminal, and communicates with the ground device and a low-orbit satellite by adopting a C wave band and a Ka wave band respectively.
Assuming that there are M users within the coverage area of the base station, each user has a computing task d i (i.epsilon.M) need to unload processing and computing task is not partitionable, user's task data set is noted asThe three layers of computing resources for providing computing services for users are respectively:
(1) A local MEC server at the base station side;
(2) A satellite MEC server mounted on the low orbit satellite;
(3) And a western cloud server built in a western region.
And the user equipment uploads the calculation task to the base station by adopting a NOMA technology, and the base station controller performs unified scheduling on the user task according to the task attribute of the user and the real-time state of each level of resources. Recording a=1 when the computing task is offloaded at the local MEC server, whereas a=0; recording b=1 when the computing task is offloaded at the satellite MEC server, whereas b=0; when the computing task is offloaded at the western cloud server, c=1 is noted, whereas c=0. We mark the offloading decision as
A NOMA transmission scheme is used in the transmission phase. In this scheme, it is assumed that each subchannel can be occupied by two users simultaneously, and that the users within the subchannels are orthogonal to each other, so that M users are divided into N pairs, i.e., there are N subchannels, denoted asAt the receiving end, decoding is performed according to the principle of NOMA uplink channel gain descending decoding. The second user is considered as interference when decoding the first user, so the transmission rate of the first decoded user is:
the second user is decoded while interfering with the user signal at the following transmission rate:
wherein B is the subcarrier bandwidth; p represents the transmit power of the user; n is n 0 Representing the noise power.
Since the user tasks are batched by the base station, and the distance between the base station and the satellite is a fixed value, the physical distance cannot be used only as a condition for measuring the channel gain and the satellite-to-ground channel user grouping. To simplify the model, we set the transmission rate of the ground satellite communication to be a constant R GS The transmission rate of the satellite-to-ground communication is constant R SG 。
In the task offloading process, time delay and offloading overhead are often two of the most interesting problems for users, and we divide the total cost of user task offloading into time cost and offloading cost, and represent their proportion of total cost by weight factors ω and v.
Wherein the time cost consists of transmission, propagation and processing delays, and is related to the distance of the user to the target server; the offloading costs consist of transmission overhead and computation overhead, related to energy consumption and local unit electricity prices.
(1) Local MEC offloading
In the local offloading scheme, since the calculation task is performed on the local MEC server at the base station side, the propagation delay is negligible. The local MEC server allocates a computational power f to UEi i L The number of CPU Cycles required per bit of data is β (Cycles/bit), so the local offload time cost is:
the first term in the formula is the transmission delay from the UE to the BS; the second term is the processing delay of the computing task.
The transmission power of the UEi is P i The transmission energy consumption of the UE to the BS is:
the most advanced CPU architecture typically employs dynamic voltage and frequency scaling techniques, dynamic power consumption P and V 2 f is proportional, and the calculated frequency f of the CPU chip is approximately linear with the power supply voltage V under the limitation of low voltage, i.e., v=af. We model the power consumption of the CPU as p=epsilonf 3 Where f is the calculation frequency of the CPU, epsilon is the coefficient of the chip architecture, and the calculation energy consumption of the user i at the local MEC server l is:
E i,l C ＝ε(f i L ) 2 d i β (23)
local unit electricity price p L The cost function available for local offloading is:
U i L (d,f)＝ωT i L +υ(E i T p L +E i,l C p L ) (24)
(2) Satellite MEC offloading
In the satellite unloading scheme, the distance from the BS to the access satellite is H, and the computing power allocated to the UEi by the satellite MEC server is f i S Therefore, the time cost of satellite unloading is:
the first term in the formula is the transmission delay from the UE to the BS; the second and third items are uplink transmission and propagation delay of the ground satellite communication, and the fourth item is processing delay of the calculation task.
The transmission power of the BS is P GS The transmission energy consumption of BS to satellite s is:
the calculation energy consumption of the UEi at the satellite MEC server s is:
E i,s C ＝ε(f i S ) 2 d i β (27)
the unit electricity price of the satellite server is p S The cost function of the available satellite offloads is:
U i S (d,f)＝ωT i S +υ(E i T p L +E gs T p L +E i,s C p S ) (28)
(3) Western cloud offloading
In the western offloading scheme, the computing task needs to be uploaded to the western cloud server through a satellite relay, and when the BS and the cloud server are not within the coverage area of the same LEO satellite, the computing task needs to be assisted by an inter-satellite link to perform data transmission. Because of the unpredictability of inter-satellite link states, transmission, propagation delays of computational tasks from one LEO satellite to another are ignored herein. The computing power allocated to UEi by the western cloud server is f i W The western unloading time costs are therefore:
the first term in the formula is the transmission delay from the UE to the BS; the second and third items are transmission delays of satellite-to-satellite communication, the fourth item is transmission delay of BS (base station) to the western cloud server through satellite relay, and the fifth item is processing delay of calculation tasks in the western cloud server.
The transmission power of the satellite is P SG The transmission energy consumption from the satellite s to the western cloud server w is:
the calculation energy consumption of the UEi at the western cloud server w is as follows:
E i,w C ＝ε(f i W ) 2 d i β (31)
the unit electricity price of the western cloud server is p W The cost function available for western offloading is:
U i W (d,f)＝ωT i W +υ(E i T p L +E gs T p L +E sg T p S +E i,w C p W ) (32)
since the calculation result is much smaller than the size of the input data, the backhaul link of the data is not considered herein.
Towards multi-tier MEC resources with latency and power cost differences, our goal is to minimize the weighted sum of all user time costs and offloading costs with limited computational power. For this, we express the optimization problem as:
in question (33), C1 indicates that the task is not separable, and the computing task can only be performed entirely by the local MEC server, satellite MEC server, or western cloud server; c2-C4 means that the computational resources allocated to the UE cannot exceed the computational power limit of the target server.
Due to offloading decisionsIs a binary variable and the resource allocation vector +.>Is dynamically changed, so the problem is a mixed integer programming problem, which is NP-hard, and the solution is very complex by adopting a traditional method, and a solution strategy based on Q-Learning is provided.
NOMA-based subchannel user recombination algorithm
During the transmission phase, the NOMA technology is adopted to allocate channel resources, and the distances from different UE to BS are differentI.e. the user channel gains are different, different combinations of users within the same sub-channel affect the transmission rate of the users. We useRepresenting a set of users;Representing a set of subchannels, wherein each subchannel comprises 2 users;Representing the sum of the rates of the users in each sub-channel, wherein +.>
Definition 1: selecting one user from any two sub-channels to perform position exchange to generate a new sub-channel user combination n i 'and n' j (i.noteq) and calculating q i 'and q' j User reorganization is successful if the following inequality is satisfied.
To find the best user combination for each sub-channel, we have employed a NOMA-based sub-channel user reorganization algorithm to increase the average transmission rate of the users according to definition 1.
Q-Learning-based computing offloading and resource allocation algorithm
Q-Learning is a model-free reinforcement Learning process that aims to enable an agent to learn a strategy in a strange environment to maximize the cumulative rewards that the agent obtains. A Q-Learning-based computing offloading and resource allocation algorithm (Q-CORAA) is adopted for multi-layer MEC resources.
The system model is defined with three key elements of state, action and rewards, which we represent as:
(1) State space: the state space is a set of available resources of the target server, expressed as
S＝{F L ，F S ，F W }。
(2) Action space: the action space is determined by unloading the decision vectorAnd resource allocation vectorTwo parts. Thus, the action space can be expressed as
A＝{af 1 …af M ,bf 1 …bf M ,cf 1 …cf M }。
(3) Rewarding: the rewards are related to the objective function, since our optimization problem is to minimize cost, while the goal of Q-Learning is to maximize rewards, we will state s k Lower execution action a k Is defined as R(s) k ,a k )＝-U(s k ,a k )。
In Q-Learning, the optimal strategy can be obtained by Q-functions, so our primary task is how to train a Q-function and let it converge. The state space S and the action space a are both finite, which is a finite markov decision process, and we can consider the Q function as a table storing the |s|row|a| columns of Q values, called a Q table, and make action selections according to the direction of the Q table.
The Q-table is initially empty and cannot direct the agent to select the optimal action, so it is required to interact with the environment to collect data, empirically populate and update the Q-table. In this process, we want the agent to be able to perform the optimal action using existing experience, and want it to be able to explore some unknown action options. Epsilon-greedy is a common strategy between trade-off utilization and exploration.
Definition: epsilon-greedy strategy
When the intelligent agent makes a decision, selecting the action with the maximum known action value according to the probability of epsilon (0 < epsilon < 1); the probability of 1-epsilon randomly selects an action to execute.
In the process of utilization or exploration, the Q table needs to be updated by equation (17) every time an action is performed. And after repeated training, the intelligent agent can be guided to select the optimal strategy until the Q table converges.
Wherein R represents a state s k Lower execution action a k The obtained instant rewards;representing the maximum Q value of all actions that can be taken after the state transition; alpha is learning rate, which means the proportion of new Q value to the whole; gamma is the discount rate, which defines the importance of future rewards, the larger the value the more important the long-term rewards are. The optimization problem (33) is solved by using algorithm 2. />
Simulation setup
Simulation was performed on MATLAB R2018b, considering a circular simulation area with a radius of 500m, with BS at the center, covering the entire area, and low-orbit satellites 500km above the simulation area.The UE is randomly distributed in the area, and each UE inputs the task bit number d i = (1-3) Mbits, the CPU cycle number β required to calculate one task bit=120 cycles/bit. The computing power of the local MEC, the satellite MEC and the western cloud server are respectively set to be 1Gcycles/s, 2Gcycles/s and 3Gcycles/s. The unit electricity price is dynamically adjusted according to the difficulty level of local power resource acquisition, p is the text L ＝0.8,p S ＝0.6,p W =0.4. The transmit power of each UE is 20dBm, the subcarrier bandwidth is 1MHz, the noise power is-100 dBm, and the UE-to-BS path loss is L (d) =15.3+37.6lgd. Assuming that there is always one LEO satellite providing full coverage to the area, the BS and LEO satellites transmit at 23dBm and the satellite-to-ground communication rate is 10Mbps.
TABLE 1 Main simulation parameters
Performance evaluation
In fig. 2 we show an iterative process of N-SURA, where the average transmission rate stabilizes after a limited number of iterations for 10 users, and it can be seen that a higher average transmission rate can be achieved with N-SURA compared to NOMA.
In fig. 3 we show the training process of the agent, it can be seen that as the training times increase, the agent gradually approaches the optimal offloading strategy, keeping the total cost at a low level.
In fig. 4, we solve the optimization problem by Q-CORA algorithm using different transmission schemes, respectively, and it can be seen that under different user numbers, the N-SURA effect is the best, NOMA times, and OFDMA is the worst.
From fig. 5 we can see that, under different numbers of users, the total cost of traditional single-layer MEC calculation offloading is the highest, the reason is mainly that the local unit electricity price is slightly higher than that of other layers, so that the cost of user offloading is greatly increased, and the feasibility of the multi-layer MEC architecture is verified. At the same time, the overall cost of using the Q-CORA algorithm is minimal under the multi-tier MEC architecture, demonstrating that the algorithm presented herein is significantly better than other offloading algorithms.

Claims (7)

1. A multi-layer MEC resource unloading method with power cost difference is characterized in that: the method comprises the following steps:
(1) Establishing a network model of a multi-layer MEC resource with power cost difference;
(2) Establishing a communication model and a calculation model under different resource levels;
(3) Allocating channel resources by using a sub-channel user recombination algorithm based on NOMA;
(4) And converting the optimization problem into an equivalent reinforcement Learning problem by using a Q-Learning-based calculation unloading and resource allocation algorithm, and converging a Q table by training the intelligent agent so as to guide the unloading decision of the intelligent agent of the base station.
2. The multi-tier MEC resource offloading method of claim 1, wherein: the establishing a network model of the multi-tier MEC resource with power cost differential of step (1) includes:
1) The user equipment downloads a calculation task to a target server through a Base Station for processing, wherein the Base Station (BS) is used as an access point of the user equipment, two devices of a traditional cellular Base Station and a ground satellite terminal are integrated, and a C wave band and a Ka wave band are respectively adopted for communication with the ground equipment and a low-orbit satellite;
2) The coverage area of the base station is provided with M users, and each user has a calculation task d i (i.epsilon.M) need to unload processing and computing task is not partitionable, user's task data set is noted asThe three layers of computing resources provide computing services for users;
3) The user equipment adopts NOMA technology to upload the calculation task to the base station, and the base station controller is used for controlling the base station according to the calculation taskThe task attribute of the user and the real-time state of each level of resource uniformly schedule the user task; recording a=1 when the computing task is offloaded at the local MEC server, whereas a=0; recording b=1 when the computing task is offloaded at the satellite MEC server, whereas b=0; when the computing task is offloaded at the western cloud server, c=1 is noted, whereas c=0. Marking offloading decisions as
3. The multi-tier MEC resource offloading method of claim 2, wherein: the three layers of computing resources for providing computing services for users are respectively:
(1) a local MEC server at the base station side;
(2) a satellite MEC server mounted on the low orbit satellite;
(3) and a western cloud server built in a western region.
4. The multi-tier MEC resource offloading method of claim 1, wherein: the establishing a communication model under different resource levels in the step (2) comprises the following steps:
1) With NOMA transmission scheme, it is assumed that each sub-channel can be occupied by two users at the same time, and the users in the sub-channels are orthogonal to each other, so that M users are divided into N pairs, i.e. N sub-channels, denoted as
2) At a receiving end, decoding is carried out according to a principle of NOMA uplink channel gain descending decoding; the second user is considered as interference when decoding the first user, so the transmission rate of the first decoded user is:
the second user is decoded while interfering with the user signal at the following transmission rate:
wherein B is the subcarrier bandwidth; p represents the transmit power of the user; n is n 0 Representing the noise power.
3) For user equipment in the coverage area of the base station, the base station controller is uniformly adopted to carry out batch processing on user tasks, and the transmission rate of the ground satellite communication is set as a constant R because the distance between the base station and the satellite is a fixed value GS The transmission rate of the satellite-to-ground communication is constant R SG 。
5. The multi-tier MEC resource offloading method of claim 1, wherein: the establishing of the calculation model under different resource levels in the step (2) comprises the following steps:
1) The total cost of user task offloading is divided into time cost and offloading cost, and its specific gravity to the total cost is represented by weight factors ω and v.
Wherein the time cost consists of transmission, propagation and processing delays, and is related to the distance of the user to the target server; the unloading cost consists of transmission cost and calculation cost, and is related to energy consumption and local unit electricity price;
2) The local MEC server allocates a computational power f to UEi i L The number of CPU Cycles required per bit of data is β (Cycles/bit), so the local offload time cost is:
the first term in the formula is the transmission delay from the UE to the BS; the second term is the processing time delay of the calculation task;
the transmission power of the UEi is P i The transmission energy consumption of the UE to the BS is:
dynamic power consumption P and V by dynamic voltage and frequency scaling 2 f is proportional, and the calculation frequency f of the CPU chip and the power supply voltage V are approximately in linear relation under the limit of low voltage, namely V=af; modeling power consumption of CPU as p=epsilonf 3 Where f is the calculation frequency of the CPU, epsilon is the coefficient of the chip architecture, and the calculation energy consumption of the user i at the local MEC server l is:
E i,l C ＝ε(f i L ) 2 d i β (6)
local unit electricity price p L The cost function available for local offloading is:
U i L (d,f)＝ωT i L +υ(E i T p L +E i,l C p L ) (7)
3) The distance from the BS to the access satellite is H, and the computing power allocated to the UEi by the satellite MEC server is f i S Therefore, the time cost of satellite unloading is:
the first term in the formula is the transmission delay from the UE to the BS; the second and third items are uplink transmission and propagation delay of the ground satellite communication, and the fourth item is processing delay of a calculation task;
the transmission power of the BS is P GS The transmission energy consumption of BS to satellite s is:
the calculation energy consumption of the UEi at the satellite MEC server s is:
E i,s C ＝ε(f i S ) 2 d i β (10)
the unit electricity price of the satellite server is p S The cost function of the available satellite offloads is:
U i S (d,f)＝ωT i S +υ(E i T p L +E gs T p L +E i,s C p S ) (11)
4) The computing power allocated to UEi by the western cloud server is f i W The western unloading time costs are therefore:
the first term in the formula is the transmission delay from the UE to the BS; the second term and the third term are transmission delays of the satellite-to-satellite communication, the fourth term is the transmission delay of the BS to the western cloud server through satellite relay, and the fifth term is the processing delay of the computing task in the western cloud server;
the transmission power of the satellite is P SG The transmission energy consumption from the satellite s to the western cloud server w is:
the calculation energy consumption of the UEi at the western cloud server w is as follows:
E i,w C ＝ε(f i W ) 2 d i β (13)
the unit electricity price of the western cloud server is p W The cost function available for western offloading is:
U i W (d,f)＝ωT i W +υ(E i T p L +E gs T p L +E sg T p S +E i,w C p W ) (14)。
6. the multi-tier MEC resource offloading method of claim 1, wherein: step (3) of allocating channel resources by using a sub-channel user reorganization algorithm based on NOMA, including:
1) By usingRepresenting a set of users;Representing a set of subchannels, wherein each subchannel comprises 2 users;Representing the sum of the rates of users within each sub-channel, where
2) Selecting one user from any two sub-channels to perform position exchange to generate a new sub-channel user combination n' i And n' j (i.noteq.j) and calculating q' i And q' j User reorganization is successful if the following inequality is satisfied:
7. the multi-tier MEC resource offloading method of claim 1, wherein: the Q-Learning-based computing unloading and resource allocation algorithm in the step (4) converts the optimization problem into an equivalent reinforcement Learning problem for solving, and the method comprises the following steps:
1) Converting the optimization problem into an equivalent reinforcement learning problem, namely:
state space: the state space is the set of available resources of the target server, denoted s= { F L ，F S ，F W }；
Action space: the action space is determined by unloading the decision vectorAnd resource allocation vector +.>Two-part composition, denoted as a= { af 1 …af M ,bf 1 …bf M ,cf 1 …cf M }；
Rewarding: state s k Lower execution action a k Is defined as R(s) k ,a k )＝-U(s k ,a k )。
2) The base station controller selects the action with the maximum known action value according to the probability of epsilon (0 < epsilon < 1); the probability of 1-epsilon randomly selects an action to execute:
3) In the process of utilizing or exploring, the Q table is required to be updated through a formula (17) every time an action is executed; through repeated training, the intelligent agent can be guided to select the optimal strategy until the Q table converges:
wherein R represents a state s k Lower execution action a k The obtained instant rewards;representing the maximum Q value of all actions that can be taken after the state transition; alpha is learning rate, which means the proportion of new Q value to the whole; gamma is the discount rate.
CN202310443772.1A 2023-04-20 2023-04-20 Multi-layer MEC resource unloading method with power cost difference Pending CN116600344A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310443772.1A CN116600344A (en) 2023-04-20 2023-04-20 Multi-layer MEC resource unloading method with power cost difference

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310443772.1A CN116600344A (en) 2023-04-20 2023-04-20 Multi-layer MEC resource unloading method with power cost difference

Publications (1)

Publication Number Publication Date
CN116600344A true CN116600344A (en) 2023-08-15

Family

ID=87598259

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310443772.1A Pending CN116600344A (en) 2023-04-20 2023-04-20 Multi-layer MEC resource unloading method with power cost difference

Country Status (1)

Country Link
CN (1) CN116600344A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117200873A (en) * 2023-11-07 2023-12-08 南京邮电大学 Calculation unloading method considering satellite mobility in satellite edge calculation network
CN117714446A (en) * 2024-02-02 2024-03-15 南京信息工程大学 Unloading method and device for satellite cloud edge cooperative computing

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117200873A (en) * 2023-11-07 2023-12-08 南京邮电大学 Calculation unloading method considering satellite mobility in satellite edge calculation network
CN117200873B (en) * 2023-11-07 2024-05-31 南京邮电大学 Calculation unloading method considering satellite mobility in satellite edge calculation network
CN117714446A (en) * 2024-02-02 2024-03-15 南京信息工程大学 Unloading method and device for satellite cloud edge cooperative computing
CN117714446B (en) * 2024-02-02 2024-04-16 南京信息工程大学 Unloading method and device for satellite cloud edge cooperative computing

Similar Documents

Publication Publication Date Title
Chai et al. Joint multi-task offloading and resource allocation for mobile edge computing systems in satellite IoT
Fadlullah et al. HCP: Heterogeneous computing platform for federated learning based collaborative content caching towards 6G networks
Yang et al. Privacy-preserving federated learning for UAV-enabled networks: Learning-based joint scheduling and resource management
Zhou et al. Machine learning-based resource allocation in satellite networks supporting internet of remote things
Xu et al. Deep reinforcement learning based joint edge resource management in maritime network
Liu et al. Energy-efficient space–air–ground integrated edge computing for internet of remote things: A federated DRL approach
CN116600344A (en) Multi-layer MEC resource unloading method with power cost difference
Shahjalal et al. Enabling technologies for AI empowered 6G massive radio access networks
Zhu et al. Deep reinforcement learning-based task offloading in satellite-terrestrial edge computing networks
CN114665952B (en) Low-orbit satellite network beam-jumping optimization method based on star-ground fusion architecture
CN114051254B (en) Green cloud edge collaborative computing unloading method based on star-ground fusion network
CN113377533B (en) Dynamic computing unloading and server deployment method in unmanned aerial vehicle assisted mobile edge computing
Yu et al. IRS assisted NOMA aided mobile edge computing with queue stability: Heterogeneous multi-agent reinforcement learning
Dinh et al. Energy efficient resource allocation optimization in fog radio access networks with outdated channel knowledge
Zhang et al. Efficient Multitask Scheduling for Completion Time Minimization in UAV‐Assisted Mobile Edge Computing
CN118250750B (en) Satellite edge computing task unloading and resource allocation method based on deep reinforcement learning
Dai et al. Hybrid quantum-behaved particle swarm optimization for mobile-edge computation offloading in internet of things
Lee et al. Multi-Agent Reinforcement Learning in Controlling Offloading Ratio and Trajectory for Multi-UAV Mobile Edge Computing
CN116886172A (en) Multi-beam satellite communication user selection and spectrum division method based on machine learning
Mahmoud et al. Federated learning resource optimization and client selection for total energy minimization under outage, latency, and bandwidth constraints with partial or no CSI
He et al. Energy minimization for UAV-enabled wireless power transfer and relay networks
Wang et al. Joint power control and task offloading in collaborative edge–cloud computing networks
Li et al. Multiagent Reinforcement Learning for Task Offloading of Space/Aerial‐Assisted Edge Computing
Maray et al. Wireless Power Assisted Computation Offloading in Mobile Edge Computing: A Deep Reinforcement Learning Approach
CN117835418A (en) Air-ground cooperative monitoring resource allocation method and device based on cellular network unmanned aerial vehicle

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination