CN116669111A - Mobile edge computing task unloading method based on blockchain - Google Patents
Mobile edge computing task unloading method based on blockchain Download PDFInfo
- Publication number
- CN116669111A CN116669111A CN202310802546.8A CN202310802546A CN116669111A CN 116669111 A CN116669111 A CN 116669111A CN 202310802546 A CN202310802546 A CN 202310802546A CN 116669111 A CN116669111 A CN 116669111A
- Authority
- CN
- China
- Prior art keywords
- time slot
- task
- blockchain
- edge
- edge device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000004364 calculation method Methods 0.000 claims abstract description 39
- 230000009471 action Effects 0.000 claims abstract description 38
- 238000005065 mining Methods 0.000 claims abstract description 27
- 230000006870 function Effects 0.000 claims abstract description 26
- 238000013468 resource allocation Methods 0.000 claims abstract description 25
- 230000008569 process Effects 0.000 claims abstract description 23
- 238000005457 optimization Methods 0.000 claims abstract description 14
- 230000002787 reinforcement Effects 0.000 claims abstract description 12
- 230000005540 biological transmission Effects 0.000 claims description 34
- 230000000875 corresponding effect Effects 0.000 claims description 16
- 238000002360 preparation method Methods 0.000 claims description 14
- 238000012549 training Methods 0.000 claims description 13
- 230000003993 interaction Effects 0.000 claims description 7
- 238000004891 communication Methods 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 5
- 230000033001 locomotion Effects 0.000 claims description 3
- 238000010295 mobile communication Methods 0.000 abstract description 2
- 238000005265 energy consumption Methods 0.000 description 9
- 230000007246 mechanism Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000001934 delay Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 206010033799 Paralysis Diseases 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000000593 degrading effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000035755 proliferation Effects 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/08—Load balancing or load distribution
- H04W28/09—Management thereof
- H04W28/0925—Management thereof using policies
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/50—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols using hash chains, e.g. blockchains or hash trees
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W12/00—Security arrangements; Authentication; Protecting privacy or anonymity
- H04W12/10—Integrity
- H04W12/106—Packet or message integrity
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W12/00—Security arrangements; Authentication; Protecting privacy or anonymity
- H04W12/10—Integrity
- H04W12/108—Source integrity
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/08—Load balancing or load distribution
- H04W28/09—Management thereof
- H04W28/0958—Management thereof based on metrics or performance parameters
Abstract
The invention belongs to the technical field of mobile communication, and particularly relates to a mobile edge computing task unloading method based on a blockchain, which comprises the steps of establishing a mobile edge computing task unloading model based on the blockchain aiming at a dynamic MEC scene of a plurality of servers; with the aim of minimizing the cost of a user for completing a calculation task and maximizing the utility obtained by the user in mining, a task unloading and resource allocation joint optimization model is established under the condition of multidimensional resource constraint; the task unloading cost problem and the blockchain mining utility problem are abstracted into a partially observable Markov decision process in consideration of the random time-varying network environment and the partial observability of the environment state; obtaining a state space and an action space of the MDP problem according to the established system model, and constructing a reward function; adopting a multi-agent reinforcement learning algorithm to make optimal unloading and resource allocation decisions; the invention realizes lower task unloading cost and higher blockchain mining utility.
Description
Technical Field
The invention belongs to the technical field of mobile communication, and particularly relates to a mobile edge computing task unloading method based on a block chain.
Background
With the development of the internet, the number of mobile devices has dramatically increased, and many time-delay sensitive, computationally intensive applications, such as virtual reality, interactive online gaming, face recognition, ultra-high definition video, and the like, have emerged. Because the computing resources and the storage resources of the mobile device are limited, the delay-sensitive and computation-intensive application cannot be completed efficiently, so that the user experience is poor. Aiming at the problems, mobile cloud computing is provided, namely a user sends a computing task to a cloud end, the computing task is completed by utilizing abundant computing resources of the cloud end, and after the computing is completed, the result is transmitted back to the user end. However, since the user is usually far from the cloud, the time delay is often large, so that the mobile cloud computing is not suitable for the time delay sensitive application.
In order to solve the problem of high latency in mobile cloud computing, mobile Edge Computing (MEC) has been developed. Mobile edge computation was first proposed by the european telecommunications standards institute in 2014. Unlike cloud computing, MEC offloads computing tasks to edge devices equipped with edge servers for computing, so that computing resources and storage resources are closer to mobile devices, time delay and user energy consumption are reduced, task offloading efficiency and user experience are improved, and therefore mobile edge computing is receiving extensive attention from academia and industry.
Along with the proliferation of the number of IOT sensors, the number of edge devices is also increased, the number of tasks to be processed in the coverage area of one MEC server is increased, and one MEC server is difficult to simultaneously meet all calculation requirements, so that the scene of cooperation of multiple MEC servers is considered. In MEC systems, MEC servers are typically from different service providers, there may be collisions of interests between different servers, trust is difficult to establish between parties, and edge devices are heterogeneous, interactions between heterogeneous edge nodes, and migration of services across nodes may raise security and privacy concerns.
By virtue of the advantages of decentralization, tamper resistance, transparency, non-variability, traceability, anonymity and the like, the blockchain technology can construct a safe and credible transaction environment in a distributed system, and the problems of safety and privacy are solved. In the blockchain auxiliary MEC system, encryption technologies such as an asymmetric encryption algorithm, a hash algorithm and the like are used in the interaction between a user and an edge server to protect privacy safety of the interaction process, and besides, the blockchain can carry out consistency confirmation on the transaction records through a consensus mechanism, so that the integrity and reliability of the transaction records are ensured. In the MEC system, a central controller is required to make an unloading decision, if the mechanism is attacked, the whole MEC system is paralyzed, which is also called single-point failure, and the blockchain has distributed characteristics and a common-knowledge mechanism, so that the normal operation of the system can be maintained under the condition that a few nodes are attacked. Therefore, the safety of the MEC system can be improved by reasonably combining the blockchain with the MEC.
Disclosure of Invention
In order to achieve lower task unloading cost and higher mining effect of a blockchain, the invention provides a mobile edge computing task unloading method based on the blockchain, which specifically comprises the following steps:
aiming at a dynamic MEC scene of multiple servers, a mobile edge computing task unloading model based on block chains is established in consideration of the problems of security and privacy caused by the computation of the MEC servers, the time-varying communication resources and the interaction between heterogeneous edge nodes;
with the aim of minimizing the cost of a user for completing a calculation task and maximizing the utility obtained by the user in mining, a task unloading and resource allocation joint optimization model is established under the condition of multidimensional resource constraint;
the task unloading cost problem and the blockchain mining utility problem are abstracted into a partially observable Markov decision process in consideration of the random time-varying network environment and the partial observability of the environment state;
obtaining a state space and an action space of the MDP problem according to the established system model, and constructing a reward function;
and adopting a multi-agent reinforcement learning algorithm to make optimal unloading and resource allocation decisions.
Further, the blockchain-based moving edge computing task offload model includes:
When one edge device has a calculation task unloading requirement, sending an unloading requirement to an MEC server layer, after receiving a plurality of reply messages, searching the reliability of candidates by inquiring a reliability table of MEC servers stored in a blockchain, selecting an appropriate server for calculation unloading according to the reliability and channel conditions and available calculation resources of the servers, and periodically updating the reliability table according to verified transactions of calculation task unloading;
storing the reliability of each MEC server in a blockchain ledger, and inquiring the reliability from the blockchain when the edge equipment makes an unloading decision;
in the consensus process, the master node is responsible for generating the block, the duplicate node is responsible for verifying the block, and the non-selected edge equipment is used as a common node and is only responsible for adding the verified block into the maintained blockchain account book.
Further, the block chain consensus process includes:
x CPU cycles are required for signing a block or transaction, y CPU cycles are required for verifying a signature, z CPU cycles are required for generating a MAC, and z CPU cycles are required for verifying a MAC;
the master node collects unverified transactions from all edge devices and then sorts the transactions according to the time stamp, assuming the block size is S b (t) the average size of the transactions is χ, then the number of transactions in a block is:where it isAt one stage, the master node needs to verify the signatures and the MACs of the L transactions, so the calculation cost of the master node is L (y+z);
the master node generates a signature and MAC for the block, N s -1 duplicate node generating a MAC for the pre-prepare message, each duplicate node verifying the MAC of the block, and the signature and MAC of the L transactions in the block, the master node calculating a cost of x+n s z, the computation cost of each duplicate node is z+L (y+z), and assuming that the transmission time of a block is proportional to the size of the block, the message transmission time is τ b S b ；
After the copy node verifies the pre-preparation information, the preparation information is sent to other consensus nodes, and after each consensus node receives 2f preparation information, the next stage is entered, in the stage, the master node needs to verify 2f MAC, so the calculation cost is 2fz; each replica node needs to generate N s 1 MAC and authentication 2f MAC, the computational cost of each duplicate node is (N s -1) z+2fz, message transmission time τ b S b ；
After receiving 2f preparation messages, each consensus node sends a commit message to other nodes, including the master node, at which stage the master node and duplicate nodes need to verify 2f MACs, yielding N s 1 MAC, so the computation cost of the master and replica nodes is (N s -1) z+2fz, message transmission time τ b S b ；
After receiving 2f commit messages, the master node and the replica node consider the block as legal, add the block to the blockchain ledger and send a reply message containing the verified block to other edge devices, update the global view after receiving f+1 reply messages, in which stage the master node and the replica node need to generate a MAC for the reply message, so that the calculation cost of the master node and the replica node is z, and the message transmission time is τ b S b 。
Further, establishing the task unloading and resource allocation combined optimization model comprises the following steps:
constraint conditions:
wherein ,xn (t) represents a local observation of edge device n at time slot t; p is p n (t) represents the transmission power at which the edge device n offloads tasks to the MEC server m at time slot t;the method comprises the steps that computing resources allocated for executing unloading tasks of edge equipment n by an MEC server m under a time slot t are allocated; />Computing resources allocated for executing tasks for edge device n under time slot t; />Computing resources allocated for block chain mining for the edge equipment under the time slot t; />The utility is the consensus of the edge device n under the time slot t; cost (t) is the Cost of the edge device under time slot t; x is x n (t) represents an offloading decision of edge device n at time slot t, x if the computation task is offloaded to MEC server i for execution at time slot t n (t) =i, x if the task is executed locally n (t)＝0；/>N is the number of the edge devices; p (P) n Is the maximum value of the transmission power; />Discretizing the system operation time into a set of T time slots; f (F) n Maximum computing resource for edge device n; f (F) m Is the maximum computing resource of server m; />Is a set of MEC servers; phi (phi) n The maximum computing resource for mining the blockchain for the edge device n; τ n Representing the maximum time delay tolerable for the task; t (T) n And (t) is the processing delay of the edge device n in the time slot t.
Further, abstracting the task offloading cost problem and the blockchain mining utility problem into partially observable markov decision processes includes: the edge device acts as an agent and defines the tuple { S, O, A, R } describes the Markov game process, wherein S represents a global state space, and the environment of the time slot t is a global state S (t) ∈S, O= { O 1 ,O 2 ,...,O N [ is the observation space set of the agent, O n A value space corresponding to the observation space of the edge equipment n; a= { a 1 ,A 2 ,...,A N Is the movement of the intelligent bodyMake space collection, A n A value space corresponding to the action space of the intelligent body of the edge equipment n; r= { R 1 ,R 2 ,...,R N And R is the reward set n A value space corresponding to rewards of the edge equipment n; at time slot t, agent N observes o from local n (t)∈O n Adopts a policy pi n :O n →A n Selecting a corresponding action a n (t)∈A n Thereby obtaining corresponding rewards r n (t)∈R n 。
Further, the state space of the MDP problem includes the size of the calculation task, the tolerable delay, the channel state, the resource state of the edge device and the state of the MEC server contained in the observed information of the single agent in the time slot t, and the state space of the time slot t is expressed as:
o n (t)＝{o task (t),o channel (t),o resource (t),o server (t)}
wherein ,otask (t)＝{C n (t),D n (t),τ n (t) } is the observation information of the task at time slot t, C n (t) is the computational resource required for the computational task at time slot t, D n (t) is the size of the calculation task at time slot t, τ n (t) is the maximum tolerable delay of the computing task at time slot t; o (o) channel (t) is the observed information of the channel at time slot t,the resource state of the edge device n under the time slot t; o (o) server (t)＝{μ m (t),F m (t) } is the MEC server status information observed by the edge device n at time slot t, μ m (t) is the reliability of MEC server m in time slot t, F m And (t) is the available computing resources of MEC server m at time slot t.
Further, the action space of the MDP problem includes actions of the single agent of the time slot t including unloading decision, transmission power selection, computing resource allocation, consensus resource allocation, and the action set of the time slot t is expressed as:
Further, the bonus function is expressed as:
wherein r (t) represents a bonus function value at time slot t; r is (r) n (t) represents a prize function value of the edge device n at the time slot t; n is the number of edge devices.
Further, an optimal unloading and resource allocation decision is made by adopting a multi-agent reinforcement learning algorithm, wherein the multi-agent reinforcement learning algorithm consists of an environment and N agents, each agent has a centralized training stage and a decentralized execution stage, in the training stage, a critic network and an actor network are trained by adopting centralized learning, and state information of other agents is needed to be used when the critic network is trained; in the decentralized execution stage, the actor only needs to know local information, and adjusts the local strategy according to the estimation strategy of other intelligent agents so as to achieve global optimum, and the method specifically comprises the following steps:
let pi= { pi 1 ,π 2 ,...,π N And (2) strategy set of all agents, wherein θ= { θ 1 ,θ 2 ,...,θ N Each agent updates the parameter theta as the parameter set of the corresponding strategy n To obtain an optimal strategy;
in the decentralized execution stage, at each time slot t, the actor network of each agent observes the state o according to the local n (t) and its own policy selection actions, expressed as:
a n (t)＝π n o n (t)
During the centralized training phase, each critic network can obtain observations o of other agents n (t) and action a n (t), the Q function of agent n can be expressed as:
the Q function evaluates the action of the actor network from the global angle and guides the actor network to select a better action;
in training, the critic network updates the network parameters by minimizing a loss function, expressed as:
the actor network updates network parameters based on the loss function calculated by the critic network and the observation information of the actor network, and outputs actions; the actor updates the network by calculating the gradient of the objective function:
the parameters of the target network are updated in a soft update mode, namely:
wherein ,gamma is a discount factor; e (E) o,a,r,o' [·]Representing the expectation of the expression, wherein o is an observation set, a is an action set, r is a reward set, and o' is an observation set of the next time slot; r is (r) n Rewards for edge device n under time slot t; o' n (t) is the observation of the next slot agent n for slot t; a' n ＝π n (o n ) To be o in the observation set n When according to policy pi n Selected action, pi n (. Cndot.) represents the policy of agent n; e (E) o,a～D [·]Representing the desire for the expression +.>Representing the parameter θ for the expression with respect to the current network of the actor n Gradient, pi n (a n |o n ) For intelligent body observe o n Action a is made downwards n Is a policy of (2); />For the soft update coefficient of the actor network, theta n Is the parameter of the current network of the actor, theta n ' is a parameter of the actor target network, +.>For soft update coefficients, ω, of critic networks n Omega 'is a parameter of the critic current network' n Is a parameter of the critic target network.
The invention researches a task unloading and resource allocation method in a multi-server MEC scene, considers that the computation of the MEC server, the time-varying communication resource and the channel state and the interaction among heterogeneous edge nodes can cause the security and privacy problems, and establishes a mobile edge computing task unloading model based on a blockchain; then, with the aim of minimizing the cost of completing the calculation task by the user and maximizing the utility obtained by the user in mining, establishing a task unloading and resource allocation joint optimization model under the condition of multidimensional resource constraint; meanwhile, the task unloading cost problem and the blockchain mining utility problem are abstracted into a partially observable Markov decision process by considering the random time-varying network environment and the partial observability of the environment state; the method adopts a deep reinforcement learning algorithm to solve, and an agent learns historical information of task unloading and resource allocation to make a better decision.
Drawings
FIG. 1 is a diagram of a typical MEC system model contemplated by the present invention;
FIG. 2 is a flow chart of a method for blockchain-based mobile edge computing task offloading and resource allocation in accordance with the present invention;
FIG. 3 is a frame diagram of the MADDPG algorithm employed in the present invention;
FIG. 4 is a chain data structure of a blockchain employed in the present invention;
fig. 5 is a consensus process of the PBFT-based consensus algorithm employed in the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The invention provides a mobile edge computing task unloading method based on a block chain, as shown in fig. 2, which specifically comprises the following steps:
aiming at a dynamic MEC scene of multiple servers, a mobile edge computing task unloading model based on blockchain is established in consideration of the problems of security and privacy caused by the computation of the MEC servers, the time-varying communication resources and the interaction between heterogeneous edge nodes;
With the aim of minimizing the cost of a user for completing a calculation task and maximizing the utility obtained by the user in mining, a task unloading and resource allocation joint optimization model is established under the condition of multidimensional resource constraint;
the task unloading cost problem and the blockchain mining utility problem are abstracted into a partially observable Markov decision process in consideration of the random time-varying network environment and the partial observability of the environment state;
obtaining a state space and an action space of the MDP problem according to the established system model, and constructing a reward function;
and adopting a multi-agent reinforcement learning algorithm to make optimal unloading and resource allocation decisions.
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.
1. System model
As shown in fig. 1, the present invention contemplates a typical MEC system consisting of four layers, one for each: the system comprises an IOT sensor layer, an edge device layer, an MEC server layer and a cloud server layer. The IOT sensor layer consists of a camera, a smart meter, a wearable device, a health detection device and the like, senses information from a physical environment and generates data to be calculated. The IOT sensor is used as a lightweight block chain node, representative edge equipment is selected to perform mining and generate blocks, and the set of the edge equipment is set as That is, the system includes N edge devices, the edge devices are responsible for collecting data from a group of IOT sensors managed by the edge devices, analyzing the data, deciding whether to execute on the edge devices or unload the data to the MEC server, and setting the set of the MEC server as->I.e. the system comprises M MEC servers, the system operation time is discretized into T time slots, denoted +.>For simplicity, assume that each edge device has only one computation task at the current slot, defined as S n ＝{C n ,D n ,τ n }，C n Indicating completion of task S n Required computational resources (CPU cycles), D n Representing the size of the task τ n Representing the maximum time delay that the task can tolerate.
The edge device collects the data to be calculated from a group of IOT sensors and makes offloading decisions by analyzing the data size, tolerable delay, channel conditions, and computational resources of the MEC server. After the task processing is completed, if the task is processed at the MEC server, the result is returned to the edge device, the edge device verifies the result and evaluates the performance of the task performer, and if the result is verified to be valid, the edge device pays service fees to the MEC and updates the reliability of the MEC server stored in the blockchain.
1. Communication model
Definition of the definitionFor offloading decisions of edge device n at time slot t, x is if the computation task is offloaded to MEC server i for execution at time slot t n (t) =i, x if the task is executed locally n (t) =0. The uplink channel gain between the edge device and the MEC server is defined as h mn (t) the transmission power of the t-slot lower edge device n for offloading tasks to the MEC server m is p n (t) the transmission power policy is +.> wherein />To offload tasks to a set of edge devices of an MEC server, p n Offloading tasks to transmission power, P, of MEC server m for edge device n n For maximum transmission power for edge device n to offload tasks to MEC server m, assuming W is the bandwidth between edge device n and MEC server m, the transmission rate between edge device n and MEC server m at time slot t is:
wherein ,σ2 (t) is background noise power of a transmission channel between the edge device n and the MEC server m under the time slot t, in the same time slot, the channel parameters are not changed, and in different time slots, the channel parameters are different; d, d mn α is the path loss index, which is the distance between the edge device n and the MEC server m.
The transmission delay between the edge device n and the MEC server m is:
2. Local computing model
Assume thatFor the computing resource allocated by the edge device n for executing the task when the time slot t is, the computing resource allocation policy of the edge device is +.> wherein Fn For the maximum computing resource of the edge device n, the delay of the edge device n to execute the computing task locally is:
the energy consumption for locally executing the computing task is:
wherein ,κn Is the energy coefficient specific to the chip structure in the edge device n.
MEC server calculation model
Assume thatFor the computing resource allocated by the MEC server m for executing the offloading task of the edge device n at the time slot t, the time delay of executing the task on the MEC server m is:
the total unloading delay is:
in offloading tasks, the energy consumption cost of the edge device n is only related to data transmission, given by:
in MEC systems, the quality of service mainly includes two aspects: task completion time T n Energy consumption E n . At time slot T, the total delay T of edge device n n And energy consumption E n Expressed as:
Cost(t)＝λ T T n (t)+λ E E n (t) (10)
where Cost (t) represents the Cost of the edge device at time slot t, lambda T Is a weight factor of time delay; lambda (lambda) E Is a weight factor of energy consumption.
4. Edge equipment model
When an edge device offloads a calculation task to an MEC server, the edge device needs to pay a service fee to the MEC server, assuming that the service fee for the MEC server to execute a calculation task is proportional to the calculation amount of the calculation task, and assuming that the calculation service unit price is q n The service unit price is determined by the MEC server, and the size is C n The calculation cost of the calculation task of (a) is as follows: q n C n . The utility of offloading a task is related to the completion time of the task, as shown in the following equation:
wherein ,τn Is the maximum tolerable delay of the task, and therefore the utility of the edge device to offload tasks is:
U n ＝u n -q n C n (12)
MEC server model
In the task unloading process, once the MEC server receives an unloading request, the MEC server allocates corresponding resources to process the task, and the energy consumption of the MEC server to execute the task is as follows:
the utility of MEC server m to perform task n is:
wherein ω is the unit price of energy consumption.
As can be seen from the above equation, the utility of the MEC server depends on the computational service unit price and the computational resources allocated for the task, and the MEC server based on the selfish behavior increases the utility of the task by allocating fewer resources for the task and reducing the energy consumption for executing the task, but this decreases the utility of the task, and makes the task unable to complete within the maximum tolerable delay, thereby decreasing the utility of the edge device. To prevent the selfish MEC server from allocating insufficient resources for the task, a reliability model is built to evaluate the MEC server's efficiency in performing the off-load task. The reliability of each MEC server is updated every other time slot t, for example, MEC server m, assuming that the number of tasks to be performed during time slot t, offloaded to MEC server m, is N m 。
Defining a normalized utility of an offload task completed within a maximum tolerable delay:
wherein log (1+τ) n ) Representing the maximum utility of the task.
The calculation efficiency update expression of the MEC server m is:
wherein ,ρm (t-1) is the computational efficiency before the MEC server m.
The task completion rate of the MEC server m is:
wherein ,ρm (t-1) is the task completion rate, L, of MEC server m before t time slot m The number of tasks completed within the maximum tolerable delay requirement for time slot t.
Thus, the reliability of MEC server m at time slot t is defined as follows:
μ m (t＝ηρ m (t)+(1-η)δ m (t),η∈(0,1) (18)
where η is a weight factor.
6. Block chain model
In a blockchain-based MEC system, the blockchain consensus process plays an important role, and an important factor affecting the performance of the blockchain system is the consensus delay, so that the optimization of the consensus delay is required in the blockchain consensus, and the chain data structure of the blockchain adopted in this embodiment is shown in fig. 4. Existing consensus algorithms (PoW, poS, etc.) suffer longer delays during the consensus process, degrading the performance of the blockchain system, so the PBFT-based enhanced consensus algorithm optimizes the consensus delay, as shown in fig. 5, where the consensus nodes are dynamically selected based on the computational resources available for consensus in the edge devices and the consensus utility of the edge devices The number of the consensus nodes is N s The edge device groups the calculation task unloading transaction records, stores the calculation task unloading transaction records on an immutable and tamper-proof blockchain ledger after passing the PBFT consensus verification, and completes blockchain uploading.
When an edge device has a computational task offloading requirement, it first sends an offloading requirement to the MEC server layer, and after receiving a plurality of reply messages, finds the reliability of the candidate by querying the reliability table of the MEC servers stored in the blockchain, selects an appropriate server for computational offloading according to the reliability and channel conditions, and available computational resources of the servers, and periodically updates the reliability table according to validated transactions for computational task offloading. The reliability of each MEC server is stored in the blockchain ledger, and when the edge equipment makes an unloading decision, the reliability can be quickly queried from the blockchain, so that the unloading decision can be quickly made, and the unloading efficiency is improved.
In the consensus process, the master node is responsible for generating the block, the duplicate node is responsible for verifying the block, and the non-selected edge equipment is used as a common node and is only responsible for adding the verified block into the maintained blockchain account book. In blockchain consensus, signature and Message Authentication Codes (MACs) are utilized to guarantee data integrity and authentication of transactions, requiring x, y, z, z CPU cycles for a block or transaction signature, verifying a signature, generating a MAC, verifying a MAC, respectively. The main steps of consensus are shown below.
a)Collect
The master node collects unverified transactions from all edge devices and then sorts the transactions according to the time stamp, assuming the block size is S b (t) the average size of the transactions is χ, then the number of transactions in a block is:at this stage, the master node needs to verify the signatures and MACs of the L transactions, so the computation cost of the master node is L (y+z); .
b)Pre-prepare
At this stage, the master node generates a signature for the blockAnd MAC of N s -1 duplicate node generating a MAC for the pre-prepare message, each duplicate node verifying the MAC of the block, and the signature and MAC of the L transactions in the block, the master node calculating a cost of x+n s z, the computation cost of each duplicate node is z+L (y+z), and assuming that the transmission time of a block is proportional to the size of the block, the message transmission time is τ b S b 。
c)Prepare
After the copy node verifies the pre-preparation information, the preparation information is sent to other consensus nodes, and after each consensus node receives 2f preparation information, the next stage is entered, in the stage, the master node needs to verify 2f MAC, so the calculation cost is 2fz; each replica node needs to generate N s 1 MAC and authentication 2f MAC, the computational cost of each duplicate node is (N s -1) z+2fz, message transmission time τ b S b . f is the number of nodes (malicious nodes) with problems in the network, the number of the malicious nodes in the network is regulated to be more than or equal to 3f+1 in the PBFT protocol, and N is the total number of the nodes in the network.
d)Commit
After the copy node verifies the pre-preparation information, the preparation information is sent to other consensus nodes, and after each consensus node receives 2f preparation information, the next stage is entered, in the stage, the master node needs to verify 2f MAC, so the calculation cost is 2fz; each replica node needs to generate N s 1 MAC and authentication 2f MAC, the computational cost of each duplicate node is (N s -1) z+2fz, message transmission time τ b S b 。
e)Reply
After receiving 2f commit messages, the master node and the replica node consider the block as legal, add the block to the blockchain ledger and send a reply message containing the verified block to other edge devices, update the global view after receiving f+1 reply messages, in which stage the master node and the replica node need to generate a MAC for the reply message, so that the calculation cost of the master node and the replica node is z, and the message transmission time is τ b S b 。
Finally, the calculation time of the master node is:
wherein ,is a computing resource for consensus in the master node.
The computation time of the duplicate nodes is:
wherein ,is a computing resource for consensus among duplicate nodes.
TTF is used to represent the delay of the consensus process:
T F ＝T I +T D +T V (21)
wherein ,TI For block interval, T D ＝4τ b S b For block transmission time, T V ＝max{T pri ,T rep And is the block verification time.
In the block chain assisted MEC system, edge equipment is used as edge miners to mine, in each round of consensus, each IOT sensor user votes the edge equipment to select the edge miners based on the consensus utility and resources available for consensus in the edge equipment, the performance of the block chain consensus is mainly determined by the block size and the computing resources available for consensus in ED, and the consensus utility of the edge equipment n is defined as follows:
wherein ,τn For the maximum tolerable delay of a task,is the consensus time of edge device n. The IOT sensor performs mining according to the calculation resource selection available for consensus in the edge equipment, wherein the more the calculation resources for consensus are, the lower the consensus time delay is, the higher the consensus utility is, and the better the performance of the blockchain system is.
2. Problem modeling
In a blockchain-based MEC system, each edge device needs to perform task offloading and blockchain mining at the same time, so the evaluation of system performance needs to take into account the performance of mining and task offloading. For task offloading, edge devices need to minimize offloading costs to maintain performance of the task offload services. For blockchain mining, edge devices need to minimize consensus delays to maintain the performance of the blockmining. Our optimization goal is therefore to maximize the common usage of the blockchain of all edge devices and minimize the offloading cost of all edge devices. The optimization problem is as follows:
Constraint conditions:/>
wherein ,Fm Is the maximum computing resource of server m; phi (phi) n The maximum computing resource for mining the blockchain for the edge device n;computing resources allocated for block chain mining for the edge equipment under the time slot t; t (T) n (t) is the processing delay of the edge device n in the time slot t; constraint (23 a) is a constraint on offloading decisions, constraint (23 b) is a constraint on transmission power, constraint (23 c) is a constraint on computing resources of the edge device, indicating that the edge device should allocate a positive computing resource to perform a computing task, but cannot exceed the resource budget; constraint (23 d) is a constraint on computing resources allocated by the MEC server for the task; constraint (23 e) indicates that the MEC server should allocate a positive computing resource for the task, but not exceed a maximum value; the constraint (23 f) is a constraint on computing resources used by the edge devices for consensus; the constraint (23 g) indicates that the processing time of the task cannot exceed a maximum value.
Since the above optimization problem is mixed integer non-convex, it is difficult to solve, in a dynamic computing offload scenario, the channel condition and the available computing resources of the edge devices and the MEC server are time-varying, and when the number of edge devices is gradually increased, the dimension of the system state space will become very large, and using the conventional optimization method will result in high computational complexity, and it is difficult to obtain an optimal offload policy and resource allocation policy. The present invention thus uses deep reinforcement learning to solve this problem.
3. Problem solving based on multi-agent deep reinforcement learning
The invention abstracts the optimization problem into a partially observable Markov decision process, takes edge equipment as an agent, defines the tuple { S, O, A, R } to describe the Markov game process, wherein S represents a global state space, and the environment of a time slot t is global state S (t) ∈S, O= { O 1 ,O 2 ,...,O N The observation space set of the intelligent agent is shown as a= { A 1 ,A 2 ,...,A N The motion space set of the agent is r= { R 1 ,R 2 ,...,R N And is a prize set. At time slot t, the agent observes o from local n (t)∈O n Adopts a policy pi n :O n →A n Selecting a corresponding action a n (t)∈A n Thereby obtaining corresponding rewards r n (t)∈R n 。
1. State space
At time slot t, the observed information of the individual agents includes the size of the computational task, the tolerable delay, the channel state, the resource state of the edge devices, and the state of the MEC server. The observation set for time slot t can thus be expressed as:
o n (t)＝{o task (t),o channel (t),o resource (t),o server (t)}
wherein ,otask (t)＝{C n (t),D n (t),τ n (t) } is the observation information of the task at time slot t, C n (t) is the computational resource required for the computational task of the edge device n at time slot t, D n (t) is the size of the computing task of the edge device n under the time slot t, τ n (t) is the maximum tolerable delay of the computing task of the edge device n under the time slot t; o (o) channel (t) is the observed information of the channel at time slot t,the resource state of the edge device n under the time slot t; o (o) server (t)＝{μ m (t),F m (t) } is the state information of the MEC server m observed by the edge device n at the time slot t, μ m (t) is the reliability of MEC server m in time slot t, F m And (t) is the available computing resources of MEC server m at time slot t.
O(t)＝{o 1 (t),o 2 (t),...o N (t) } is the observation set at time slot t, consisting of the states of all agents.
2. Action space
At time slot t, the actions of the single agent include offloading decisions, transmission power selection, computing resource allocation, consensus resource allocation. The set of actions can thus be expressed as:
wherein ,xn (t) is the offloading decision of edge device n at time slot t, p n (t) the edge device n offloads tasks to the transmission power selected by the server for the time slot t. A (t) = { a 1 (t),a 2 (t),...,a N (t) } is a set of actions at time slot t, consisting of actions of all agents.
3. Reward function
According to the optimization objective, define the system rewards as the sum of rewards of all agents at time slot t, and the rewards of agent n as r n (t), the system awards are:
4. MADDPG algorithm
In this section, it will be demonstrated how to solve the above-described problems using multi-agent deep reinforcement learning methods within the framework of centralized learning and distributed execution.
As shown in fig. 3, the framework of madddpg in the present invention is composed of an environment and N agents, each having a centralized training phase and a decentralized execution phase. In the training stage, centralized learning is adopted to train the critic network and the actor network, and state information of other intelligent agents is required to be used in the training of the critic network. In the execution stage, the actor only needs to know local information, and adjusts the local strategy according to the estimation strategies of other intelligent agents so as to achieve global optimum.
Let pi= { pi 1 ,π 2 ,...,π N And (2) strategy set of all agents, wherein θ= { θ 1 ,θ 2 ,...,θ N Each agent updates the parameter theta as the parameter set of the corresponding strategy n To obtain an optimal strategy.
In the decentralized execution stage, at each time slot t, the actor network of each agent observes the state o according to the local n (t) and its own policy pi n :O n →A n Selecting:
a n (t)＝π n o n (t)
during the centralized training phase, each critic network can obtain observations o of other agents n (t) and action a n (t), the Q function of agent n can be expressed as:
the Q function evaluates the action of the actor network from a global perspective and directs the actor network to select a more optimal action. During training, the critic network updates the network parameters by minimizing a loss function defined as follows:
wherein ,gamma is a discount factor; e (E) o,a,r,o' [·]The expression is expected when the observation space of the current agent is o, the observation space of the agent at the next time is o'. />
Meanwhile, the actor network updates network parameters based on the loss function calculated by the critic network and observation information of the actor network, and outputs an action a. The actor updates the network by calculating the gradient of the objective function:
wherein D is a set for experience replay; e (E) o,a-D [·]The expression is expected in the case where the observation space of the current agent is o and the distribution of the action space a of the agent is subject to the experience replay set D.
The parameters of the target network are updated in a soft update mode, namely:
wherein ,θn '、ω' n Is the updated policy parameter set and the agent policy set.
Pseudo code for the task offloading and resource allocation policy in MADDPG-based MEC system is as follows:
/>
wherein ,representing the pair expression with respect to policy parameter θ n Seeking a derivative; A≡B indicates that the value of B is given to A.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (9)
1. The mobile edge computing task unloading method based on the block chain is characterized by comprising the following steps of:
aiming at a dynamic MEC scene of multiple servers, a mobile edge computing task unloading model based on blockchain is established in consideration of the problems of security and privacy caused by the computation of the MEC servers, the time-varying communication resources and the interaction between heterogeneous edge nodes;
with the aim of minimizing the cost of a user for completing a calculation task and maximizing the utility obtained by the user in mining, a task unloading and resource allocation joint optimization model is established under the condition of multidimensional resource constraint;
the task unloading cost problem and the blockchain mining utility problem are abstracted into a partially observable Markov decision process in consideration of the random time-varying network environment and the partial observability of the environment state;
obtaining a state space and an action space of the MDP problem according to the established system model, and constructing a reward function;
and adopting a multi-agent reinforcement learning algorithm to make optimal unloading and resource allocation decisions.
2. The blockchain-based moving edge computing task offloading method of claim 1, wherein the blockchain-based moving edge computing task offloading model comprises:
When one edge device has a calculation task unloading requirement, sending an unloading requirement to an MEC server layer, after receiving a plurality of reply messages, searching the reliability of candidates by inquiring a reliability table of MEC servers stored in a blockchain, selecting an appropriate server for calculation unloading according to the reliability and channel conditions and available calculation resources of the servers, and periodically updating the reliability table according to verified transactions of calculation task unloading;
storing the reliability of each MEC server in a blockchain ledger, and inquiring the reliability from the blockchain when the edge equipment makes an unloading decision;
in the consensus process, the master node is responsible for generating the block, the duplicate node is responsible for verifying the block, and the non-selected edge equipment is used as a common node and is only responsible for adding the verified block into the maintained blockchain account book.
3. The blockchain-based mobile edge computing task offloading method of claim 2, wherein the blockchain consensus process comprises:
x CPU cycles are required for signing a block or transaction, y CPU cycles are required for verifying a signature, z CPU cycles are required for generating a MAC, and z CPU cycles are required for verifying a MAC;
The master node collects unverified transactions from all edge devices and then sorts the transactions according to the time stamp, assuming the block size is S b (t) the average size of the transactions is χ, then the number of transactions in a block is:at this stage, the master node needs to verify the signatures and MACs of the L transactions, so the computation cost of the master node is L (y+z);
the master node generates a signature and MAC for the block, N s -1 duplicate node generating a MAC for the pre-prepare message, each duplicate node verifying the MAC of the block, and the signature and MAC of the L transactions in the block, the master node calculating a cost of x+n s z, the computation cost of each duplicate node is z+L (y+z), and assuming that the transmission time of a block is proportional to the size of the block, the message transmission time is τ b S b ；
After the copy node verifies the pre-preparation information, the preparation information is sent to other consensus nodes, after each consensus node receives 2f preparation information, the next stage is entered, and the next stage is executedIn this stage, the master node needs to verify 2f MACs, so the calculation cost is 2fz; each replica node needs to generate N s 1 MAC and authentication 2f MAC, the computational cost of each duplicate node is (N s -1) z+2fz, message transmission time τ b S b ；
After receiving 2f preparation messages, each consensus node sends a commit message to other nodes, including the master node, at which stage the master node and duplicate nodes need to verify 2f MACs, yielding N s 1 MAC, so the computation cost of the master and replica nodes is (N s -1) z+2fz, message transmission time τ b S b ；
After receiving 2f commit messages, the master node and the replica node consider the block as legal, add the block to the blockchain ledger and send a reply message containing the verified block to other edge devices, update the global view after receiving f+1 reply messages, in which stage the master node and the replica node need to generate a MAC for the reply message, so that the calculation cost of the master node and the replica node is z, and the message transmission time is τ b S b 。
4. The blockchain-based mobile edge computing task offloading method of claim 1, wherein building a task offloading and resource allocation joint optimization model comprises:
constraint conditions:
wherein ,xn (t) represents a local observation of edge device n at time slot t; p is p n (t) represents the transmission power at which the edge device n offloads tasks to the MEC server m at time slot t;the method comprises the steps that computing resources allocated for executing unloading tasks of edge equipment n by an MEC server m under a time slot t are allocated; / >Computing resources allocated for executing tasks for edge device n under time slot t; />Computing resources allocated for block chain mining for the edge equipment under the time slot t; />The utility is the consensus of the edge device n under the time slot t; cost (t) is the Cost of the edge device under time slot t; x is x n (t) representsOffloading decision of edge device n at time slot t, x if the computation task is offloaded to MEC server i for execution at time slot t n (t) =i, x if the task is executed locally n (t)＝0；/>N is the number of the edge devices; p (P) n Is the maximum value of the transmission power; />Discretizing the system operation time into a set of T time slots; f (F) n Maximum computing resource for edge device n; f (F) m Is the maximum computing resource of server m; />Is a set of MEC servers; phi (phi) n The maximum computing resource for mining the blockchain for the edge device n; τ n Representing the maximum time delay tolerable for the task; t (T) n And (t) is the processing delay of the edge device n in the time slot t.
5. The blockchain-based mobile edge computing task offloading method of claim 1, wherein abstracting the task offloading cost problem and the blockchain mining utility problem into partially observable markov decision processes comprises:
The edge device acts as an agent and defines the tuple { S, O, A, R } describes the Markov game process, wherein S represents a global state space, and the environment of the time slot t is a global state S (t) ∈S, O= { O 1 ,O 2 ,...,O N [ is the observation space set of the agent, O n A value space corresponding to the observation space of the edge equipment n; a= { a 1 ,A 2 ,...,A N The motion space set of the intelligent agent is A n A value space corresponding to the action space of the intelligent body of the edge equipment n; r= { R 1 ,R 2 ,...,R N And R is the reward set n Is provided with edgePreparing a value space corresponding to the n rewards; at time slot t, agent N observes o from local n (t)∈O n Adopts a policy pi n :O n →A n Selecting a corresponding action a n (t)∈A n Thereby obtaining corresponding rewards r n (t)∈R n 。
6. The method for offloading mobile edge computing tasks based on blockchain as in claim 1, wherein the state space of the MDP problem includes the size of the computing task, tolerable delay, channel state, resource state of the edge device and state of the MEC server of the observation information of the single agent of the time slot t, and the state space of the time slot t is expressed as:
o n (t)＝{o task (t),o channel (t),o resource (t),o server (t)}
wherein ,otask (t)＝{C n (t),D n (t),τ n (t) } is the observation information of the task at time slot t, C n (t) is the computational resource required for the computational task at time slot t, D n (t) is the size of the calculation task at time slot t, τ n (t) is the maximum tolerable delay of the computing task at time slot t; o (o) channel (t) is the observed information of the channel at time slot t,resource status for edge device n under time slot t, < >>Computing resources allocated to the edge device n for executing tasks at the time slot t +.>Computing resources allocated for block chain mining for the edge device n under the time slot t; o (o) server (t)＝{μ m (t),F m (t) } is the MEC server status information observed by the edge device n at time slot t, μ m (t) MEC server m at time slot tReliability, F m And (t) is the available computing resources of MEC server m at time slot t.
7. The method of claim 1, wherein the action space of the MDP problem includes actions of a single agent of time slot t including unloading decision, transmission power selection, computing resource allocation, consensus resource allocation, and the set of actions of time slot t is expressed as:
wherein ,xn (t) is the offloading decision of edge device n at time slot t, p n (t) offloading tasks to the transmission power selected by the server for time slot t;computing resources allocated to the edge device n for executing tasks at the time slot t +.>The computing resources allocated for the blockchain mining for the edge device n under time slot t.
8. The blockchain-based moving edge computing task offloading method of claim 1, wherein the reward function is expressed as:
wherein r (t) represents a bonus function value at time slot t; r is (r) n (t) represents a prize function value of the edge device n at the time slot t; n is the number of edge devices;for time slot t lower edge device nIs a common recognition utility of (a); cost (t) is the Cost of the edge device at the bottom of slot t.
9. The mobile edge computing task unloading method based on the blockchain as in claim 1, wherein an optimal unloading and resource allocation decision is made by adopting a multi-agent reinforcement learning algorithm, wherein the multi-agent reinforcement learning algorithm consists of an environment and N agents, each agent has a centralized training stage and a decentralized execution stage, and in the training stage, a critic network and an actor network are trained by adopting centralized learning, and state information of other agents is needed to be used when the critic network is trained; in the decentralized execution stage, the actor only needs to know local information, and adjusts the local strategy according to the estimation strategy of other intelligent agents so as to achieve global optimum, and the method specifically comprises the following steps:
let pi= { pi 1 ,π 2 ,...,π N And (2) strategy set of all agents, wherein θ= { θ 1 ,θ 2 ,...,θ N Each agent updates the parameter theta as the parameter set of the corresponding strategy n To obtain an optimal strategy;
in the decentralized execution stage, at each time slot t, the actor network of each agent observes the state o according to the local n (t) and its own policy selection actions, expressed as:
a n (t)＝π n o n (t)
during the centralized training phase, each critic network can obtain observations o of other agents n (t) and action a n (t), the Q function of agent n can be expressed as:
the Q function evaluates the action of the actor network from the global angle and guides the actor network to select a better action;
in training, the critic network updates the network parameters by minimizing a loss function, expressed as:
the actor network updates network parameters based on the loss function calculated by the critic network and the observation information of the actor network, and outputs actions; the actor updates the network by calculating the gradient of the objective function:
the parameters of the target network are updated in a soft update mode, namely:
wherein ,gamma is a discount factor; e (E) o,a,r,o' [·]Representing the expectation of the expression, wherein o is an observation set, a is an action set, r is a reward set, and o' is an observation set of the next time slot; r is (r) n Rewards for edge device n under time slot t; o' n (t) is the observation of the next slot agent n for slot t; a' n ＝π n (o n ) To be o in the observation set n When according to policy pi n Selected action, pi n (. Cndot.) represents the policy of agent n; e (E) o,a～D [·]Representing the desire for the expression +.>Representing the parameter θ for the expression with respect to the current network of the actor n Gradient determination，π n (a n |o n ) For intelligent body observe o n Action a is made downwards n Is a policy of (2); />For the soft update coefficient of the actor network, theta n Is the parameter of the current network of the actor, theta n ' is a parameter of the actor target network, +.>For soft update coefficients, ω, of critic networks n Omega 'is a parameter of the critic current network' n Is a parameter of the critic target network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310802546.8A CN116669111A (en) | 2023-07-03 | 2023-07-03 | Mobile edge computing task unloading method based on blockchain |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310802546.8A CN116669111A (en) | 2023-07-03 | 2023-07-03 | Mobile edge computing task unloading method based on blockchain |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116669111A true CN116669111A (en) | 2023-08-29 |
Family
ID=87719213
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310802546.8A Pending CN116669111A (en) | 2023-07-03 | 2023-07-03 | Mobile edge computing task unloading method based on blockchain |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116669111A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117202173A (en) * | 2023-11-07 | 2023-12-08 | 中博信息技术研究院有限公司 | Edge computing unloading method for user privacy protection |
| CN117499491A (en) * | 2023-12-27 | 2024-02-02 | 杭州海康威视数字技术股份有限公司 | Internet of things service arrangement method and device based on double-agent deep reinforcement learning |
-
2023
- 2023-07-03 CN CN202310802546.8A patent/CN116669111A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117202173A (en) * | 2023-11-07 | 2023-12-08 | 中博信息技术研究院有限公司 | Edge computing unloading method for user privacy protection |
| CN117499491A (en) * | 2023-12-27 | 2024-02-02 | 杭州海康威视数字技术股份有限公司 | Internet of things service arrangement method and device based on double-agent deep reinforcement learning |
| CN117499491B (en) * | 2023-12-27 | 2024-03-26 | 杭州海康威视数字技术股份有限公司 | Internet of things service arrangement method and device based on double-agent deep reinforcement learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Lu et al. | Low-latency federated learning and blockchain for edge association in digital twin empowered 6G networks | |
| Lu et al. | Blockchain empowered asynchronous federated learning for secure data sharing in internet of vehicles | |
| Lu et al. | Differentially private asynchronous federated learning for mobile edge computing in urban informatics | |
| Lin et al. | Task offloading for wireless VR-enabled medical treatment with blockchain security using collective reinforcement learning | |
| Xu et al. | A blockchain-enabled energy-efficient data collection system for UAV-assisted IoT | |
| CN111132175B (en) | Cooperative computing unloading and resource allocation method and application | |
| Shi et al. | DRL-based V2V computation offloading for blockchain-enabled vehicular networks | |
| Asheralieva et al. | Reputation-based coalition formation for secure self-organized and scalable sharding in iot blockchains with mobile-edge computing | |
| CN116669111A (en) | Mobile edge computing task unloading method based on blockchain | |
| Kong et al. | A reliable and efficient task offloading strategy based on multifeedback trust mechanism for IoT edge computing | |
| Yang et al. | Sharded blockchain for collaborative computing in the Internet of Things: Combined of dynamic clustering and deep reinforcement learning approach | |
| Fu et al. | Performance optimization for blockchain-enabled distributed network function virtualization management and orchestration | |
| US20220067063A1 (en) | Apparatus and method for adaptively managing sharded blockchain network based on deep q network | |
| CN114626547A (en) | Group collaborative learning method based on block chain | |
| CN112637822B (en) | Cellular network safe transaction platform based on block chain | |
| CN112783662A (en) | CPU resource trusted sharing system in sensing edge cloud task unloading of integrated block chain | |
| Halgamuge et al. | Trust model to minimize the influence of malicious attacks in sharding based blockchain networks | |
| CN114301935A (en) | Reputation-based method for selecting edge cloud collaborative federated learning nodes of Internet of things | |
| Qiu et al. | Blockchain and K-means algorithm for edge AI computing | |
| Nguyen et al. | Intelligent blockchain-based edge computing via deep reinforcement learning: solutions and challenges | |
| Al Ridhawi et al. | An intelligent blockchain-assisted cooperative framework for industry 4.0 service management | |
| CN110061863B (en) | Distributed multi-task crowd-sourcing sensing method based on fairness in sparse network | |
| Wang et al. | PoTA: A hybrid consensus protocol to avoid miners’ collusion for BaaS platform | |
| Qi et al. | STFM: a blockchain sharding algorithm based on trust field model for heterogeneous Internet of Things | |
| Zhu et al. | ESFL: Efficient Split Federated Learning over Resource-Constrained Heterogeneous Wireless Devices |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |