CN113590232B - Relay edge network task unloading method based on digital twinning - Google Patents

Relay edge network task unloading method based on digital twinning Download PDF

Info

Publication number
CN113590232B
CN113590232B CN202110965259.XA CN202110965259A CN113590232B CN 113590232 B CN113590232 B CN 113590232B CN 202110965259 A CN202110965259 A CN 202110965259A CN 113590232 B CN113590232 B CN 113590232B
Authority
CN
China
Prior art keywords
task
user terminal
model
relay node
edge server
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110965259.XA
Other languages
Chinese (zh)
Other versions
CN113590232A (en
Inventor
徐江
柏基成
李斌
谈昊哲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Information Science and Technology
Original Assignee
Nanjing University of Information Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Information Science and Technology filed Critical Nanjing University of Information Science and Technology
Priority to CN202110965259.XA priority Critical patent/CN113590232B/en
Publication of CN113590232A publication Critical patent/CN113590232A/en
Application granted granted Critical
Publication of CN113590232B publication Critical patent/CN113590232B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/445—Program loading or initiating
    • G06F9/44594—Unloading
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5072—Grid computing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/509—Offload
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention discloses a relay edge network task unloading method based on digital twinning, which comprises the steps of constructing a relay edge network task unloading strategy model; updating the state of each corresponding part in the digital twin body environment; transmitting the parameters of the digital twin bodies into an analog task unloading system for iterative training to obtain an optimal task unloading strategy model; transmitting the optimal task unloading strategy model to a simulation manual control interface for backup; transmitting the current digital twin parameter training model and the optimal task unloading strategy model to a digital twin environment cache, and forwarding the digital twin environment cache to each relay node by an edge server in reality, and forwarding the relay node to a user terminal in communication with the relay node; and the user terminal and the relay node perform corresponding task unloading according to the optimal task unloading strategy model. The invention can reduce the trial-and-error cost of the real 5G edge computing technology in the landing process and improve the landing efficiency.

Description

Relay edge network task unloading method based on digital twinning
Technical Field
The invention belongs to the technical field of mobile edge calculation, and particularly relates to a relay edge network task unloading method based on digital twinning.
Background
With the rapid development of 5G and industrial internet, the demand for edge computing is increasing, and the fields of intelligent manufacturing, smart city, internet of vehicles, cloud games and the like all provide requirements for edge computing services.
At present, edge computing technology test points are mostly carried out on 4G or early 5G networks, but the development of the edge computing technology and the deployment of edge servers are limited by the ecology of limited resources and fragmentation at present, so that most users still cannot directly enjoy the services of the edge computing technology. These users may apply to offload tasks that cannot be calculated locally in time to an edge server for calculation. However, the communication link cannot be established directly with the edge server due to factors such as being too far away or being blocked by an obstacle such as a building.
It is an unavoidable challenge to reasonably allocate edge computing resources in the face of differentiated user demands and terminal devices of varying performance. In the pilot process, the change of the resource allocation policy can have a considerable impact on the edge server and the end user in reality.
Most of the existing edge computing related technologies directly assume that states of an edge server and end user equipment are known to perform decision optimization, so that energy consumption and time delay are reduced. However, the optimal solution is not necessarily achieved for more complex reality situations.
The digital twin technology can fully utilize data such as a physical model, a sensor, an operation history and the like, integrate multidisciplinary and multiscale simulation processes, construct images of entities in a virtual space, reflect the full life cycle process of the corresponding physical entities, and is very suitable for adapting and landing the reality situation by the aid of an edge computing technology in the current stage.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a relay edge network task unloading method based on digital twinning, which helps a user terminal to carry out task unloading, obtains good effect within an acceptable cost range, and helps an edge computing technology at the current stage to adapt to the actual situation and land.
In order to achieve the technical purpose, the invention adopts the following technical scheme:
a relay edge network task offloading method based on digital twinning includes:
step (1): building a relay edge network task offloading policy model, comprising: the system comprises a physical communication unloading environment, a digital twin environment, an analog task unloading system and an analog manual control interface;
the physical communication offload environment includes: a physical edge server, a relay node and a user terminal set;
the digital twin environment is an environment constructed by aggregating digital twin parameter models obtained by training a relay node and a user terminal by an edge server, and comprises the following steps: the state of the edge server, the state of each relay node and the state of the user terminal;
the simulated task offloading system includes: an artificial intelligent algorithm model library corresponding to each unloading condition, an unloading strategy selection module based on an DQN algorithm and a task unloading strategy model cache module;
the simulated manual control interface is a virtual control environment constructed by an edge server through virtual and real information transmission with a real manual control interface, and a digital twin parameter training model and a task unloading strategy model which are really used are determined;
step (2): the physical entity updates the states of the corresponding parts in the digital twin body environment through the digital twin parameter model;
step (3): the digital twin body environment transmits the parameters of the digital twin body into the simulation task unloading system for iterative training to obtain an optimal task unloading strategy model;
step (4): transmitting the optimal task unloading strategy model to a simulation manual control interface for backup;
step (5): the simulation manual control interface transmits the current digital twin parameter training model and the optimal task unloading strategy model to the digital twin environment cache, and the current digital twin parameter training model and the optimal task unloading strategy model are forwarded to each relay node by an edge server in reality, and the relay nodes are forwarded to a user terminal in communication with the relay nodes;
step (6): and the user terminal and the relay node perform corresponding task unloading according to the optimal task unloading strategy model.
In order to optimize the technical scheme, the specific measures adopted further comprise:
the user terminal comprises a smart phone, a notebook computer, a mobile tablet and other devices;
the user terminal is out of coverage of the edge server.
The states of the edge server include the processor frequency, the available memory capacity, the available channels and the working state of the edge server;
the state of the relay node comprises the processor frequency, transmission power, available channels and working state of the relay node;
the state of the user terminal includes the processor frequency of the user terminal, the transmission power, the data size and computational complexity of the task, the task type, the device type, the remaining energy.
The step (2) includes:
firstly, a user terminal trains a digital twin parameter model in a local iteration mode, and the trained digital twin parameter model is transmitted to a relay node together when a task is unloaded;
the relay node packages and transmits the digital twin parameter model trained by the user terminal and the relay node to the edge server, the edge server aggregates the models of the user terminal and the relay node while training the edge server model, and the state of the corresponding part in the digital twin environment is updated after training is completed.
Step (3) above, the training process in the simulation task offloading system includes:
step (3.1): and (3) calling an artificial intelligence algorithm model library to obtain: the task is locally calculated at the user terminal without task unloading, task unloading is carried out to the relay node for calculation, task unloading is carried out to the edge server for calculation, and the optimal task cost of each of the three conditions is transmitted into the unloading strategy selection module;
step (3.2): the unloading strategy selection module firstly synthesizes the optimal task cost of the three conditions in the step (3.1) into final unloading cost, combines the state parameters of the digital twin body environment, establishes an optimal model for minimizing the final cost, and trains by using a DQN algorithm to obtain an optimal task unloading strategy model;
step (3.3): detecting and evaluating an optimal task unloading strategy model by using historical data of a digital twin environment, and temporarily storing the model and the score to a task unloading model cache module;
step (3.4): and (3.1) repeating the step (3.3) until the score meets the standard or training is finished, and obtaining a final optimal task unloading strategy model.
In the step (3.1), the task is locally calculated at the user end, and when the task is not unloaded, the optimal task cost obtained by calling the related artificial intelligence algorithm is recorded as
Figure GDA0004137950330000031
In the step (3.1), when the task is unloaded to the relay node for calculation, the corresponding optimal task cost calculation method is as follows:
the transmission rate between the relay node j and the user terminal i is noted as
Figure GDA0004137950330000032
The transmission delay is recorded as->
Figure GDA0004137950330000033
The energy consumption during transmission is +.>
Figure GDA0004137950330000034
The time required for the task to calculate at relay node j is noted +.>
Figure GDA0004137950330000035
The time required by the relay node j to calculate the digital twin parameter training model and package is as follows:
Figure GDA0004137950330000036
wherein ,
Figure GDA0004137950330000037
for the CPU frequency of the relay node j, D j For the local data set of the relay node j, H is the number of user terminals communicating with the relay node j, and the digital twin parameter model of the relay node j is obtained after training and packaging>
Figure GDA0004137950330000038
The energy consumption of the relay node is ignored;
when the task is unloaded to the relay node for calculation, the corresponding optimal task cost is as follows:
Figure GDA0004137950330000039
calling related artificial intelligence algorithm to obtain optimal cost
Figure GDA00041379503300000310
In the step (3.1), when the task is unloaded to the edge server for calculation, the corresponding optimal task cost calculation method is as follows:
the signal of the user terminal i which can be directly received by the edge server is:
Figure GDA0004137950330000041
wherein ,
Figure GDA0004137950330000042
is the channel between the edge server and the user terminal i,/i>
Figure GDA0004137950330000043
Is a noise signal between the edge server and the user terminal i. The edge server receives the signal of the user terminal i in an auxiliary way through the relay node j as follows:
Figure GDA0004137950330000044
wherein ,
Figure GDA0004137950330000045
is the transmission power of relay node j, +.>
Figure GDA0004137950330000046
Is the auxiliary channel that the relay node j distributes to the user terminal i to the edge server, +.>
Figure GDA0004137950330000047
Is the noise signal at the edge server on the corresponding channel,/or->
Figure GDA0004137950330000048
Is a normalization parameter;
the signal to noise ratio of the user terminal i is obtained by maximum ratio amplitude synthesis at the edge server as follows:
Figure GDA0004137950330000049
wherein ,Pi UT Is the transmission power of the user terminal i,
Figure GDA00041379503300000410
is the channel between the relay node j and the user terminal i;
the transmission rate between the user terminal i and the edge server is:
Figure GDA00041379503300000411
wherein ,Wi Is the bandwidth between the edge server and the user terminal i. The transmission delay is recorded as
Figure GDA00041379503300000412
The transmission energy consumption is recorded as
Figure GDA00041379503300000413
The time required for the task to calculate at the edge server is recorded as +.>
Figure GDA00041379503300000414
The time required by the edge server to calculate the digital twin parameter training model and aggregate the digital twin parameter models of the relay node and the user terminal is as follows:
Figure GDA00041379503300000415
wherein ,fECS CPU frequency of edge server, D ECS The method comprises the steps that N is the number of relay nodes communicated with an edge server and is a local data set of the edge server;
due to
Figure GDA00041379503300000416
Is small, and f ECS Very high, compared to the time taken to train the edge server model, the time taken to aggregate the relay node model +.>
Figure GDA0004137950330000051
Negligible;
obtaining the digital twin parameter model of the edge server after training
Figure GDA0004137950330000052
The energy consumption of the edge server is ignored, and when the task is unloaded to the edge server for calculation, the corresponding optimal task cost is as follows:
Figure GDA0004137950330000053
calling related artificial intelligence algorithm to obtain optimal cost
Figure GDA0004137950330000054
In the step (3.2), the unloading policy selection module builds an optimization model that minimizes the final cost:
the final offload cost for user terminal i in three cases of step (3.1) is expressed as:
Figure GDA0004137950330000055
wherein ,ai E {0,1}, i=1, 2,3 and
Figure GDA0004137950330000056
for each calculation task of the user terminal i, selecting which case to calculate can minimize the final cost, and an optimization model for minimizing the final cost is as follows:
Figure GDA0004137950330000057
Figure GDA0004137950330000058
Figure GDA0004137950330000059
Figure GDA00041379503300000510
wherein ,
Figure GDA00041379503300000511
Figure GDA00041379503300000512
is the expected energy consumption threshold of the user terminal.
In step (3.2) above, the present invention uses DQN as a framework for the DRL algorithm.
In the training process, the unloading strategy selection module interacts with the digital twin body environment to obtain the state of each iteration t task unloading system:
Figure GDA00041379503300000513
the action of the learning Agent is expressed as:
A t ＝{a t |a t ∈I t }
wherein at Is from a set of possible decision actions I t The selected action;
the bonus function reflects that the selected action is in the system state s t Is expressed as:
Figure GDA0004137950330000061
/>
wherein ψ is a guaranteed R t A fixed parameter that is positive, λ is learning rate, μ i (t) is the final cost at iteration t;
and approximating the optimal action cost function by using a neural network Q (s, a; w) in combination with a time difference algorithm to obtain an optimal task unloading strategy model, and transmitting the optimal task unloading strategy model into a task unloading model buffer module.
The invention has the following beneficial effects:
(1) The invention adopts a digital twin method to carry out the unloading decision of the simulation task, so that the trial-and-error cost of the real 5G edge computing technology in the process of landing can be reduced to a great extent; helping a user obtain a result within an acceptable cost range under the condition of limited edge computing resources at the present stage;
(2) Compared with other task unloading methods, the digital twin environment provided by the invention is updated along with the change of physical entities, so that an unloading strategy model obtained by a simulated task unloading decision system is more close to the real situation, and the landing efficiency is improved.
Drawings
FIG. 1 is a schematic diagram of a physical communication offload environment of the present invention;
FIG. 2 is a frame structure diagram of a relay edge network task offloading method based on digital twinning;
FIG. 3 is a workflow diagram of a task offloading policy model.
Detailed Description
Embodiments of the present invention are described in further detail below with reference to the accompanying drawings.
Referring to fig. 1 and 2, a relay edge network task offloading method based on digital twinning of the present invention includes:
step (1): building a relay edge network task offloading policy model, comprising: the system comprises a physical communication unloading environment, a digital twin environment, an analog task unloading system and an analog manual control interface;
the physical communication offload environment includes: a physical edge server, a relay node and a user terminal set;
the digital twin environment is an environment constructed by aggregating digital twin parameter models obtained by training a relay node and a user terminal by an edge server, and comprises the following steps: the state of the edge server, the state of each relay node and the state of the user terminal;
the simulated task offloading system includes: an artificial intelligent algorithm model library corresponding to each unloading condition, an unloading strategy selection module based on an DQN algorithm and a task unloading strategy model cache module;
the simulated manual control interface is a virtual control environment constructed by an edge server through virtual and real information transmission with a real manual control interface, and a digital twin parameter training model and a task unloading strategy model which are really used are determined;
in an embodiment, the user terminal includes a smart phone, a notebook computer, a mobile tablet, and other devices;
the user terminal is out of coverage of the edge server.
The state of the edge server comprises the processor frequency, the available memory capacity, the available channels and the working state of the edge server;
the state of the relay node comprises the processor frequency, transmission power, available channels and working state of the relay node;
the state of the user terminal includes the processor frequency of the user terminal, the transmission power, the data size and computational complexity of the task, the task type, the device type, the remaining energy.
Step (2): the physical entity updates the states of the corresponding parts in the digital twin body environment through the digital twin parameter model;
step (3): the digital twin body environment transmits the parameters of the digital twin body into the simulation task unloading system for iterative training to obtain an optimal task unloading strategy model;
step (4): transmitting the optimal task unloading strategy model to a simulation manual control interface for backup;
step (5): the simulation manual control interface trains the model m with the current digital twin parameters ECS 、m RN 、m UT Optimal task offloading policy model um RN 、um UT Transmitting to the digital twin body environment cache, and forwarding to each relay node by the edge server in reality, and forwarding the relay node to the user terminal in communication with the relay node;
step (6): and the user terminal and the relay node perform corresponding task unloading according to the optimal task unloading strategy model.
In an embodiment, the step (2) includes:
firstly, a user terminal trains a digital twin parameter model in a local iteration mode, and the trained digital twin parameter model is transmitted to a relay node together when a task is unloaded;
the relay node packages and transmits the digital twin parameter model trained by the user terminal and the relay node to the edge server, the edge server aggregates the models of the user terminal and the relay node while training the edge server model, and the state of the corresponding part in the digital twin environment is updated after training is completed.
In an embodiment, the step (3) includes simulating a training process in the task offloading system including:
step (3.1): and (3) calling an artificial intelligence algorithm model library to obtain: the task is locally calculated at the user terminal without task unloading, task unloading is carried out to the relay node for calculation, task unloading is carried out to the edge server for calculation, and the optimal task cost of each of the three conditions is transmitted into the unloading strategy selection module;
step (3.2): the unloading strategy selection module firstly synthesizes the optimal task cost of the three conditions in the step (3.1) into final unloading cost, combines the state parameters of the digital twin body environment, establishes an optimal model for minimizing the final cost, and trains by using a DQN algorithm to obtain an optimal task unloading strategy model;
step (3.3): detecting and evaluating an optimal task unloading strategy model by using historical data of a digital twin environment, and temporarily storing the model and the score to a task unloading model cache module;
step (3.4): and (3.1) repeating the step (3.3) until the score meets the standard or training is finished, and obtaining a final optimal task unloading strategy model.
In an example, step (3.1): and (5) calling an artificial intelligent algorithm model library to obtain the optimal task cost corresponding to the three conditions.
Case one: when the task is locally calculated at the user side and task unloading is not performed, the corresponding optimal task cost calculating method comprises the following steps:
for each user terminal i, the task to be calculated is denoted as T i ＝(C i ,L i), wherein Ci Is the computational complexity of the task, L i Is the data size of the task;
the time required for the task to calculate locally is noted as
Figure GDA0004137950330000081
The energy consumption required is +.>
Figure GDA0004137950330000082
The time required by the user terminal i to calculate the digital twin parameter training model is as follows:
Figure GDA0004137950330000083
wherein ,fi UT For CPU frequency of user terminal i, D i The local data set of the user terminal i is trained to obtain a digital twin parameter model of the user terminal i
Figure GDA0004137950330000084
The energy consumption required for this process is +.>
Figure GDA0004137950330000085
The total cost required by the user terminal i to complete the task is:
Figure GDA0004137950330000086
/>
wherein ,αi E (0, 1) and beta i E (0, 1) is a weight coefficient of time delay and energy consumption determined based on task type and equipment type; calling related artificial intelligence algorithm to obtain optimal task cost
Figure GDA0004137950330000087
And a second case: when the task is unloaded to the relay node for calculation, the corresponding optimal task cost calculation method comprises the following steps:
the transmission rate between the relay node j and the user terminal i is noted as
Figure GDA0004137950330000088
The transmission delay is recorded as->
Figure GDA0004137950330000089
The energy consumption during transmission is +.>
Figure GDA00041379503300000810
The time required for the task to calculate at relay node j is noted +.>
Figure GDA00041379503300000811
The time required by the relay node j to calculate the digital twin parameter training model and package is as follows:
Figure GDA00041379503300000812
wherein ,
Figure GDA0004137950330000091
for the CPU frequency of the relay node j, D j For the local data set of the relay node j, H is the number of user terminals communicating with the relay node j, and the digital twin parameter model of the relay node j is obtained after training and packaging>
Figure GDA0004137950330000092
The energy consumption of the relay node is ignored;
when the task is unloaded to the relay node for calculation, the corresponding optimal task cost is as follows:
Figure GDA0004137950330000093
calling related artificial intelligence algorithm to obtain optimal cost
Figure GDA0004137950330000094
And a third case: when the task is unloaded to the edge server for calculation, the corresponding optimal task cost calculation method comprises the following steps:
the signal of the user terminal i which can be directly received by the edge server is:
Figure GDA0004137950330000095
wherein ,
Figure GDA0004137950330000096
is the channel between the edge server and the user terminal i,/i>
Figure GDA0004137950330000097
Is a noise signal between the edge server and the user terminal i. The edge server receives the signal of the user terminal i in an auxiliary way through the relay node j as follows:
Figure GDA0004137950330000098
wherein ,
Figure GDA0004137950330000099
is the transmission power of relay node j, +.>
Figure GDA00041379503300000910
Is the auxiliary channel that the relay node j distributes to the user terminal i to the edge server, +.>
Figure GDA00041379503300000911
Is the noise signal at the edge server on the corresponding channel,/or->
Figure GDA00041379503300000912
Is a normalization parameter;
the signal to noise ratio of the user terminal i is obtained by maximum ratio amplitude synthesis at the edge server as follows:
Figure GDA00041379503300000913
wherein ,Pi UT Is the transmission power of the user terminal i,
Figure GDA00041379503300000914
is the channel between the relay node j and the user terminal i;
the transmission rate between the user terminal i and the edge server is:
Figure GDA00041379503300000915
wherein ,Wi Is the bandwidth between the edge server and the user terminal i. The transmission delay is recorded as
Figure GDA00041379503300000916
The transmission energy consumption is->
Figure GDA00041379503300000917
The time required for the task to calculate at the edge server is recorded as +.>
Figure GDA0004137950330000101
The time required by the edge server to calculate the digital twin parameter training model and aggregate the digital twin parameter models of the relay node and the user terminal is as follows:
Figure GDA0004137950330000102
wherein ,fECS CPU frequency of edge server, D ECS The method comprises the steps that N is the number of relay nodes communicated with an edge server and is a local data set of the edge server;
due to
Figure GDA0004137950330000103
Is small, and f ECS Very high, compared to the time taken to train the edge server model, the time taken to aggregate the relay node model +.>
Figure GDA0004137950330000104
Negligible;
obtaining the digital twin parameter model of the edge server after training
Figure GDA0004137950330000105
The energy consumption of the edge server is ignored, and when the task is unloaded to the edge server for calculation, the corresponding optimal task cost is as follows:
Figure GDA0004137950330000106
calling related artificial intelligence algorithm to obtain optimal cost
Figure GDA0004137950330000107
In an embodiment, in the step (3.2), the unloading policy selection module builds an optimization model that minimizes the final cost:
the final offload cost for user terminal i in three cases of step (3.1) is expressed as:
Figure GDA0004137950330000108
wherein ,ai E {0,1}, i=1, 2,3 and
Figure GDA0004137950330000109
for each calculation task of the user terminal i, selecting which case to calculate can minimize the final cost, and an optimization model for minimizing the final cost is as follows:
Figure GDA00041379503300001010
Figure GDA00041379503300001011
Figure GDA00041379503300001012
Figure GDA00041379503300001013
wherein ,
Figure GDA00041379503300001014
Figure GDA00041379503300001015
is the expected energy consumption threshold of the user terminal.
In said step (3.2), the present invention uses DQN as a framework for the DRL algorithm.
In the training process, the unloading strategy selection module interacts with the digital twin body environment to obtain the state of each iteration t task unloading system:
Figure GDA0004137950330000111
the action of the learning Agent is expressed as:
A t ＝{a t |a t ∈I t }
wherein at Is from a set of possible decision actions I t The selected action;
the bonus function reflects that the selected action is in the system state s t Is expressed as:
Figure GDA0004137950330000112
wherein ψ is a guaranteed R t A fixed parameter that is positive, λ is learning rate, μ i (t) is the final cost at iteration t;
and approximating the optimal action cost function by using a neural network Q (s, a; w) in combination with a time difference algorithm to obtain an optimal task unloading strategy model, and transmitting the optimal task unloading strategy model into a task unloading model buffer module.
Fig. 3 specifically illustrates a process of executing a task offloading policy model by a user terminal and a relay node, where the user terminal offloading policy model determines a final object that is responsible for a computing task, and executes a corresponding optimization target policy according to a task type during local computing, and the offloading policy model of the relay node and an edge server executes the corresponding optimization target policy according to the task type.
The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above examples, and all technical solutions belonging to the concept of the present invention belong to the protection scope of the present invention. It should be noted that modifications and adaptations to the invention without departing from the principles thereof are intended to be within the scope of the invention as set forth in the following claims.

Claims (3)

1. The relay edge network task offloading method based on digital twinning is characterized by comprising the following steps of:
step (1): building a relay edge network task offloading policy model, comprising: the system comprises a physical communication unloading environment, a digital twin environment, an analog task unloading system and an analog manual control interface;
the physical communication offload environment includes: a physical edge server, a relay node and a user terminal set;
the digital twin environment is an environment constructed by aggregating digital twin parameter models obtained by training a relay node and a user terminal by an edge server, and comprises the following steps: the state of the edge server, the state of each relay node and the state of the user terminal;
the simulated task offloading system includes: an artificial intelligent algorithm model library corresponding to each unloading condition, an unloading strategy selection module based on an DQN algorithm and a task unloading strategy model cache module;
the simulated manual control interface is a virtual control environment constructed by an edge server through virtual and real information transmission with a real manual control interface, and a digital twin parameter training model and a task unloading strategy model which are really used are determined;
step (2): the physical entity updates the states of the corresponding parts in the digital twin body environment through the digital twin parameter model;
step (3): the digital twin body environment transmits the parameters of the digital twin body into the simulation task unloading system for iterative training to obtain an optimal task unloading strategy model;
step (4): transmitting the optimal task unloading strategy model to a simulation manual control interface for backup;
step (5): the simulation manual control interface transmits the current digital twin parameter training model and the optimal task unloading strategy model to the digital twin environment cache, and the current digital twin parameter training model and the optimal task unloading strategy model are forwarded to each relay node by an edge server in reality, and the relay nodes are forwarded to a user terminal in communication with the relay nodes;
step (6): the user terminal and the relay node carry out corresponding task unloading according to the optimal task unloading strategy model;
the step (2) comprises:
firstly, a user terminal trains a digital twin parameter model in a local iteration mode, and the trained digital twin parameter model is transmitted to a relay node together when a task is unloaded;
the relay node packages and transmits the digital twin parameter model trained by the user terminal and the relay node to the edge server, the edge server aggregates the models of the user terminal and the relay node while training the edge server model, and the state of the corresponding part in the digital twin environment is updated after training is completed;
the step (3) is to simulate the training process in the task unloading system, which comprises the following steps:
step (3.1): and (3) calling an artificial intelligence algorithm model library to obtain: the task is locally calculated at the user terminal without task unloading, task unloading is carried out to the relay node for calculation, task unloading is carried out to the edge server for calculation, and the optimal task cost of each of the three conditions is transmitted into the unloading strategy selection module;
step (3.2): the unloading strategy selection module firstly synthesizes the optimal task cost of the three conditions in the step (3.1) into final unloading cost, combines the state parameters of the digital twin body environment, establishes an optimal model for minimizing the final cost, and trains by using a DQN algorithm to obtain an optimal task unloading strategy model;
step (3.3): detecting and evaluating an optimal task unloading strategy model by using historical data of a digital twin environment, and temporarily storing the model and the score to a task unloading model cache module;
step (3.4): repeating the steps (3.1) - (3.3) until the score meets the standard or training is finished, and obtaining a final optimal task unloading strategy model;
in the step (3.1), the task is locally calculated at the user terminal, and when the task is not unloaded, the corresponding optimal task cost calculating method comprises the following steps:
for each user terminal i, the task to be calculated is denoted as T i ＝(C i ,L i), wherein Ci Is the computational complexity of the task, L i Is the data size of the task;
the time required for the task to calculate locally is noted as
Figure FDA0004137950320000021
The energy consumption required is +.>
Figure FDA0004137950320000022
The time required by the user terminal i to calculate the digital twin parameter training model is as follows:
Figure FDA0004137950320000023
wherein ,fi UT For CPU frequency of user terminal i, D i The local data set of the user terminal i is trained to obtain a digital twin parameter model of the user terminal i
Figure FDA0004137950320000024
The energy consumption required for this process is +.>
Figure FDA0004137950320000025
The total cost required by the user terminal i to complete the task is:
Figure FDA0004137950320000026
wherein ,αi E (0, 1) and beta i E (0, 1) is a weight coefficient of time delay and energy consumption determined based on task type and equipment type;
invoking artificial intelligence algorithm to obtain optimal task cost
Figure FDA0004137950320000027
In the step (3.1), when the task is unloaded to the relay node for calculation, the corresponding optimal task cost calculation method comprises the following steps:
the transmission rate between the relay node j and the user terminal i is noted as
Figure FDA0004137950320000028
The transmission delay is recorded as->
Figure FDA0004137950320000029
The energy consumption during transmission is +.>
Figure FDA00041379503200000210
The time required for the task to calculate at relay node j is noted +.>
Figure FDA00041379503200000211
The time required by the relay node j to calculate the digital twin parameter training model and package is as follows:
Figure FDA00041379503200000212
wherein ,
Figure FDA0004137950320000031
for the CPU frequency of the relay node j, D j For the local data set of the relay node j, H is the number of user terminals communicating with the relay node j, and the digital twin parameter model of the relay node j is obtained after training and packaging>
Figure FDA0004137950320000032
The energy consumption of the relay node is ignored;
when the task is unloaded to the relay node for calculation, the corresponding optimal task cost is as follows:
Figure FDA0004137950320000033
invoking artificial intelligence algorithm to obtain optimal cost
Figure FDA0004137950320000034
In the step (3.1), when the task is unloaded to the edge server for calculation, the corresponding optimal task cost calculation method comprises the following steps:
the signal of the user terminal i which can be directly received by the edge server is:
Figure FDA0004137950320000035
wherein ,
Figure FDA0004137950320000036
is the channel between the edge server and the user terminal i,/i>
Figure FDA0004137950320000037
Is a noise signal between the edge server and the user terminal i; the edge server receives the signal of the user terminal i in an auxiliary way through the relay node j as follows:
Figure FDA0004137950320000038
wherein ,
Figure FDA0004137950320000039
is the transmission power of relay node j, +.>
Figure FDA00041379503200000310
Is the auxiliary channel that the relay node j distributes to the user terminal i to the edge server, +.>
Figure FDA00041379503200000311
Is the noise signal at the edge server on the corresponding channel,/or->
Figure FDA00041379503200000312
Is a normalization parameter;
the signal to noise ratio of the user terminal i is obtained by maximum ratio amplitude synthesis at the edge server as follows:
Figure FDA00041379503200000313
wherein ,Pi UT Is the transmission power of the user terminal i,
Figure FDA00041379503200000314
is the channel between the relay node j and the user terminal i;
the transmission rate between the user terminal i and the edge server is:
Figure FDA00041379503200000315
wherein ,Wi Is the bandwidth between the edge server and the user terminal i; the transmission delay is recorded as
Figure FDA00041379503200000316
The transmission energy consumption is->
Figure FDA00041379503200000317
The time required for the task to calculate at the edge server is recorded as +.>
Figure FDA00041379503200000318
The time required by the edge server to calculate the digital twin parameter training model and aggregate the digital twin parameter models of the relay node and the user terminal is as follows:
Figure FDA0004137950320000041
wherein due to
Figure FDA0004137950320000042
Is small, and f ECS Very high, compared to the time taken to train the edge server model, the time taken to aggregate the relay node model +.>
Figure FDA0004137950320000043
Negligible;
f ECS CPU frequency of edge server, D ECS The method comprises the steps that N is the number of relay nodes communicated with an edge server and is a local data set of the edge server;
obtaining the digital twin parameter model of the edge server after training
Figure FDA0004137950320000044
The energy consumption of the edge server is negligible, and the task is offloaded to the edge serverWhen the server calculates, the corresponding optimal task cost is as follows:
Figure FDA0004137950320000045
invoking artificial intelligence algorithm to obtain optimal cost
Figure FDA0004137950320000046
In the step (3.2), the unloading policy selection module builds an optimization model that minimizes the final cost:
the final offload cost for user terminal i in three cases of step (3.1) is expressed as:
Figure FDA0004137950320000047
wherein ,ai E {0,1}, i=1, 2,3 and
Figure FDA0004137950320000048
for each computing task of user terminal i, the optimization model that minimizes the final cost is:
Figure FDA0004137950320000049
Figure FDA00041379503200000410
Figure FDA00041379503200000411
Figure FDA00041379503200000412
/>
wherein ,
Figure FDA00041379503200000413
Figure FDA00041379503200000414
an expected energy consumption threshold for the user terminal;
in said step (3.2), DQN is used as a framework for the DRL algorithm;
in the training process, the unloading strategy selection module interacts with the digital twin body environment to obtain the state of each iteration t task unloading system:
Figure FDA0004137950320000051
the action of the learning Agent is expressed as:
A t ＝{a t |a t ∈I t }
wherein at Is from a set of possible decision actions I t The selected action;
the bonus function reflects that the selected action is in the system state s t Is expressed as:
Figure FDA0004137950320000052
wherein ψ is a guaranteed R t A fixed parameter that is positive, λ is learning rate, μ i (t) is the final cost at iteration t;
the neural network Q (s, a; w) is used in combination with a time difference algorithm to approximate the optimal action cost function to obtain an optimal task offloading strategy model.
2. The method for offloading tasks in a relay edge network based on digital twinning according to claim 1, wherein the user terminal comprises a smart phone, a notebook computer, a mobile tablet;
the user terminal is out of coverage of the edge server.
3. The method for offloading tasks in a relay edge network based on digital twinning according to claim 1, wherein the states of the edge server include processor frequency, available memory capacity, available channels and operating states of the edge server;
the state of the relay node comprises the processor frequency, transmission power, available channels and working state of the relay node;
the state of the user terminal includes the processor frequency of the user terminal, the transmission power, the data size and computational complexity of the task, the task type, the device type, the remaining energy.
CN202110965259.XA 2021-08-23 2021-08-23 Relay edge network task unloading method based on digital twinning Active CN113590232B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110965259.XA CN113590232B (en) 2021-08-23 2021-08-23 Relay edge network task unloading method based on digital twinning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110965259.XA CN113590232B (en) 2021-08-23 2021-08-23 Relay edge network task unloading method based on digital twinning

Publications (2)

Publication Number Publication Date
CN113590232A CN113590232A (en) 2021-11-02
CN113590232B true CN113590232B (en) 2023-04-25

Family

ID=78238836

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110965259.XA Active CN113590232B (en) 2021-08-23 2021-08-23 Relay edge network task unloading method based on digital twinning

Country Status (1)

Country Link
CN (1) CN113590232B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114466356B (en) * 2022-01-29 2022-10-14 重庆邮电大学 Task unloading edge server selection method based on digital twin
CN116962199A (en) * 2022-04-15 2023-10-27 北京邮电大学 Model selection method and device based on environment awareness
CN114609917B (en) * 2022-05-11 2022-08-05 曜石机器人(上海)有限公司 Servo driver and servo system based on digital twin technology
CN116521377B (en) * 2023-06-30 2023-09-29 中国电信股份有限公司 Service computing unloading method, system, device, equipment and medium
CN116820778A (en) * 2023-07-13 2023-09-29 中国电信股份有限公司技术创新中心 Method, system, device, equipment and medium for allocating edge equipment resources

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020023115A1 (en) * 2018-07-27 2020-01-30 Futurewei Technologies, Inc. Task offloading and routing in mobile edge cloud networks
WO2020216135A1 (en) * 2019-04-25 2020-10-29 南京邮电大学 Multi-user multi-mec task unloading resource scheduling method based on edge-end collaboration
CN112104494A (en) * 2020-09-09 2020-12-18 南京信息工程大学 Task security unloading strategy determination method based on air-ground cooperative edge computing network
CN112118601A (en) * 2020-08-18 2020-12-22 西北工业大学 Method for reducing task unloading delay of 6G digital twin edge computing network
CN112419775A (en) * 2020-08-12 2021-02-26 华东师范大学 Digital twin intelligent parking method and system based on reinforcement learning
CN112600912A (en) * 2020-12-10 2021-04-02 西安君能清洁能源有限公司 Unmanned aerial vehicle-assisted edge computing unloading algorithm distributed excitation method
CN113010282A (en) * 2021-03-03 2021-06-22 电子科技大学 Edge cloud collaborative serial task unloading method based on deep reinforcement learning

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020023115A1 (en) * 2018-07-27 2020-01-30 Futurewei Technologies, Inc. Task offloading and routing in mobile edge cloud networks
WO2020216135A1 (en) * 2019-04-25 2020-10-29 南京邮电大学 Multi-user multi-mec task unloading resource scheduling method based on edge-end collaboration
CN112419775A (en) * 2020-08-12 2021-02-26 华东师范大学 Digital twin intelligent parking method and system based on reinforcement learning
CN112118601A (en) * 2020-08-18 2020-12-22 西北工业大学 Method for reducing task unloading delay of 6G digital twin edge computing network
CN112104494A (en) * 2020-09-09 2020-12-18 南京信息工程大学 Task security unloading strategy determination method based on air-ground cooperative edge computing network
CN112600912A (en) * 2020-12-10 2021-04-02 西安君能清洁能源有限公司 Unmanned aerial vehicle-assisted edge computing unloading algorithm distributed excitation method
CN113010282A (en) * 2021-03-03 2021-06-22 电子科技大学 Edge cloud collaborative serial task unloading method based on deep reinforcement learning

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
Qingqing Tang ; Zesong Fei ; Bin Li ; Zhu Han.Computation Offloading in LEO Satellite Networks With Hybrid Cloud and Edge Computing.《IEEE Internet of Things Journal》.2021,第8卷(第11期),第 9164 - 9176页. *
Wen Sun ; Haibin Zhang ; Rong Wang ; Yan Zhang.Reducing Offloading Latency for Digital Twin Edge Networks in 6G.《IEEE Transactions on Vehicular Technology》.2020,第69卷(第10期),第12240-12251页. *
Yiwen Wu ; Ke Zhang ; Yan Zhang.Digital Twin Networks: A Survey.《IEEE Internet of Things Journal》.2021,第8卷(第18期),第13789-13804页. *
梁广俊 ; 王群 ; 辛建芳 ; 李梦 ; 许威.移动边缘计算资源分配综述.《信息安全学报》.2021,第06卷(第03期),第227-256页. *
贺仁龙."5G+产业互联网"时代数字孪生安全治理探索.《中国信息安全》.2019,(第11期),第32-36页. *
高寒 ; 李学俊 ; 周博文 ; 刘晓 ; 徐佳 ; .移动边缘计算环境中基于能耗优化的深度神经网络计算任务卸载策略.计算机集成制造系统.2020,(第06期),第167-175页. *

Also Published As

Publication number Publication date
CN113590232A (en) 2021-11-02

Similar Documents

Publication Publication Date Title
CN113590232B (en) Relay edge network task unloading method based on digital twinning
CN110347500B (en) Task unloading method for deep learning application in edge computing environment
CN111835827B (en) Internet of things edge computing task unloading method and system
CN109862610B (en) D2D user resource allocation method based on deep reinforcement learning DDPG algorithm
CN111800828B (en) Mobile edge computing resource allocation method for ultra-dense network
CN112181666A (en) Method, system, equipment and readable storage medium for equipment evaluation and federal learning importance aggregation based on edge intelligence
CN114340016B (en) Power grid edge calculation unloading distribution method and system
CN116684925B (en) Unmanned aerial vehicle-mounted intelligent reflecting surface safe movement edge calculation method
CN114422349B (en) Cloud-edge-end-collaboration-based deep learning model training and reasoning architecture deployment method
CN113687875B (en) Method and device for unloading vehicle tasks in Internet of vehicles
Yang et al. Deep reinforcement learning based wireless network optimization: A comparative study
CN113626104A (en) Multi-objective optimization unloading strategy based on deep reinforcement learning under edge cloud architecture
CN114265631A (en) Mobile edge calculation intelligent unloading method and device based on federal meta-learning
CN112540845A (en) Mobile edge calculation-based collaboration system and method
CN113286329A (en) Communication and computing resource joint optimization method based on mobile edge computing
CN115659803A (en) Intelligent unloading method for computing tasks under unmanned aerial vehicle twin network mapping error condition
CN115189908B (en) Random attack survivability evaluation method based on network digital twin
CN114363857B (en) Method for unloading edge calculation tasks in Internet of vehicles
CN113411826A (en) Edge network equipment caching method based on attention mechanism reinforcement learning
CN114385272B (en) Ocean task oriented online adaptive computing unloading method and system
CN111611069B (en) Multi-type task migration method among multiple data centers
CN116781141A (en) LEO satellite cooperative edge computing and unloading method based on deep Q network
CN112600869A (en) Calculation unloading distribution method and device based on TD3 algorithm
CN116684851A (en) MAPPO-based multi-RIS auxiliary Internet of vehicles throughput improving method
CN115001937B (en) Smart city Internet of things-oriented fault prediction method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant