CN116963182A

CN116963182A - Time delay optimal task unloading method and device, electronic equipment and storage medium

Info

Publication number: CN116963182A
Application number: CN202311047271.8A
Authority: CN
Inventors: 方超; 罗海振; 杨小平; 胡钊鸣; 石鸿伟
Original assignee: Network Communication and Security Zijinshan Laboratory
Current assignee: Network Communication and Security Zijinshan Laboratory
Priority date: 2023-08-18
Filing date: 2023-08-18
Publication date: 2023-10-27

Abstract

The invention discloses a time delay optimal task unloading method, a time delay optimal task unloading device, electronic equipment and a storage medium. Wherein the method comprises the following steps: receiving at least one subtask in a request task sent by a user terminal, and analyzing target content to be acquired from the subtasks; constructing a minimum total time delay model based on a cloud edge cooperative network according to the subtasks; determining an optimal processing node and an optimal routing decision corresponding to the subtask according to the solving result of the minimum total delay model; and unloading the subtasks to the optimal processing node for processing according to the optimal routing decision, and returning the target content acquired by the optimal processing node to the user terminal. The minimum total time delay model is built for at least one subtask of a request task under the cloud edge cooperative network, and the optimal processing node and the optimal routing decision of the subtask corresponding to the unloading processing are determined by solving the minimum total time delay model, so that the intelligent unloading of the request task based on the task segmentation in the network is realized, and the total time delay of the network is reduced.

Description

Time delay optimal task unloading method and device, electronic equipment and storage medium

Technical Field

The present invention relates to the field of communications technologies, and in particular, to a method and apparatus for offloading a time delay optimal task, an electronic device, and a storage medium.

Background

The task offloading technology is used as one of key technologies of a mobile edge computing (Multi-access Edge Computing, MEC) network, and can reasonably allocate tasks to each network node for task processing according to resource distribution conditions in the network and characteristics of the tasks, so that reasonable allocation of network resources is realized, and the resource utilization rate is improved.

Although the MEC computing network achieves a certain effect in the aspects of improving the network service capability, reducing the network time delay and the like, the link bandwidth resources in the MEC computing network are always limited, so that the transmission of large tasks in the network still has the defects of congestion and high time delay, the service capability of the network is limited, and the requirement of a user on low time delay is difficult to meet.

Disclosure of Invention

The invention provides a method, a device, electronic equipment and a storage medium for unloading a time delay optimal task, which can realize intelligent unloading of a request task based on task segmentation in a network, improve the utilization rate of network resources, reduce the total time delay of the network and further improve the user experience.

According to an aspect of the present invention, there is provided a method for offloading a latency-optimized task, the method comprising:

receiving at least one subtask in a request task sent by a user terminal, and analyzing target content to be acquired from the subtasks;

constructing a minimum total time delay model based on a cloud edge cooperative network according to the subtasks;

determining an optimal processing node and an optimal routing decision corresponding to the subtask according to the solving result of the minimum total delay model;

and unloading the subtasks to the optimal processing node for processing according to the optimal routing decision, and returning the target content acquired by the optimal processing node to the user terminal.

According to another aspect of the present invention, there is provided a latency optimal task offloading apparatus, comprising:

the task analysis module is used for receiving at least one subtask in the request task sent by the user terminal and analyzing target content to be acquired from the subtasks;

the model construction module is used for constructing a minimum total time delay model based on the cloud edge cooperative network according to the subtasks;

the model solving module is used for determining an optimal processing node and an optimal routing decision corresponding to the subtask according to the solving result of the minimum total delay model;

And the task unloading module is used for unloading the subtasks to the optimal processing node for processing according to the optimal routing decision, and returning the target content acquired by the optimal processing node to the user terminal.

According to another aspect of the present invention, there is provided an electronic apparatus including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein, the liquid crystal display device comprises a liquid crystal display device,

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the latency optimal task offloading method of any one of the embodiments of the present invention.

According to another aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to implement the latency optimal task offloading method of any embodiment of the present invention when executed.

According to the technical scheme, in the time delay optimal task unloading method, at least one sub-task in a request task sent by a user terminal is received, and target content to be acquired is analyzed from the sub-task; constructing a minimum total time delay model based on a cloud edge cooperative network according to the subtasks; determining an optimal processing node and an optimal routing decision corresponding to the subtask according to the solving result of the minimum total delay model; and unloading the subtasks to the optimal processing node for processing according to the optimal routing decision, and returning the target content acquired by the optimal processing node to the user terminal. According to the embodiment of the invention, the corresponding minimum total time delay model is constructed for at least one subtask of one large-scale request task under the cloud edge cooperative network, the optimal processing node and the optimal routing decision corresponding to the subtask are determined by solving the minimum total time delay model, and further the subtask is unloaded according to the optimal processing node and the optimal routing decision, so that the intelligent unloading of the request task based on task segmentation in the network can be realized, the network resource utilization rate is improved, the total time delay of the network is reduced, and the user experience is further improved.

The time delay optimal task unloading device, the electronic equipment and the computer readable storage medium provided by the embodiment also have the technical effects.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a method for latency optimal task offloading according to a first embodiment of the present invention;

FIG. 2 is a flowchart of a method for latency optimal task offloading according to a second embodiment of the present invention;

FIG. 3 is a flow chart of a time-lapse optimal task offloading method according to a third embodiment of the present invention;

fig. 4 is a topology diagram of a cloud edge cooperative network model according to a third embodiment of the present invention;

FIG. 5 is a flow chart of a request routing provided in accordance with a third embodiment of the present invention;

FIG. 6 is a flow chart of a DQN based task offloading algorithm provided in accordance with a third embodiment of the invention;

fig. 7 is a flowchart of a distributed caching strategy based on LRFU according to a third embodiment of the present invention;

fig. 8 is a performance example diagram of base station transmission power versus total network delay according to the third embodiment of the present invention;

FIG. 9 is a diagram showing an example of the performance of a buffer memory with a buffer memory coefficient size corresponding to the total delay of the network according to the third embodiment of the present invention;

FIG. 10 is a diagram showing an example of performance of a content category number versus total network delay according to the third embodiment of the present invention;

FIG. 11 is a diagram showing an example of performance of a content popularity corresponding to the total delay of a network according to the third embodiment of the present invention;

fig. 12 is a performance example diagram of packet sending rate versus total network delay according to the third embodiment of the present invention;

FIG. 13 is a graph showing an exemplary performance of the sum of average prize values for each time slot at different learning rates based on task non-segmentation according to the third embodiment of the present invention;

FIG. 14 is a graph showing an exemplary performance of the sum of average prize values for each time slot at different learning rates based on task segmentation according to the third embodiment of the present invention;

FIG. 15 is a graph showing an exemplary performance of the sum of average prize values for each time slot based on different buffer sizes for task non-splitting according to the third embodiment of the present invention;

FIG. 16 is a diagram showing an exemplary performance of the sum of average prize values for each time slot for different buffer sizes based on task segmentation according to the third embodiment of the present invention;

fig. 17 is a schematic structural diagram of a latency optimization task offloading device according to a fourth embodiment of the present invention;

fig. 18 is a schematic structural diagram of an electronic device implementing a latency optimization task offloading method according to an embodiment of the present invention.

Detailed Description

In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Example 1

Fig. 1 is a flowchart of a method for offloading tasks with optimal time delay, which is applicable to a situation of offloading tasks on the premise of ensuring that the total time delay of a network is optimal in a cloud-edge cooperative network, according to an embodiment of the present invention. As shown in fig. 1, the method for unloading a time-delay optimal task provided in the first embodiment specifically includes the following steps:

s110, receiving at least one subtask in the request tasks sent by the user terminal, and analyzing the target content to be acquired from the subtasks.

Wherein, the user terminal may refer to a terminal device for sending a request task, and the user terminal may include, but is not limited to: smart phones, tablet computers, intelligent wearable devices, vehicle terminals and the like. The request task may refer to a task for a user to request various target contents, and the request task may be composed of at least one sub-task, and the target contents may include, but are not limited to: web video content, web text content, web audio content, vehicle status information, and the like.

In the embodiment of the invention, the user can send a corresponding request task by using the user terminal based on the actual target content requirement, wherein the request task can be composed of at least one subtask, the target content which the user wants to acquire can be determined by analyzing the subtask, and the target content can include but is not limited to: network video content, network text content, network audio content, vehicle status information, etc., to which embodiments of the present invention are not limited.

It should be understood that in the existing cloud-edge cooperative network, because network resources such as computing resources and link resources are limited, when the data volume of a single request task sent by a user terminal is too large, the link capacity may be insufficient to transmit the data volume of the request task, and at this time, the request task may generate packet loss, so that the request delay of the request task is increased; on the other hand, the delay for processing a complete task request by a node processing the requested task alone is relatively long, thus severely reducing the service capacity of the network. Based on the above problems, the embodiment of the invention provides a task unloading method based on task segmentation, namely, a complete request task is divided into a plurality of mutually independent sub-tasks for distributed processing and transmission, so that the requirement of the request task on network resources is reduced, the resource utilization rate is improved, the total delay of a network is further reduced, and the service capability of the network is improved. In addition, in the cloud-edge cooperative network, the division flow based on task segmentation is usually completed by the cloud center, and the flow does not occupy the computing resources of the edge network, so the detailed flow of task segmentation is not described herein.

S120, constructing a minimum total time delay model based on the cloud edge cooperative network according to the subtasks.

The cloud-edge cooperative network may be a three-layer topological network structure, and is a cloud center, a Base Station (BS) and a User terminal (MU) from top to bottom, where the Base stations in the network may be macro Base stations (Macro Base Station, MBS), small Base stations (Small Base Station, SBS) or other types of Base stations, which is not limited in the embodiment of the present invention. The minimum total time delay model can be a minimum total time delay model after a task unloading problem based on task segmentation is formulated under a cloud edge cooperative network.

In the embodiment of the invention, the cloud edge cooperative network can be a three-layer topological network structure comprising a user terminal, a base station and a cloud center, and the task unloading process of the subtasks is approximately as follows: firstly, a user terminal sends all subtasks of a request task to a local base station; then, the local base station and the adjacent processing nodes cooperatively process all subtasks of the user terminal according to the self cache content and the computing resources; finally, each processing node transmits the obtained corresponding target content back to the corresponding user terminal after processing the corresponding subtask. Based on the above analysis, the total delay generated by all subtasks of the request task in the cloud-edge cooperative network may include the following two parts: the round trip transmission delay that each subtask generates on the communication link between the user terminal and the final processing node, and the processing delay that each subtask generates from offloading processing at its corresponding processing node. Therefore, the method can construct a corresponding minimum total delay model aiming at the final objective of minimizing the total delay, further formulate the task unloading problem based on task segmentation under the cloud-edge cooperative network, and conveniently select the optimal processing node and the corresponding optimal routing decision from the cloud-edge cooperative network by using a corresponding algorithm.

And S130, determining an optimal processing node and an optimal routing decision corresponding to the subtask according to the solving result of the minimum total delay model.

The optimal processing node may be a corresponding processing node with the minimum total time delay required for processing the subtasks in the cloud-edge cooperative network, and the optimal processing node may be one of a local base station, an adjacent base station of the local base station, and a cloud center. The optimal routing decision may refer to a routing path of a communication link between the user terminal and an optimal processing node corresponding to the subtask.

In the embodiment of the present invention, the manner of determining the optimal processing node and the optimal routing decision corresponding to the subtask according to the solution result of the minimum total delay model may include, but is not limited to, the following: a pre-trained Deep learning Network model such as a Deep Q Network (DQN) algorithm can be called, the built minimum total delay model is solved, an optimal processing node meeting the minimum total delay is selected from a cloud-edge cooperative Network according to the output result of the Deep Q Network algorithm, and a routing path of a communication link between a user terminal and the optimal processing node is used as an optimal routing decision; and a routing algorithm based on node traversal can be called to solve the built minimum total delay model, namely all nodes in the cloud edge cooperative network are traversed in sequence, the nodes can comprise a local base station directly connected with the user terminal, a base station adjacent to the local base station and a cloud center, whether each node can meet the service requirement of a subtask is judged, the node with the minimum total delay is selected from the nodes meeting the service requirement of the subtask to serve as an optimal processing node, and a routing path of a communication link between the user terminal and the optimal processing node is used as an optimal routing decision.

And S140, unloading the subtasks to an optimal processing node for processing according to the optimal routing decision, and returning the target content acquired by the optimal processing node to the user terminal.

In the embodiment of the invention, after the optimal processing node and the optimal routing decision corresponding to the subtask are determined, the subtask can be controlled to be sent to the optimal processing node for unloading processing according to the optimal routing decision, and the target content acquired by the optimal processing node is fed back to the corresponding user terminal.

According to the technical scheme, at least one sub-task in the request task sent by the user terminal is received, and target content to be acquired is analyzed from the sub-tasks; constructing a minimum total time delay model based on a cloud edge cooperative network according to the subtasks; determining an optimal processing node and an optimal routing decision corresponding to the subtask according to the solving result of the minimum total delay model; and unloading the subtasks to the optimal processing node for processing according to the optimal routing decision, and returning the target content acquired by the optimal processing node to the user terminal. According to the embodiment of the invention, the corresponding minimum total time delay model is constructed for at least one subtask of one large-scale request task under the cloud edge cooperative network, the optimal processing node and the optimal routing decision corresponding to the subtask are determined by solving the minimum total time delay model, and further the subtask is unloaded according to the optimal processing node and the optimal routing decision, so that the intelligent unloading of the request task based on task segmentation in the network can be realized, the network resource utilization rate is improved, the total time delay of the network is reduced, and the user experience is further improved.

Example two

Fig. 2 is a flowchart of a time-delay optimal task offloading method according to a second embodiment of the present invention, which is further optimized and extended based on the foregoing embodiments, and may be combined with each of the optional technical solutions in the foregoing embodiments. As shown in fig. 2, the method for unloading a time-delay optimal task provided in the second embodiment specifically includes the following steps:

s210, receiving at least one subtask in the request tasks sent by the user terminal, and analyzing target content to be acquired from the subtasks.

S220, taking the ratio of the data volume of the subtask to the uplink transmission rate as the uplink transmission delay of a communication link between the user terminal and a processing node of the cloud-edge cooperative network, taking the ratio of the data volume of the target content to the downlink transmission rate as the downlink transmission delay of the communication link between the processing node and the user terminal, and taking the sum of the uplink transmission delay and the downlink transmission delay as the round trip transmission delay corresponding to the subtask.

In the embodiment of the invention, the transmission paths of all the subtasks of the request task in the cloud-edge cooperative network are approximately as follows: the ue- > processing node (base station or cloud center) - > ue, so the round trip delay corresponding to the subtask may include: the uplink transmission delay of the subtask is generated on the uplink, and the downlink transmission delay of the target content corresponding to the subtask is generated on the downlink, wherein the uplink transmission delay can be determined based on the ratio of the data volume corresponding to the subtask to the uplink transmission rate, and the downlink transmission delay can be determined based on the ratio of the data volume corresponding to the target content to the downlink transmission rate.

S230, the ratio of the number of revolutions required by the central processing unit of the processing node for processing the subtasks to the number of revolutions of the central processing unit of the processing node is used as the processing time delay of the processing node for processing the subtasks.

The processing node may be a node for processing subtasks in the cloud edge cooperative network, and the processing node may include: a local base station directly connected with the user terminal, a base station adjacent to the local base station, a cloud center and the like. The CPU demand revolution number may refer to a CPU (Central Processing Unit, CPU) revolution number required for the processing node to process the subtasks. The central processor speed of a processing node may refer to the number of CPU revolutions of the processing node per unit time.

In the embodiment of the invention, the time delay generated by processing the subtasks by the processing node, namely the processing time delay, is mainly the time delay generated by the CPU of the processing node when processing the subtasks, and specifically, the processing time delay of the processing subtasks by the processing node can be determined based on the ratio of the number of revolutions required by the CPU of the processing node to the rotational speed of the CPU of the processing node.

S240, determining a total delay model corresponding to the request task based on the round trip transmission delay and the processing delay, and constructing a task unloading optimization function according to the total delay model.

The total delay model may refer to a model of total delay required by requesting a task to obtain service satisfaction under a cloud-edge cooperative network, and the total delay model may be determined based on round trip transmission delay and processing delay corresponding to each subtask in the task request. The task offload optimization function may be an optimization objective function constructed from a total latency model corresponding to the requested task by the pointer.

In the embodiment of the invention, the total time delay required by the user terminal for sending the task request and receiving the corresponding target content can comprise the round trip transmission time delay and the processing time delay corresponding to each subtask, so that the total time delay model corresponding to the requested task can be determined based on the round trip transmission time delay and the processing time delay of each subtask.

Further, on the basis of the above embodiment of the present invention, the total delay model corresponding to the request task, that is, in the time slot t, the j-th user terminal under the i-th base station (the base station directly connected to the user terminal, that is, the local base station) sends the request task f, and obtains the total delay of the corresponding target content from the processing node m Can be expressed as:

in the method, in the process of the invention,indicating the round trip transmission delay generated by the data transmission between the user terminal j and the local base station i in the time slot t, tr indicating transmission (transmission); />Representing the processing subtask f of the local base station i _k The resulting processing delay, p, represents processing; />Representing a local base stationi a subtask set of the request task f processed; />Is a Boolean variable and indicates whether the local base station i caches subtasks f in the t-th time slot _k Taking 1 to represent that the corresponding target content is cached, and taking 0 to represent that the corresponding target content is not cached; />As boolean variable, it is indicated whether the local base station i offloads the subtask f in the t-th time slot _k Taking 1 to represent unloading treatment and taking 0 to represent unloading treatment;indicating that the local base station i sends a subtask f to its connected neighboring processing node i' (i.e. the base station and cloud center neighboring the local base station) _k And obtains the total delay from the adjacent processing node i' for the corresponding target content, wherein,representing the round trip transmission delay between the local base station i and the neighboring processing node i ', +_, the processing node i' is a processing node>Representing a processing subtask f of an adjacent processing node i _k The generated processing time delay; />Representing a set of neighboring processing nodes of the local base station i; / >Representing a subtask set of the requested task f processed by the neighboring processing node i'.

Further, the objective is to minimize the total delay model, so as to obtain a corresponding task unloading optimization function.

S250, determining a minimum total time delay model according to a preset task unloading optimization constraint condition and a task unloading optimization function.

The preset task unloading optimization constraint condition may be a preset condition for constraining various parameter variables in the task unloading optimization function.

In the embodiment of the invention, the parameter variable in the task unloading optimization function can be constrained based on one or more preset task unloading optimization constraint conditions, so that a minimum total time delay model under the cloud-edge cooperative network is constructed. By formulating the task unloading problem based on task segmentation under the cloud-edge cooperative network into a corresponding minimum total time delay model, the optimal processing nodes and corresponding optimal routing decisions can be conveniently selected from the cloud-edge cooperative network by using a corresponding algorithm.

Further, on the basis of the above embodiment of the present invention, the preset task unloading constraint condition may include at least one of the following:

in any time slot, the transmitting power of any user terminal in the cloud-edge cooperative network is smaller than or equal to a first preset transmitting power threshold value of a corresponding user terminal, and in any base station in the cloud-edge cooperative network, the sum of the transmitting power of all the user terminals after power distribution is smaller than or equal to a second preset transmitting power threshold value of the corresponding base station;

All the subtasks of the request task are combined to form a complete request task, and all the subtasks are mutually independent;

in any time slot, all network content data amount cached by any base station in the cloud edge cooperative network is smaller than or equal to a preset cache capacity threshold value of the corresponding base station;

in any time slot, the same network content cannot be cached between any two adjacent base stations in the cloud edge cooperative network;

in any time slot, the total calculation capacity required by any node in the cloud edge cooperative network for processing the subtasks is smaller than or equal to a preset calculation capacity threshold value of the corresponding node;

the same subtask can not be repeatedly processed among different nodes in the cloud edge cooperative network;

a first subtask set of a request task is processed by a local base station in the cloud edge cooperative network, and a union set of a second subtask set of the request task processed by an adjacent processing node in the cloud edge cooperative network is required to form a complete request task;

the total transmission flow of any communication link in the cloud-edge cooperative network is smaller than or equal to a preset link bandwidth threshold value of the corresponding communication link;

all boolean variables in the total delay model have values of 1 or 0.

S260, determining a network state of the cloud edge cooperative network in the current time slot under the subtask as a model input parameter of a minimum total time delay model, wherein the network state comprises: processing nodes, processing node types, subtasks, adjacent processing node sets, processing node buffer status, link traffic.

In the embodiment of the invention, the DQN algorithm can be invoked to solve the constructed minimum total time delay model, and specifically, the network state of the subtask in the current time slot under the cloud-edge cooperative network can be obtained, wherein the network state can comprise the following parameters: the method comprises the steps of processing nodes, processing node types, subtasks, adjacent processing node sets, processing node cache states and link traffic, and taking the network states as model input parameters of a minimum total time delay model, wherein the processing nodes in the network states refer to nodes of the current processing subtasks, the processing node types represent node types of the current nodes, namely base stations or cloud centers, the adjacent processing node sets represent the adjacent processing node sets of the current nodes, the processing node cache states represent the cache states of the current nodes, and the link traffic represents traffic of a communication link between a user terminal and the current nodes.

S270, determining a delay value output by the minimum total delay model under the network state of the current node in the cloud edge cooperative network.

In the embodiment of the invention, the current node may refer to a certain node in the cloud edge cooperative network, the current node may be a local base station directly connected with the user terminal, or may be an adjacent processing node of the local base station, and exemplary, the local base station may be selected as the current node and used as an initial node input by a minimum total delay model, and a delay value output by the minimum total delay model of the current node in the current network state is determined, where the delay value may include a round trip transmission delay generated by a subtask of a task request transmitted between the user terminal and the current node and a processing delay generated by the current node.

S280, determining the rewarding value of the time delay value according to a preset rewarding value function.

The preset reward value function may be a function which is preconfigured to determine a corresponding reward value according to a time delay value of the subtask.

In the embodiment of the invention, a preset reward value function which is preset can be called to determine the reward value corresponding to the time delay value of the current node, wherein the preset reward value function can be generated based on the proportional relation of the total consumption time delay of the subtasks in the current time slot.

Further, on the basis of the embodiment of the invention, under the time slot t, the subtask f _k Corresponding prize value functionCan be expressed as

Wherein, gamma represents a discount factor for adjusting the proportional relation between the rewarding value and the time delay value; t (T) _t ^fk Indicating that the user terminal transmits subtask f in time slot t _k And obtains the total consumption time delay of the corresponding target content.

And S290, judging whether the rewarding value meets the preset service requirement of the subtask, if so, taking the current node as an optimal processing node, and if not, selecting other nodes as new current nodes in the cloud-edge cooperative network according to a preset greedy strategy, and determining the network state of the new current nodes.

The preset service requirement may refer to a preset service requirement for a subtask, and the preset service requirement may include, but is not limited to: whether the corresponding rewarding value of the subtask is larger than a certain rewarding value threshold value when the service is satisfied, whether the corresponding total rewarding value of all subtasks of the request task is larger than a certain rewarding value threshold value when the service is satisfied, and the like. The preset greedy policy may refer to a preconfigured epsilon-greedy policy, and the preset greedy policy may be used to randomly select other nodes as new current nodes in the cloud-edge cooperative network.

In the embodiment of the invention, whether the determined rewarding value meets the preset service requirement of the subtask can be judged according to the determined rewarding value, and if the determined rewarding value meets the preset service requirement, the current node is used as the optimal processing node for finally processing the subtask; if the current node is not satisfied, a pre-configured greedy strategy can be called to select other nodes in the cloud edge cooperative network as new current nodes, namely, the next action in the DQN algorithm, and the network state corresponding to the new current nodes is redetermined.

Further, on the basis of the above embodiment of the present invention, S290 may further include: after determining the network state of the new current node, the network state is saved to an empirical playback pool so that data can be subsequently retrieved from the empirical playback pool using the DQN algorithm to train the neural network.

And S2100, determining a reward value of a new current node in the network state according to the minimum total time delay model and a preset reward value function, and repeatedly executing the process until the current node or the subtask meeting the subtask is determined to be lost.

In the embodiment of the present invention, according to the steps from S270 to S280, the prize value of the new current node in the corresponding network state may be redetermined, and further, whether the new prize value meets the preset service requirement may be judged, if yes, the new current node is used as the optimal processing node, and if not, other nodes are selected as the next new current node in the cloud edge cooperative network according to the preset greedy policy, and the above process is repeatedly executed until the optimal processing node meeting the preset service requirement is obtained, or the subtasks are lost.

S2110, determining a routing path corresponding to the optimal processing node as an optimal routing decision.

In the embodiment of the present invention, the optimal routing decision may refer to a routing path of a communication link between a user terminal and an optimal processing node corresponding to a subtask, and if the optimal processing node is a local base station directly connected to the user terminal, the corresponding optimal routing decision is a routing path of the communication link between the user terminal and the local base station; if the optimal processing node is an adjacent processing node (base station or cloud center) of the local base station, the corresponding optimal routing decision is a routing path from the user terminal to the local base station and then to a communication link between the adjacent processing nodes, wherein the local base station serves as an intermediate node in the routing path.

S2120, unloading the subtasks to the optimal processing node for processing according to the optimal routing decision, and returning the target content acquired by the optimal processing node to the user terminal.

Further, on the basis of the embodiment of the present invention, the cloud edge cooperative network further includes a caching policy of at least one of the following: LRFU-based distributed caching strategy, offline collaborative caching strategy based on content popularity.

In the embodiment of the present invention, since the edge cache plays a leading role in the cloud-edge cooperative network, the cache capacity of the MEC node is limited, so that an efficient cache policy is needed to further reduce the access delay of the user, where the cache policy may include at least one of the following: an LRFU-based distributed caching strategy and an offline collaborative caching strategy based on content popularity, wherein the LRFU-based distributed caching strategy combines the advantages of both LRU (Least Recently Used) algorithm and LFU (Least Frequently Used) algorithm, and a node dynamically updates cache network content according to historical space-time request information of a request task of each time slot, wherein the historical space-time request information comprises access time and access frequency of the request task; the offline collaborative caching strategy based on the content popularity is a theoretically optimal caching deployment mode, and can ensure that network contents with high popularity are processed at edge nodes, and the network contents with lower popularity are easier to transmit to a cloud center for processing.

According to the technical scheme, at least one sub-task in the request task sent by the user terminal is received, and target content to be acquired is analyzed from the sub-tasks; taking the ratio of the data volume of the subtask to the uplink transmission rate as the uplink transmission delay of a communication link between the user terminal and a processing node of the cloud edge cooperative network, taking the ratio of the data volume of the target content to the downlink transmission rate as the downlink transmission delay of the communication link between the processing node and the user terminal, and taking the sum of the uplink transmission delay and the downlink transmission delay as the round trip transmission delay corresponding to the subtask; taking the ratio of the number of revolutions required by the central processor of the subtasks processed by the processing node to the number of revolutions of the central processor of the processing node as the processing time delay of the subtasks processed by the processing node; determining a total delay model corresponding to the request task based on the round trip transmission delay and the processing delay, and constructing a task unloading optimization function according to the total delay model; determining a minimum total time delay model according to a preset task unloading optimization constraint condition and a task unloading optimization function; determining a network state of the cloud edge cooperative network in a current time slot under a subtask as a model input parameter of a minimum total time delay model, wherein the network state comprises: processing nodes, processing node types, subtasks, adjacent processing node sets, processing node buffer status and link traffic; determining a time delay value output by a minimum total time delay model under the network state of a current node in the cloud edge cooperative network; determining a reward value of the time delay value according to a preset reward value function; judging whether the rewarding value meets the preset service requirement of the subtask, if so, taking the current node as an optimal processing node, if not, selecting other nodes as new current nodes in the cloud-edge cooperative network according to a preset greedy strategy, and determining the network state of the new current nodes; determining a reward value of a new current node in the network state according to the minimum total time delay model and a preset reward value function, and repeatedly executing the process until the current node meeting the subtask or the subtask is determined to be lost; determining a routing path corresponding to the optimal processing node as an optimal routing decision; and unloading the subtasks to the optimal processing node for processing according to the optimal routing decision, and returning the target content acquired by the optimal processing node to the user terminal. According to the method and the device, the optimal task unloading problem based on task segmentation is formulated into the minimum total time delay model under the cloud edge cooperative network, the constructed minimum total time delay model is solved through the DQN algorithm to determine the optimal processing node and the optimal routing decision corresponding to the subtasks, and then the subtasks are unloaded according to the optimal processing node and the optimal routing decision, so that intelligent unloading of the task-based request in the network can be realized, the optimal unloading decision can be made according to the current network state information, the self-adaptive allocation of network resources is realized, the network resource utilization rate is improved, the total time delay of the network is reduced, and the user experience is further improved.

Example III

Fig. 3 is a flowchart of a method for offloading a task with optimal time delay, which is provided in the third embodiment of the present invention, and this embodiment provides an implementation manner of the method for offloading a task with optimal time delay based on task segmentation in a cloud-edge cooperative network, so as to ensure that the total time delay of the task is optimal. As shown in fig. 3, the method for unloading a time-delay optimal task provided in the third embodiment of the present invention specifically includes the following steps:

s310, constructing a cloud edge collaborative network model, and requesting a task model and a content popularity model based on task segmentation.

S3101, constructing a cloud edge cooperative network model of three-layer topology.

In order to effectively analyze network content delay under a heterogeneous network with cloud edge coordination, the embodiment of the invention designs a cloud edge coordination network model shown in fig. 4, wherein the network model is of a three-layer topological network structure, namely a cloud center, base stations BS and user terminals MU from top to bottom, the base stations BS are connected with the cloud center through a complex backbone network, and each base station BS is connected with a part of user terminals through wireless transmission. The network model also considers the situation of direct connection between the base stations BS, and the cache content and the request task in the base stations which are directly connected with each other can be mutually transmitted at high speed through the optical fiber network. Only the cloud center and the base station BS have the buffering capability and the computing capability, and the user terminal MU has no buffering and computing capability. In the cloud edge cooperative network model, the cache capacity of the cloud center is set to be a maximum value, so that the cloud center can cache all contents in a network, and the computing capacity of the cloud center can meet the computing requirements of all the contents in the network; while the base station BS has limited buffering and computing power, only part of the content in the network can be buffered and computed.

For the mobility problem of users, the embodiment of the present invention assumes that the number of users connected under each base station varies randomly within each time slot. For the problem of the number of requests sent by the users, it may be assumed that the number of request packets actually sent by each user in each time slot is randomly changed, where the total number of request packets sent by all users is equal to the total number of request packets in the time slot.

S3102, constructing a request task model based on task segmentation.

According to the embodiment of the invention, the complete request task is divided into the plurality of mutually independent sub-tasks for distributed processing and transmission, so that the requirement of the request task on network resources is reduced, the resource utilization rate is improved, the total delay of the network is further reduced, and the service capability of the network is improved. In the request task model, f is used to represent a request task, f is used _k The kth subtask of the request task f is represented.A subtask set representing a request task F, where F ^f The number of subtasks for the request task f, and the subtask f _k All sub-tasks that have to satisfy a request task f must be combined to form a complete request task f constraint.

S3103, constructing a content popularity model.

The embodiment of the invention designs a popularity model of the network content according to Ji Pufu law. For a given period of time, the total number of requests by the base station is R, and then the distribution of popularity of the content with the content number k is:

wherein, the Ji Pufu skewness coefficient alpha represents popularity of network content requests, and the larger the alpha value is, the more concentrated the network content requests of users are, and the larger the request amount of popular data is; f represents the number of categories of different web content.

S320, designing a routing strategy based on task segmentation.

In the embodiment of the present invention, the routing policy based on task segmentation provided in the embodiment of the present invention is shown in fig. 5, and the policy of request routing is approximately as follows: first, the user terminal j will request all subtasks f of task f _k Transmitting to a local base station i; then, the local base station i and the adjacent processing nodes i' (base station or cloud center) cooperatively process all subtasks of the user terminal j according to the self cache content and the computing resource; finally, each processing node is processing its corresponding subtask f _k And then the obtained corresponding target content is sent back to the user terminal j. Notably, in the cloud-edge cooperative network, the local base station i and its neighboring processing nodes i' process their respective subtasks f _k And simultaneously.

As shown in fig. 5, in the step of the initialization phase, the following parameters need to be initialized: (1) the state identifier Flag is taken as 1 to indicate that the subtask is satisfied by service, and taken as 0 to indicate that the subtask loses packets; (2) the total task Route array Route [ ] is a two-dimensional array which is used for recording the Route paths corresponding to all subtasks; (3) a subtask request delay array t_sum_list [ ], which is used for recording the corresponding processing delay of each subtask and the round-trip transmission delay from the local base station to the processing node; (4) and the node popularity array is used for recording the request satisfaction times of each subtask. Before each time slot starts, the allocation of buffering, link capacity resetting, computing power, node popularity and transmitting power in the cloud-edge cooperative network is adjusted according to the node popularity.

In the step of judging whether the link bandwidth is limited, it should be noted that the request packet of the subtask can be uploaded only when the uplink bandwidth is greater than 1; only if the downlink bandwidth is greater than 1, the target content data packet corresponding to the subtask can be returned.

In the step of counting packet loss (flag=0), it is necessary to set: (1) counting the packet loss content, and establishing a packet loss group for recording subtasks of packet loss and user nodes (namely user terminals) of packet loss; (2) and updating the link capacity bandwidth on the packet loss path.

In the step of querying the content cache, the computing power and the bandwidth of the local base station connected to the user terminal, a function Query (node 1, node2, request) may be invoked, the function of which is: inquiring whether the node meets the requirements of content caching, computing capacity and bandwidth, returning a result=1 to the function to indicate that the inquired node meets the service requirements of the subtasks, and returning a result=0 to the function to indicate that the inquired node does not meet the service requirements of the subtasks; where node1 represents the current node number, node2 represents the query node number, and the request represents the requested content (i.e., the target content corresponding to the subtask).

In the step of recording the path (flag=1), it is necessary to set: (1) storing the Route path corresponding to the subtask into a Route [ ] array; (2) updating the link capacity, and subtracting 1 from all the link capacities in the routing path; (3) updating the occupation condition of the computing resources, namely subtracting the computing capacity required by the current subtask from the computing resources of each processing node on the routing path; (4) updating node popularity, namely adding 1 to request content corresponding to all node popularity groups on a path after the subtask is satisfied by service; (5) and counting the arrival number of the subtasks reaching the node in one time slot and the number of the subtasks processed in the node.

In the step of sequentially querying the content cache, the computing power and the bandwidth of the neighboring nodes of the local base station, a function requirement (node 1, node_list, request) may be invoked, the function of which is: the Query function Query is called for each node in the adjacent node array, and a nodes array meeting the condition is returned, wherein the nodes array is used for recording all nodes meeting the condition; where node1 represents the neighbor number, node_list represents the neighbor array, and request represents the request content.

In the step of determining the delay to each node, a function mindelay node (node 1, node_list, request) may be invoked, the function of which is: calculating the time delay of each node, finding out the node with the minimum round trip transmission time delay and processing time delay, and finally returning to the node with the minimum time delay of min_delay_node; where node1 represents the neighbor node number, node_list represents the node number array, and request represents the request content.

In the step of determining the round trip transmission Delay and the processing Delay corresponding to the current subtask, a function Delay (node 1, route, i, request) may be called, and the function returns: the specific function functions of the round trip transmission delay trans_delay between the local base station and the final processing node and the processing delay com_delay of the final processing node are as follows: if the local base station is the final processing node, the trans_delay returns to null, and the com_delay returns to the processing time delay of the final processing node; if the local base station is an intermediate node, the trans-delay returns to the sum of round trip transmission delays between the local base station and the final processing node, and the com-delay returns to the processing delay of the final processing node; wherein node1 represents the number of the adjacent node, route represents the two-dimensional array of routing paths corresponding to all subtasks of the current request task in the current time slot, i represents the ith subtask in the current time slot, and request represents the request content.

In the step of determining the total latency of the currently requested task, a function TaskDelay (node 1, t_sum_list, route, request) may be invoked, which returns: the total delay task of the current request task has the following specific function functions: the last end node of the routing path is compared to judge which processing node the current subtask finally performs unloading processing, so that the time delay of all the subtasks processed by a certain processing node can be counted, after the time delays of all the processing nodes are counted, a maximum value is found out, the request uploading time delay and the data returning time delay corresponding to the subtasks are added, finally, whether the subtasks of the current request task are contained in the packet loss data group or not is determined, if yes, the corresponding packet loss retransmission time delay is added, if not, the total time delay of the current request task is obtained; wherein node1 represents the number of the user node, t_sum_list represents a two-dimensional array of processing delay and round trip transmission delay corresponding to each subtask, route represents a two-dimensional array of routing paths corresponding to all subtasks of the current request task in the current time slot, and request represents request content.

S330, constructing a network content total time delay model and a minimum total time delay model.

In the embodiment of the invention, the request task is sent from the user side, and is transmitted to the edge node or the cloud center for processing based on the set routing mode, and finally, the target content corresponding to the request task is transmitted back to the user, and the time difference from sending to receiving is the total time delay for transmitting the request task. The embodiment of the invention divides the total time delay into two parts: round trip transmission delays generated on links and processing delays generated in base stations or clouds. Next, specific steps for constructing the network content total delay model and the minimum total delay model are described.

S3301, constructing a network task transmission delay model.

The transmission delay generated by the link is mainly the delay generated by the link transmission data, and is divided into uplink transmission delay and downlink transmission delay. Specifically, user terminal j sends subtask f to the local base station in time slot t _k And obtains the round trip transmission delay of the corresponding target content from the processing node m as follows:

in which the first term to the right of the equation equal sign indicates the uplink transmission delay between the user terminal j and the processing node m at time slot t,representing subtask f _k Data amount r of (2) _jm Representing an uplink transmission rate between the user terminal j and the processing node m; the second term to the right of the formula equal sign indicates that at time slot t, processing node mSubtask f _k The downlink transmission delay of the corresponding target content sent back to user terminal j,/>Data amount r representing target content _mj Representing the downlink transmission rate between processing node m and user terminal j.

S3302, constructing a processing time delay model.

The delay generated by the processing node is mainly the processing delay generated by the CPU when processing the subtasks. Specifically, processing node m processes subtask f _k The processing time delay generated is as follows:

in the method, in the process of the invention,representing processing node m processing subtask f _k The number of revolutions required by the CPU, i.e. processing subtask f _k The required number of CPU revolutions; v (V) _m The rotation speed of the central processing unit of the processing node m, namely the rotation speed of the CPU of the processing node m in unit time is represented.

S3303, constructing a network content total time delay model.

Based on the analysis, constructing a network content total time delay model, namely, in a time slot t, the j-th user terminal under the i-th base station (the base station directly connected with the user terminal, namely, the local base station) sends a request task f, and obtains the total time delay of corresponding target content from a processing node m Can be expressed as:

s3304, constructing a minimum total time delay model.

Based on the analysis, the task offloading problem based on task segmentation in the cloud-edge collaborative network can be formulated as a minimum total delay model, wherein the task offloading optimization function is expressed as follows:

in the method, in the process of the invention,representing a set of time slots t; />Representing a set of local base stations i; />Representing a set of user terminals j; />Representing a set of requested tasks f.

The corresponding preset task offloading constraints are as follows:

/>

wherein, constraint condition C1 indicates the transmitting power P of any user terminal j in the cloud-edge cooperative network in any time slot t _ji (t) is less than or equal to a first preset transmit power threshold for the corresponding user terminal jIn any base station i in the cloud-edge cooperative network, the sum of the transmission power of all user terminals j after power distribution is smaller than or equal to a second preset transmission power threshold value +_of the corresponding base station i>

Constraint C2 represents all subtasks f of the request task f _k After merging, the complete request task f must be formed, and each subtask f _k Are mutually independent;

constraint C3 indicates that in any time slot t, all network content data amounts cached by any base station i in the cloud edge cooperative network Less than or equal to a preset buffer capacity threshold O for the corresponding base station i _i ；

Constraint condition C4 indicates that in any time slot t, the same network content cannot be cached between any two adjacent base stations in the cloud edge cooperative network, namely, the adjacent base stations should be cooperatively cached;

constraint C5 represents the total amount of computing power required by any node i in the cloud-edge cooperative network to process the subtasks during any time slot tLess than or equal to a preset computational capability threshold C of the corresponding node i _i ；

Constraint condition C6 indicates that the same subtask cannot be repeatedly processed among different nodes in the cloud edge cooperative network;

constraint C7 indicates that a local base station i in the cloud edge cooperative network processes a first subtask set of a request task fA second subtask set of a request task f is processed by an adjacent processing node i' in the cloud edge cooperative network>The union of (a) must form the complete request task f;

constraint C8 represents any communication link l in cloud-edge cooperative network _ij Total transport traffic (wired or wireless)Less than or equal to the corresponding communication link _ij Is>

Constraint C9 represents all Boolean variables in the total delay model, i.e Must be either 1 or 0.

And S340, determining an optimal processing node and an optimal routing decision corresponding to the subtask according to the solving result of the minimum total delay model.

The embodiment of the invention solves the constructed minimum total time delay model by utilizing the DQN algorithm of one of branches under the deep reinforcement learning (Deep Reinforcement L earning, DRL) algorithm, so as to find the optimal processing node and the optimal routing decision corresponding to the subtask. Next, first, a description will be given of a DQN-based task offloading algorithm that does not subtask the request task f. Fig. 6 is a flowchart of a task offloading algorithm based on DQN according to a third embodiment of the invention. As shown in figure 6 of the drawings,indicating the current network state of task request f in time slot t +.>Is an action to be performed in the current state of task f, and +.>Is to request task f to be in the current network state +.>Execution of action down->The actual rewards obtained. />Indicating that the requesting task f is performing an action +.>And then enter the next state. The DQN algorithm includes two neural networks: the evaluation network and the target network have the same neural network structure but different parameters. Evaluation networkThe function of (2) is by entering the state +.>To output the appropriate action value. Wherein, in order to avoid the situation that the algorithm falls into local optimum, the evaluation network selects the probability as epsilon-epsilon [0,1 ] through epsilon-greedy strategy ]The random action or the maximum action of Q value with probability of 1-epsilon. The DQN algorithm is performed substantially as follows: firstly, the evaluation network and the target network take out a set of history data from the experience pool +.>Wherein the evaluation network is via the input->And->Generate->And the target network is input +.>GeneratingThen, the system obtains the difference value between the actual Q value and the estimated Q value according to the calculated loss function, and obtains the parameter with the minimum difference value between the actual Q value and the estimated Q value by using a gradient descent method, and further uses the parameters to update the related parameters of the evaluation network. The loss function in an embodiment of the present invention is defined as a mean square error (Mean Squared Error, MSE) function as follows:

wherein γ represents a discount rate; w represents the weight of the evaluation network; w (w) ^- Representing the rights of a target networkWeighing;representing the actual Q value; />Representing the estimated Q value.

The DQN-based task offloading algorithm for subtask division of the requested task f is described below.

In the embodiment of the invention, the request task f is divided into N independent sub-tasks, and each sub-task can be routed by using the DQN algorithm. Specifically, in a network environment, subtask f _k The network state generated in time slot t is defined as Wherein n is _t Representing the currently processed subtask f _k Is a processing node of (a); />Representing the current processing node n _t Node type (base station or cloud center); f (f) _k A kth subtask representing a current task; />Representing the current processing node n _t Is a set of adjacent processing nodes; />Representation about node n _t Is a cache state of (a); />Representing the current processing node n _t Communication link with user terminal j>Is a flow rate of (a). While at time slot t, subtask f _k Action of (2)Is defined as +.>It represents the behavioral actions of the current node to select the next node.

To achieve the best task offload strategy, embodiments of the present invention design a reward value function that may combine the environmental feedback signal with the task offload optimization function. During the routing of the subtasks, the system will receive the reward signal according to the delay value when its service is satisfied. Meanwhile, if the request is satisfied at the network edge (local BS and neighbor BS), the system will get more prize values to reduce the content transmission delay; if the subtask request packet is lost during the routing process, the system cannot get rewards. Specifically, under time slot t, subtask f _k The prize value function of (2) is:

in the DQN task offloading method based on task segmentation, the core goal is to find the optimal decision strategy by training and learning the historical data in each time slot t so as to maximize the sum of the expected reward values of all the requested tasks sent by all the user terminals. Because the prize value is inversely proportional to the time delay, the time delay is correspondingly minimized when the prize value is maximized.

During the course of the DQN route, if the current base station is able to meet subtask f _k The system immediately terminates the current subtask f _k And sub-task f _k And returning the corresponding target content to the corresponding user node.

Conversely, if the current node cannot meet subtask f _k The system will randomly select an action or state information based on epsilon-greedy policyTo the evaluation network and to obtain the next action from the evaluation networkIn executing action->The system will then feed back the corresponding prize value +.>And in the next state->At the same time, the system will also add this status information->And storing the data into an experience playback pool so as to acquire the data from the experience playback pool for training the neural network. The above process is circularly executed until the subtask f _k Is satisfied or dropped. It should be noted that in the DQN task offloading method based on task segmentation, all subtasks f of a requested task f need to be processed _k And after the DQN route searching is carried out, the DQN route searching of the next request task is carried out.

In summary, by using the DQN task offloading method based on task segmentation, an optimal processing node and an optimal routing decision corresponding to a subtask can be determined in the cloud-edge cooperative network according to current network state information.

Furthermore, since the edge cache plays a leading role in the cloud-edge cooperative network, the embodiment of the invention provides the following two cache strategies, and the performance of the two cache strategies is verified in the following simulation test.

(1) LRFU-based distributed cache policy

A flow chart of an LRFU based distributed caching strategy is shown in fig. 7. The LRFU-based distributed caching strategy is a dynamic caching scheme, combines the advantages of the LRU algorithm and the LFU algorithm, and dynamically updates the cached network content according to the historical space-time request information of the request task of each time slot, wherein the historical space-time request information comprises the access time and the access frequency of the request task. Theoretically, repeatedly replaced is low popularity content, because low popularity content is not frequently accessed, often has the longest residence time in the cache area, and popular content is often accessed, often has its residence time cleared, so cached content is not often replaced. Caching the most popular content is desirable because it can increase the hit rate of requests at the edge as much as possible. Meanwhile, the real-time caching strategy can be realized in practical engineering, because the basis of the base station for caching the content is the type of the content stored in the cache region and the corresponding residence time thereof, and the type of the currently-incoming request content. The base station does not need to make a prejudgment of popular content, but makes a real-time caching decision according to the information which is owned before and now.

(2) Offline collaborative caching strategy based on content popularity

The offline collaborative caching strategy based on the popularity of the content is a theoretically optimal caching deployment mode, and can ensure that network content with high popularity is processed at an edge node, and the network content with lower popularity is easier to transmit to a cloud center for processing. In the subsequent simulation test, the content which is suitable for the set cache size is stored in advance, the content is stored in the edge cache area from high to low according to popularity ranking, the base station closest to the user is stored in priority, cloud edges cooperate with three layers of complementary storage of the network, and a cloud center stores all possible request content. According to the ziff distribution, the higher the popularity of the request content, the higher the ratio of the request quantity to the total request quantity. The greater the probability that the request hits at the edge. This theoretically optimal offline collaborative caching strategy is to minimize the transmission delay of the network content from the perspective of the caching strategy.

S350, unloading the subtasks to the optimal processing nodes for processing according to the optimal routing decision, and returning the target content acquired by the optimal processing nodes to the corresponding user terminals.

The time delay optimal task unloading method provided by the embodiment of the invention is verified and performance analyzed by combining simulation tests.

(1) Simulation parameter setting

The settings of some constant values in the simulation test are shown in the following table:

the settings of some of the variable values in the simulation test are shown in the following table:

/>

(2) performance analysis

In the simulation test of the embodiment of the invention, the following 8 solutions are verified by comparison:

scheme 1: OSPF+LRFU+task; scheme 2: ospf+pop+task, scheme 3: ospf+lrfu+subtask, scheme 4: ospf+pop+subtask, scheme 5: drl+lrfu+task, scheme 6: DRL+POP+task, scheme 7: drl+lrfu+subtask, scheme 8: DRL+POP+subtask. Wherein, OSPF represents a traditional routing mode based on an open shortest path first algorithm (Open Shortest Path First, OSPF); DRL represents an intelligent routing manner based on DRL (DQN); LRFU represents an LRFU-based distributed caching strategy; POP represents an offline collaborative caching policy based on content popularity; task represents a Task unloading method based on Task non-segmentation; subtask represents a task offloading method based on task segmentation.

Figure 8 shows the total network delay for all solutions as the transmit power of each base station varies. As can be seen from fig. 8, the overall network delay for all solutions shows a decreasing trend. This is because as the transmit power increases, the signal-to-noise ratio of the channel transmitted by the base station to the user terminal increases, thereby increasing the transmission rate from the base station to the user terminal. Furthermore, it can be seen that the performance of the task segmentation based offload method is better than the performance of the task non-segmentation offload method. This is because the network resource utilization is improved after the requested task is split, and the transmission and offloading of all subtasks can be processed in parallel. Meanwhile, the packet loss time delay of the request task is much larger than that of the subtasks, so that the delay after the task segmentation is lower than that after the task is not segmented. Finally, it can be seen that the DRL-based intelligent routing approach has better performance than the traditional OSPF routing approach. This is because the DRL can implement the best offloading decision in the network based on the current network state and the prize value of the previous decision, thereby reducing the overall latency of the network.

Figure 9 shows the total network delay for all solutions as the buffer factor size of each base station node changes. As can be seen from fig. 9, the overall network delay for all solutions shows a decreasing trend. This is because, as the size of the cache coefficient increases, more content files of interest to the user will be cached in the base station, so that the base station can more satisfy the requests of nearby users, and the cache hit rate is improved. Notably, as the cache coefficient size increases, the performance gap between LRFU cache policy based solutions and POP cache policy based solutions is also gradually increasing. This is because as the size of the cache coefficient increases, the solution based on the POP cache policy always defaults to caching the most popular content in the network, while the dynamic adjustment of the solution based on the LRFU cache policy cannot store all popular content in the cache, resulting in a slower cache hit rate of the solution based on the LRFU cache policy than the solution based on the POP cache policy, and thus the performance gap between them increases with the increase in the size of the cache coefficient. However, it can be seen that the performance gap between a solution with task segmentation and a solution without segmentation is getting smaller and smaller. This is because as the size of the cache coefficient increases, the cache hit rate increases, thereby reducing the number of lost packets. The performance gap between the tasks is gradually reduced as the packet loss delay of the tasks is far as the packet loss delay of the Gao Yuzi tasks.

Figure 10 shows the total network delay for all solutions when the number of different contents in the network varies. As can be seen from fig. 10, the total network delay for all solutions shows an upward trend. This is because as the number of different contents increases, the user sends more requests to acquire the unpopular contents, which reduces the cache hit rate per base station. It is noted that as content diversity increases, the performance gap between LRFU cache policy-based solutions and POP cache policy-based solutions will become larger and larger. This is because the accuracy of content popularity decreases with increasing content diversity, so that the cache hit rate of LRFU cache policy-based solutions decreases faster than POP cache policy-based solutions. In addition, it can be seen that the performance gap between the solution with task segmentation and the solution without segmentation is also gradually increasing. This is because as the diversity of contents increases, more contents need to be offloaded through the cloud center, resulting in an increase in link transmission burden, thereby increasing the number of lost packets. The performance gap between the tasks is gradually increased because the packet loss delay of the tasks is far from the packet loss delay of the Gao Yuzi tasks.

Fig. 11 shows the total network latency for all solutions as the popularity of the content changes in the network. As can be seen from fig. 11, the overall network delay for all solutions shows a decreasing trend. This is because as the popularity of content increases, users send more requests to acquire popular content, thereby increasing the cache hit rate. Notably, the performance gap between LRFU cache policy based solutions and POP cache policy based solutions increases and decreases. Specifically, when the popularity of the content is small, the probability of the content being requested in the network is approximately the same, and no matter what cache mode is adopted, the cache hit rate is almost the same; with the gradual increase of the popularity of the content, the solution based on the POP caching strategy caches the most popular content, and the difference of the number of times different content is requested perceived by the solution based on the LRFU caching strategy is not large, so the difference of the cache hit rates of the solution based on the LRFU caching strategy and the solution based on the POP caching strategy can be gradually increased; when the popularity of the content increases to a certain extent, the difference of the request times of different contents increases, so that the solution based on the LRFU cache policy is easier to perceive popular contents, and therefore, the performance difference of the solution based on the LRFU cache policy and the solution based on the POP cache policy is gradually reduced. Furthermore, it can be seen that the performance gap between a solution with task segmentation and a solution without segmentation becomes progressively smaller. This is because as the popularity of content increases, the cache hit rate increases, thereby reducing the number of lost packets. The performance gap between the tasks is gradually reduced because the packet loss delay of the tasks is far from the packet loss delay of the Gao Yuzi tasks.

Figure 12 shows the total network delay for all solutions as the packet rate of each user changes. As can be seen from fig. 12, the overall network delay for all solutions shows an upward trend. This is because as the packet rate increases, the number of requests in the network increases. Furthermore, the performance gap between LRFU cache policy based solutions and POP cache policy based solutions is increasing. Specifically, as the number of requests increases, the gap between the cache hit rate of the LRFU cache policy based solution and the POP cache policy based solution increases, resulting in a gradual increase in the performance gap therebetween. Furthermore, it can be seen that the performance gap between the task split solution and the non-split solution is also gradually increasing. This is because as the number of requests increases, the link transmission burden in the network becomes heavier, resulting in an increase in the number of lost packets. The performance gap between the tasks is gradually increased because the packet loss delay of the tasks is far from the packet loss delay of the Gao Yuzi tasks.

Fig. 13 and 14 show the sum of the average prize values for each time slot at different learning rates based on task non-segmentation and on task segmentation, respectively. As can be seen from fig. 13 and 14, the DRL scheme at different learning rates always converges rapidly and works best when the learning rate is 0.0003. This is because a larger learning rate indicates that when the system task offload is decided, the old Q will have a stronger impact on the new Q; while a smaller learning rate indicates that the old Q will have a weaker impact on the new Q when the system makes a task offloading decision. So that the learning rate is not as large as possible, nor as small as possible, only a suitable learning rate will make the solution perform best.

Fig. 15 and 16 show the sum of the average prize values for each time slot for different buffer sizes based on task non-segmentation and based on task segmentation, respectively. As can be seen from fig. 15 and 16, the larger the cache capacity, the more popular content is cached at the base station, and the higher the cache hit rate, the lower the delay, so the average weighted reward will be larger as the cache capacity increases. Furthermore, the average weighted rewards for solutions based on task segmentation are larger than those based on task non-segmentation, because the time delay for solutions based on task segmentation is smaller than those based on task non-segmentation, as can be seen from the reward value function presented by the above described inventive embodiments, the smaller the time delay the larger the average weighted rewards.

According to the technical scheme, a cloud edge cooperative network model is built, and a task request model and a content popularity model based on task segmentation are provided; designing a routing strategy based on task segmentation; constructing a network content total time delay model and a minimum total time delay model; determining an optimal processing node and an optimal routing decision corresponding to the subtask according to the solving result of the minimum total delay model; and unloading the subtasks to the optimal processing node for processing according to the optimal routing decision, and returning the target content acquired by the optimal processing node to the user terminal. According to the intelligent unloading scheme based on task segmentation under the cloud edge cooperative network, provided by the embodiment of the invention, the total time delay of the network can be reduced under the condition of meeting the user requirement; and making an optimal unloading decision based on the DQN algorithm according to the current network state information, and realizing self-adaptive allocation of network resources, thereby improving the utilization rate of the network resources and reducing the total time delay of the network.

Example IV

Fig. 17 is a schematic structural diagram of a delay-optimized task unloading device according to a fourth embodiment of the present invention. As shown in fig. 17, the apparatus includes:

the task parsing module 41 is configured to receive at least one sub-task in the request task sent by the user terminal, and parse the target content to be acquired from the sub-task.

The model construction module 42 is configured to construct a minimum total delay model based on the cloud edge cooperative network according to the subtasks.

And the model solving module 43 is configured to determine an optimal processing node and an optimal routing decision corresponding to the subtask according to the solving result of the minimum total delay model.

And the task unloading module 44 is configured to unload the subtasks to the optimal processing node for processing according to the optimal routing decision, and return the target content acquired by the optimal processing node to the user terminal.

According to the technical scheme, at least one subtask in the request task sent by the user terminal is received through the task analysis module, and target content to be acquired is analyzed from the subtasks; the model construction module constructs a minimum total time delay model based on the cloud edge cooperative network according to the subtasks; the model solving module determines an optimal processing node and an optimal routing decision corresponding to the subtask according to the solving result of the minimum total time delay model; and the task unloading module unloads the subtasks to the optimal processing node for processing according to the optimal routing decision, and returns the target content acquired by the optimal processing node to the user terminal. According to the embodiment of the invention, the corresponding minimum total time delay model is constructed for at least one subtask of one large-scale request task under the cloud edge cooperative network, the optimal processing node and the optimal routing decision corresponding to the subtask are determined by solving the minimum total time delay model, and further the subtask is unloaded according to the optimal processing node and the optimal routing decision, so that the intelligent unloading of the request task based on task segmentation in the network can be realized, the network resource utilization rate is improved, the total time delay of the network is reduced, and the user experience is further improved.

Further, on the basis of the above embodiment of the invention, the model construction module 42 includes:

the round trip transmission delay determining unit is used for taking the ratio of the data volume of the subtask to the uplink transmission rate as the uplink transmission delay of a communication link between the user terminal and a processing node of the cloud edge cooperative network, taking the ratio of the data volume of the target content to the downlink transmission rate as the downlink transmission delay of the communication link between the processing node and the user terminal, and taking the sum of the uplink transmission delay and the downlink transmission delay as the round trip transmission delay corresponding to the subtask.

And the processing time delay determining unit is used for taking the ratio of the number of revolutions required by the central processor of the processing node for processing the subtasks to the number of revolutions of the central processor of the processing node as the processing time delay of the processing node for processing the subtasks.

The task unloading optimization function construction unit is used for determining a total delay model corresponding to the request task based on the round trip transmission delay and the processing delay, and constructing a task unloading optimization function according to the total delay model.

And the minimum total time delay model determining unit is used for determining a minimum total time delay model according to a preset task unloading optimization constraint condition and a task unloading optimization function.

Further, on the basis of the above embodiment of the present invention, the preset task unloading constraint condition includes at least one of:

all boolean variables in the total delay model have values of 1 or 0.

Further, on the basis of the above embodiment of the invention, the model solving module 43 includes:

the model input parameter determining unit is used for determining a network state of the cloud edge cooperative network in a current time slot under a subtask as a model input parameter of a minimum total time delay model, wherein the network state comprises: processing nodes, processing node types, subtasks, adjacent processing node sets, processing node buffer status, link traffic.

And the delay value determining unit is used for determining the delay value output by the minimum total delay model under the network state of the current node in the cloud edge cooperative network.

And the first rewarding value determining unit is used for determining rewarding values of the time delay values according to a preset rewarding value function.

The node judging unit is used for judging whether the rewarding value meets the preset service requirement of the subtask, if so, the current node is used as an optimal processing node, and if not, other nodes are selected as new current nodes in the cloud edge cooperative network according to a preset greedy strategy, and the network state of the new current nodes is determined.

And the second rewarding value determining unit is used for determining the rewarding value of the new current node in the network state according to the minimum total time delay model and the preset rewarding value function, and repeatedly executing the processes until the current node meeting the subtask or the subtask is determined to be lost.

And the optimal routing decision determining unit is used for determining a routing path corresponding to the optimal processing node as an optimal routing decision.

Further, on the basis of the embodiment of the invention, the preset reward value function is generated based on the proportional relation of the total consumption time delay of the subtasks in the current time slot.

Further, on the basis of the above embodiment of the present invention, the node determining unit is further configured to:

after determining the network state of the new current node, the network state is saved to the experience playback pool.

The time delay optimal task unloading device provided by the embodiment of the invention can execute the time delay optimal task unloading method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

Example five

Fig. 18 shows a schematic diagram of an electronic device 50 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.

As shown in fig. 18, the electronic device 50 includes at least one processor 51, and a memory such as a Read Only Memory (ROM) 52, a Random Access Memory (RAM) 53, etc. communicatively connected to the at least one processor 51, wherein the memory stores a computer program executable by the at least one processor, and the processor 51 can perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 52 or the computer program loaded from the storage unit 58 into the Random Access Memory (RAM) 53. In the RAM 53, various programs and data required for the operation of the electronic device 50 can also be stored. The processor 51, the ROM 52 and the RAM 53 are connected to each other via a bus 54. An input/output (I/O) interface 55 is also connected to bus 54.

Various components in the electronic device 50 are connected to the I/O interface 55, including: an input unit 56 such as a keyboard, a mouse, etc.; an output unit 57 such as various types of displays, speakers, and the like; a storage unit 58 such as a magnetic disk, an optical disk, or the like; and a communication unit 59 such as a network card, modem, wireless communication transceiver, etc. The communication unit 59 allows the electronic device 50 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunications networks.

The processor 51 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 51 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 51 performs the various methods and processes described above, such as the time-lapse optimal task offloading method.

In some embodiments, the latency optimal task offloading method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as the storage unit 58. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 50 via the ROM 52 and/or the communication unit 59. When the computer program is loaded into RAM 53 and executed by processor 51, one or more steps of the latency optimal task offloading method described above may be performed. Alternatively, in other embodiments, the processor 51 may be configured to perform the latency optimal task offloading method in any other suitable manner (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.

The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.

The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims

1. A method for latency optimal task offloading, the method comprising:

receiving at least one subtask in a request task sent by a user terminal, and analyzing target content to be acquired from the subtask;

determining an optimal processing node and an optimal routing decision corresponding to the subtask according to the solving result of the minimum total time delay model;

2. The method according to claim 1, wherein constructing a minimum total delay model based on a cloud-edge cooperative network according to the subtasks comprises:

taking the ratio of the data volume of the subtask to the uplink transmission rate as the uplink transmission delay of a communication link between the user terminal and a processing node of the cloud edge cooperative network, taking the ratio of the data volume of the target content to the downlink transmission rate as the downlink transmission delay of the communication link between the processing node and the user terminal, and taking the sum of the uplink transmission delay and the downlink transmission delay as the round trip transmission delay corresponding to the subtask;

taking the ratio of the number of revolutions required by the CPU of the subtask processed by the processing node to the number of revolutions of the CPU of the processing node as the processing time delay of the subtask processed by the processing node;

determining a total delay model corresponding to the request task based on the round trip transmission delay and the processing delay, and constructing a task unloading optimization function according to the total delay model;

And determining the minimum total time delay model according to a preset task unloading optimization constraint condition and the task unloading optimization function.

3. The method of claim 2, wherein the preset task offloading constraints include at least one of:

in any time slot, the transmitting power of any user terminal in the cloud-edge cooperative network is smaller than or equal to a first preset transmitting power threshold corresponding to the user terminal, and in any base station in the cloud-edge cooperative network, the sum of the transmitting powers of all the user terminals after power distribution is smaller than or equal to a second preset transmitting power threshold corresponding to the base station;

in any time slot, all network content data amount cached by any base station in the cloud edge cooperative network is smaller than or equal to a preset cache capacity threshold value corresponding to the base station;

In any time slot, the total calculation capacity required by any node in the cloud edge cooperative network for processing the subtasks is smaller than or equal to a preset calculation capacity threshold value corresponding to the node;

the same subtask cannot be repeatedly processed among different nodes in the cloud edge cooperative network;

a local base station in the cloud edge cooperative network processes a first subtask set of the request task, and a union set of a second subtask set of the request task processed by an adjacent processing node in the cloud edge cooperative network must form a complete request task;

the total transmission flow of any communication link in the cloud-edge cooperative network is smaller than or equal to a preset link bandwidth threshold corresponding to the communication link;

and the values of all the Boolean variables in the total time delay model are 1 or 0.

4. The method according to claim 1, wherein the determining the optimal processing node and the optimal routing decision corresponding to the subtask according to the solution result of the minimum total delay model includes:

determining a network state of the cloud edge cooperative network in a current time slot under the subtask as a model input parameter of the minimum total time delay model, wherein the network state comprises: processing nodes, processing node types, subtasks, adjacent processing node sets, processing node buffer status and link traffic;

Determining a time delay value output by the minimum total time delay model under the network state of a current node in the cloud edge cooperative network;

determining the rewarding value of the time delay value according to a preset rewarding value function;

judging whether the rewarding value meets the preset service requirement of the subtask, if so, taking the current node as the optimal processing node, if not, selecting other nodes as new current nodes in the cloud edge cooperative network according to a preset greedy strategy, and determining the network state of the new current nodes;

determining a new rewarding value of the current node in the network state according to the minimum total time delay model and the preset rewarding value function, and repeatedly executing the processes until the current node or the subtask meeting the subtask is determined to be lost;

and determining a routing path corresponding to the optimal processing node as the optimal routing decision.

5. The method of claim 4, wherein the pre-determined prize value function is generated based on a proportional relationship of a total consumed time delay of the subtasks within the current time slot.

6. The method as recited in claim 4, further comprising:

After the network state of the new current node is determined, the network state is saved to an empirical playback pool.

7. The method of claim 1, wherein the cloud-edge co-network further comprises a caching policy of at least one of: LRFU-based distributed caching strategy, offline collaborative caching strategy based on content popularity.

8. A latency optimal task offloading apparatus, the apparatus comprising:

the task analysis module is used for receiving at least one subtask in the request task sent by the user terminal and analyzing target content to be acquired from the subtask;

the model solving module is used for determining an optimal processing node and an optimal routing decision corresponding to the subtask according to the solving result of the minimum total time delay model;

9. An electronic device, the electronic device comprising:

at least one processor; and

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the latency optimal task offloading method of any one of claims 1-7.

10. A computer readable storage medium storing computer instructions for causing a processor to perform the time-lapse optimal task offloading method of any one of claims 1-7.