CN115981807A

CN115981807A - Trusted distributed computing unloading method based on block chain

Info

Publication number: CN115981807A
Application number: CN202211556120.0A
Authority: CN
Inventors: 杨磊; 唐孜
Original assignee: Hunan University
Current assignee: Hunan University
Priority date: 2022-12-06
Filing date: 2022-12-06
Publication date: 2023-04-18

Abstract

The invention discloses a block chain-based trusted distributed computing unloading method, which relates to the technical field of edge computing in a distributed environment and comprises the following steps: (1) Establishing a decentralized distributed alliance chain architecture facing the unloading of the edge computing task, (2) providing a clustering computing task unloading method aiming at the internal relevance among the subtasks of the complex computing task; (3) And reducing the influence of the block chain on the task unloading delay requirement in a chain uplink and downlink data cooperation mode. The invention provides a block chain-based trusted distributed computing offloading method for the first time, which is used for safely and reliably offloading computing tasks in an open and untrusted edge network environment. The method has the advantages that the 'credibility' characteristic of the block chain is introduced, meanwhile, the time delay influence of the block chain on calculation unloading is reduced, the gains of a task unloading demand publisher and an edge service provider are balanced as a target, and the calculation unloading time delay is minimized and the calculation unloading success rate is maximized.

Description

Trusted distributed computing unloading method based on block chain

Technical Field

The invention belongs to the technical field of edge computing in a distributed environment, and relates to a block chain-based trusted distributed computing unloading method.

Background

With the development of various key task applications of the industrial internet of things, such as the development of acquisition and analysis of data of image recognition, video streaming and the like, the internet of things equipment corresponding to the prior art often does not have enough computing power, and needs to provide supporting computing power and ultra-low delay connection for the internet of things equipment. Traditional cloud computing has been unable to meet the technical requirements facing "low latency" requirements, and computing centers gradually subside from cloud computing to edge computing. Edge computing refers to computing close to the source of things, data, and actions, with lower latency, faster response speed, and better user experience compared to traditional centralized cloud computing systems. The existing edge computing network is full of a large number of scattered and heterogeneous resources, an effective resource integration means is lacked between the network and the network, the trust problem between the resources cannot be solved, and the safety and privacy problems of computing task unloading are always important challenges of edge computing unloading. Because edge computing naturally matches the decentralized mode of blockchains, and blockchains have traceable, tamper-resistant features, many studies attempt to address security and privacy issues faced in edge computing offloading in conjunction with blockchain techniques.

The blockchain can realize the security of transactions and high consistency of data in a distributed network environment without trust, however, one of the price paid by the blockchain "trust" is performance, which is contrary to the requirement of ultra-low delay of edge calculation in the industrial internet of things scene. Taking bitcoin as an example, the blockchain can process 5 to 7 transactions per second on average, but takes 10 minutes to get 1 block, and 6 blocks to reach the transaction, and it takes 1 hour to get one transaction record. On one hand, a blockchain indiscriminately stores massive data, so that the cost is huge, on the other hand, complex computing tasks such as Gaussian elimination and the like can be represented by directed acyclic graphs, and subtasks generally have different degrees of data and time relevance, so that a plurality of limitations exist in task unloading in a distributed environment. Therefore, the block chain is applied in combination with the edge calculation, and the following defects exist: 1. how to utilize the block chain to effectively integrate various heterogeneous resources in the edge computing network; 2. the blockchain transaction is high in cost and low in efficiency, and cost reduction and efficiency improvement are needed to be carried out on the blockchain.

Disclosure of Invention

The invention provides a trusted computing task unloading system and method based on a block chain in edge computing, aiming at the defects of task unloading in the existing distributed edge computing.

The invention provides the following technical scheme: based on a distributed system model formed by a user or a task unloading demand side, an edge computing service provider or a resource provider and a alliance chain, providing an alliance chain-based uplink and downlink data cooperation method;

the user or the task unloading demand side is an agent side which needs to carry out task unloading request at the current moment; at the current moment, a plurality of users issue task unloading requests, or one user provides a plurality of unloading tasks, and the tasks may have data relevance, namely the calculation result of the first task is the input data of the second task;

the edge computing service provider or the resource provider is an agent of the edge computing server with idle resources at the current moment; a plurality of edge computing service providers exist at the current moment, or one edge computing service provider provides a plurality of edge computing servers, and all the servers are distributed in different areas;

the alliance chain is responsible for processing and recording the unloading request of a user and the resource distribution condition of an edge computing service provider, and providing distributed services including registration, authentication, unloading request and resource release through a deployed intelligent contract;

the distributed non-relational database Mongodb is a database based on distributed file storage and is used for storing data generated in the whole task unloading process, including intermediate document records; mongodb is strong in data consistency and provides data support for task unloading and coalition chain traceability;

the data store on the chain holds only a summary of important data, including: user index, online node index, transaction list or task unload record, compute digital fingerprint MD5 of file (MD 5 checksum consists of 128 bit value, which is usually expressed in hexadecimal format, consisting of 32 characters. Information digest is usually generated on a piece of information by MD5 to prevent information from being tampered), user's signature and resource node's signature; the data is stored in a block chain account book of the alliance chain, called as Ledger for short; the Ledger refers to a block chain account book and consists of a world state and a block chain, the world state stores the current value of the account book data, the current value can be frequently changed, the direct access of an application program is facilitated, all change logs causing the current world state are recorded by the block chain, the change logs cannot be changed after being written, and the history tracking is facilitated;

the data store under the chain holds fully replicated data, including: the method comprises the steps that a task list uploaded by a user, user information, user behavior records, resource node information, resource node behavior records, algorithm information and storage information of intermediate data files generated in task calculation are stored; the data are stored in the collection of the non-relational database of Mongodb, which is a table in the Mongodb database.

The intelligent contract deployed in the alliance chain is used for restricting the registration and initialization of a user or an edge computing service provider, the intelligent contract is an automatically executable program for describing a business model and interaction rules, and the intelligent contract defines the rules among different organizations; the application invokes the smart contract to generate a transaction that is recorded onto the ledger, including:

a user or an edge computing service provider registers and logs in a alliance chain according to the identity ID of the user or the edge computing service provider;

the user or the edge computing service provider signs the hash value, the timestamp and the encrypted identity information according to the private key of the user or the edge computing service provider, and stores the signed hash value, the signed timestamp and the encrypted identity information index in a current value node of the alliance chain;

the edge computing service provider updates the resource information according to the actual situation;

the edge computing service provider downloads the latest algorithm code to the specified URL to ensure the normal operation of task unloading;

and the intelligent contract respectively performs uplink of submitted records and abstract indexes, and stores complete user information and resource node information in a downlink database.

When the uninstallation application of the intelligent contract occurs, the user uploads task data to the alliance chain, and the process comprises the following steps:

a user initiates a task unloading request to a alliance chain;

the alliance chain judges whether the request of the user is reasonable according to the self intelligent contract and the current resource condition, and stores a task index contained in the request on the chain;

when the judgment result is yes, the intelligent contract in the alliance chain authorizes the user uploading task data, and the block chain link points call an authorization interface defined in the intelligent contract in the alliance chain according to authorization confirmation transaction submitted by the user to enable a task data user to obtain authorization; storing the authorized task data in a link; then sending the unloading request to a dispatcher to wait for task unloading, and performing down-link storage on other main information such as user behavior;

if the judgment result is negative, rejecting the task unloading request;

and calling an unloading algorithm in the scheduler to obtain a pre-unloading result, and performing actual unloading scheduling.

When the unloading application of the intelligent contract occurs, the edge computing service provider accesses the task data through the alliance chain, and the method comprises the following steps:

receiving an unloading request broadcasted by an alliance chain, updating the resource residual condition after the alliance chain receives the unloading, and storing the resource condition index uplink and other main information chains;

requesting the decrypted task index from the alliance chain, obtaining a designated task URL and an algorithm from a linked database according to the index, downloading a task file to the designated URL, and preloading the algorithm;

after the task file is obtained, calculating the MD5 of the file, comparing the MD5 with the information corresponding to the chain, verifying the digital signature, and judging whether the correct and complete file is received;

when the judgment result is yes, performing task calculation work, returning the calculation result to the requesting user, and waiting for the user to confirm; after the user confirms, chaining the index information of the unloading result, and storing the complete information of the unloading result in a downlink database;

and when the judgment result is negative, submitting an error report to the alliance chain, and retransmitting the transaction data after the request of decryption.

The task unloading decision method on the alliance chain is deployed in the form of an intelligent contract and comprises the following steps:

step 1, data acquisition, namely acquiring task data through Internet of things equipment by a user, uploading and unloading task information to an intelligent contract of a alliance chain, and actively updating resource conditions of all edge computing service providers by the intelligent contract;

step 2, sequencing: the intelligent contracts are classified and ordered according to the batch task information to be unloaded by a clustering method, which is called Kmeans for short, so that the internal relevance among the tasks is ensured to the maximum extent, and the unloading sequence of the batch tasks is determined;

step 3, filtering, namely traversing the tasks according to the unloading sequence, adding the ready tasks into a ready queue, and adding other tasks which are not ready into a waiting queue;

step 4, pre-binding: traversing the running queue, finding an edge server node which meets the computing resource required by the task and enables the time delay of the edge server node to be minimum for each task in the queue, and carrying out unloading pre-binding;

step 5, binding: and (4) unloading the tasks according to the pre-unloading result obtained in the step (4), and adding the tasks which fail to be unloaded into a retry queue. And when the ready queue is empty, adding the ready tasks in the waiting queue into the ready queue. Tasks in the retry queue start retry when the ready queue and the waiting queue are both empty;

and 6, repeating the step 4 and the step 5 until all the queues are empty, completing the unloading of the task, and recording the unloading result by the intelligent contract.

In the step 1, the task unloading request only uploads the task abstract information, small data volume writing of an account book on a link is performed once between a user and a alliance link, the transmission time is short enough to be ignored, and updating of a resource table maintained in the system and uploading of the task unloading request abstract information are performed simultaneously;

distributed computing offloading research commonly uses a Directed Acyclic Graph, DAG for short, to refer to a loop-free Directed Graph to represent dependencies between tasks, denoted as G = (V, E),

where V represents a set of nodes in the DAG, each node ie V representing a subtask in the running complex computing task(ii) a E represents a set of edges in the DAG, each edge E (i, j) belongs to E and represents the dependency relationship between the tasks i and j, namely the task j needs the output data of the task i as the input of the task j, the task i is called as the precursor of j, the task j is the successor of i, and the execution of one task can be started only after all the precursor tasks of the task are completed; in a DAG, tasks without predecessor tasks are called ingress tasks, tasks without successor tasks are called egress tasks, and an application allows multiple ingress and egress tasks at the same time. Suppose a user set is denoted as U = { U = { (U) } ₁ ，u ₂ ，…，u _n U, each user u _i The uploaded task set is denoted as T = { T = } ₁ ,t ₂ ，…，t _m That can represent a collection of different types of computing tasks, each t _i Are all composed of a set of vectors S _i ，C _i ，D _i ，T _i ，F _i ，VM _i Structure of describing the unloading task t _i (ii) an attribute of (d); wherein S is _i Representing a task t _i Size of uploaded data, C _i Indicating completion of task t _i Number of CPU cycles required, D _i Representing a task t _i Tolerable maximum delay, T _i Representing a task t _i Whether there is dependency, fi denotes task t _i From a set of vectors t _i1 ,t _i2 ,…,t _im Is composed of (i) }, t _ij =0or1 to characterize the computation offload task t _j Whether or not it is t _i Predecessor task of (3), VM _i For recording task t _i The result of the pre-unloading.

In the step 2, the probability of the task submitted by the same user is related, the intelligent contract determines the initial classification K of the task classification according to the elbow method, then judges whether to classify the resource node according to the K value, and classifies the task attribute T _i The intelligent contract is specifically introduced as follows as an important attribute participating in clustering:

the elbow method evaluates the quality of the K value by calculating SSE, when the K reaches the optimal clustering number, the descending amplitude of the SSE obtained by increasing the K is reduced rapidly, so as to determine the optimal clustering number K, and the calculation method of the SSE is represented as follows:

wherein, C _i Is the ith cluster, p is C _i Sample point of (1), m _i Is C _i The center of mass of;

the total number of tasks needing task unloading at present is set as alpha, the intelligent contract checks the number beta of online resource nodes, and the classification of the resource nodes has the following constraints:

when the beta is not more than alpha, the number of the resource nodes participating in the unloading is beta = beta;

when the temperature is higher than the set temperature

Then, the number of resource nodes participating in the unloading is beta = min { alpha, beta };

when the beta is larger than the alpha, the quantity of the resource nodes participating in the unloading is beta = max { alpha, beta multiplied by A }, A belongs to [0,1], the percentage of sampling from the online resource nodes is represented, the default is set to be 0.5, and the method can be customized through configuration;

the inter-class quantity adjustment has the following constraints: actualskaw = count [ allocated ] -min (count [ allocated ]), actualskaw < MaxSkaw. And the MaxSkaew is determined by configuration, the task classification defaults to 5, and the resource node classification defaults to 2.

In the step 3, the judgment task t is passed _i If there are not completed predecessor tasks, ti must wait for the task data of the predecessor task to be able to execute its own computing task.

In the step 4, the unloading pre-binding is performed through an intelligent contract, and the intelligent contract is specifically introduced as follows: task t _i Predecessor task t _j Task result data mu of _i j must be transferred to t _i Local edge server, t _i Can be executed, assuming task t _i All the predecessor tasks are executed and completed, and task result data are transmitted to the current task t _i At the computing resource node, then the predecessor task F _i To t _i Is calculated as

R _ij The method comprises the steps that the transmission rate from a resource node i to a resource node j is shown, and no data transmission delay exists when a predecessor task and a current task are on the same resource node;

pre-binding ordering is by priority, priority

Wherein it is present>

Representing the execution delay of a task on a resource node i, f _i Representing the computing power of resource node i. level is usually computed not as an integer but as a floating point number. And then obtaining a pre-binding unloading sequence according to the level ascending sequence.

In step 5, the fragmentation rate of the current resource node is calculated, and scheduling is actually performed according to the fragmentation rate and the node condition allocated in the pre-binding stage, where MemFragmentationRatio = memmallocatable/MemRequest, and when MemFragmentationRatio is greater than or equal to 1 and less than 1.6, it indicates that the current node is schedulable, and the following constraints are present in the actual unloading process:

(1) All tasks submitted by the user should be offloaded to the edge for execution;

(2) Each uploaded task is atomicity after being divided and cannot be further divided;

(3) A task is assumed to be offloaded to only one online resource node;

(4) Queue insert operations also preserve priority constraints.

In the step 5 and the step 6, the time delay of the batch task completion and the energy consumption of the edge server are always accumulated, and after all tasks are completed, the task completion rate is calculated by statistics:

the method for calculating the total time delay of batch task completion is defined as follows:

Ttotal＝Tfinal，T _total indicating the total delay for completion of the batch tasks, and the completion time of the last completed task in the batch tasksI.e. the final completion time of the batch of tasks;

edge computing server for user submitted computing task t _i The calculated energy consumption of ti is defined as:

indicating that the edge compute server is performing task t _i The energy consumption in the process is greater or less>

Indicating that the edge compute server is waiting for a task t _i Idle energy consumption in the process;

the total energy consumption for unloading the batch tasks of the edge computing server is defined as follows:

E _ij representing the unloading task j uploaded by the user i;

the task completion rate is defined as:

F _ij indicating whether the unloading task j uploaded by the user i is in D _i Internal completion of F _ij And =0or1, wherein alpha represents the total number of tasks needing task unloading at this time.

The invention has the advantages and beneficial effects that:

(1) The invention discloses a credible distributed computation unloading method based on a block chain, which fuses the block chain technology and the edge computation unloading technology, solves the traceability problem of resource security sharing and computation unloading process by utilizing a credible access control mechanism of the block chain, realizes the interaction mode of data collaboration under the distributed chain, solves the problem of time delay influence of huge overhead of the block chain on edge computation to a certain extent, and has important significance for computation unloading performance guarantee;

(2) According to the credible distributed computing unloading method based on the block chain, the task dependence problem is taken into consideration as a core problem, the optimal unloading sequence and the unloading strategy of the computing task unloading are obtained through a clustering algorithm, and the overall unloading time delay of a complex computing task and the computing operation energy consumption of an edge server are reduced.

Drawings

For the purpose of making the objects, aspects and advantages of the present invention more apparent, the invention will be described in detail below with reference to the accompanying drawings, in which:

FIG. 1 is a block chain-based system model for distributed computing task offloading,

FIG. 2 is a diagram of a computation task offloading framework based on downlink-uplink data coordination,

figure 3 is a diagram of a priority queue design based on a clustering method,

FIG. 4 is a flow diagram illustrating federation chain-based offloading of distributed computing tasks.

Detailed Description

The embodiments of the present invention are illustrated below by specific examples. Other advantages and effects of the present invention will be readily apparent to those skilled in the art from the disclosure herein. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict. The invention will be described in more detail below with reference to the accompanying figures 1 to 4:

based on a distributed system model formed by a user or a task unloading demand side, an edge computing service provider or a resource provider and a alliance chain, providing an alliance chain-based uplink and downlink data cooperation method;

the edge computing service provider or the resource provider is an agent of the edge computing server with idle resources at the current moment; at the present moment, a plurality of edge computing service providers exist, or one edge computing service provider provides a plurality of edge computing servers, and all the servers are distributed in different areas;

the data store on the chain holds only a summary of important data, including: user index, online node index, transaction list or task unload record, compute digital fingerprint of file MD5 (MD 5 checksum consists of 128 bit value, which is usually expressed in hexadecimal format and consists of 32 characters.a message digest is usually generated for a piece of message by MD5 to prevent the message from being tampered with.), user's signature and resource node's signature; the data is stored in a block chain account book of the alliance chain, called as Ledger for short; the Ledger refers to a block chain account book and consists of a world state and a block chain, the world state stores the current value of account book data, the current value can be frequently changed and is convenient for an application program to directly access, and the block chain records all change logs causing the current world state, can not be changed after being written in and is convenient for history tracking;

the data store under the chain holds fully replicated data, including: the method comprises the steps that a task list uploaded by a user, user information, user behavior records, resource node information, resource node behavior records, algorithm information and storage information of intermediate data files generated in task calculation are obtained; the data are stored in the collection of the non-relational database of Mongodb, which is a table in the Mongodb database.

the user or the edge computing service provider signs the hash value, the timestamp and the encrypted identity information according to the private key of the user or the edge computing service provider, and stores the signed hash value, the signed timestamp and the encrypted identity information index in the current value node of the alliance chain;

a user initiates a task unloading request to a alliance chain;

when the judgment result is yes, the intelligent contract in the alliance chain authorizes the user uploading task data, and the block chain link points call an authorization interface defined in the intelligent contract in the alliance chain according to authorization confirmation transaction submitted by the user to enable a task data user to obtain authorization; storing the authorized task data in a link; then sending an unloading request to a scheduler to wait for task unloading, and performing down-link storage on other main information such as user behaviors and the like;

if the judgment result is negative, rejecting the task unloading request;

receiving an unloading request broadcasted by an alliance chain, updating the resource residual condition after the resource residual condition is unloaded by the alliance chain, and storing the resource condition index uplink and other main information links;

when the judgment result is yes, performing task calculation work, returning the calculation result to the requesting user, and waiting for the user to confirm; linking the index information of the unloading result after the user is confirmed, and storing the complete information of the unloading result in a downlink database;

and when the judgment result is negative, submitting an error report to the alliance chain, and retransmitting the decrypted transaction data to the request.

The intelligent contract ensures that the two parties unload the task on the chain in a cooperative mode when the task is unloaded, and the task plaintext is transmitted under the chain. The mode of chain uplink and downlink cooperation enables the task data to be isolated from the P2P network of the block chain, and the task computing file and the unloading process can be verified with the on-chain transaction information, so that the task data is prevented from being leaked, the block chain cost is reduced, the block chain operation efficiency is improved, and the balance among cost, efficiency and privacy safety is achieved.

step 3, pre-binding: traversing the running queue, finding an edge server node which meets the computing resource required by the task and enables the time delay of the edge server node to be minimum for each task in the queue, and carrying out unloading pre-binding;

step 4, pre-binding: traversing the running queue, finding out an edge server node which meets the computing resource required by the task and enables the time delay of the edge server node to be minimum for each task in the queue, and carrying out unloading pre-binding;

wherein V represents a set of nodes in the DAG, and each node i belongs to V and represents a subtask in the running complex computing task; e represents the set of edges in the DAG, each edge E (i, j) belongs to E and represents the dependency relationship between the tasks i and j, namely the task j needs the output data of the task i as the input of the task j, the task i is called as the precursor of j, the task j is the successor of i, and one task can be executed only after all the precursor tasks are completed; in a DAG, tasks without predecessor tasks are called ingress tasks, tasks without successor tasks are called egress tasks, and an application allows multiple ingress and egress tasks at the same time.

Suppose that the user set is represented as U = { U = ₁ ，u ₂ ，…，u _n U, each user u _i The uploaded task set is denoted as T = { T = } ₁ ,t ₂ ，…，t _m That can represent a collection of different types of computing tasks, each t _i Are all composed of a set of vectors S _i ，C _i ，D _i ，T _i ，F _i ，VM _i Structure of describing the unloading task t _i (ii) an attribute of (d); wherein S is _i Representing a task t _i Size of uploaded data, C _i Indicating completion of task t _i Number of CPU cycles required, D _i Representing a task t _i Tolerable maximum delay, T _i Representing a task t _i Whether there is dependency, fi denotes task t _i From a set of vectors t _i1 ,t _i2 ,...,t _im Formation of t _ij =0or1, to characterize the computation offload task t _j Whether or not it is t _i Predecessor task of (1), VM _i For recording tasks t _i The result of the pre-unloading.

the elbow method evaluates the quality of the K value by calculating SSE, when K reaches the optimal clustering number, the descending amplitude of the SSE obtained by increasing K is reduced rapidly, so as to determine the optimal clustering number K, and the calculation method of the SSE is represented as follows:

wherein, C _i Is the ith cluster, p is C _i Sample point of (1), m _i Is C _i A centroid of (a);

the total number of tasks needing task unloading at present is set as alpha, the intelligent contract is used for detecting the number beta of online resource nodes, and the classification of the resource nodes has the following constraints:

when in use

when the beta is larger than the alpha, the quantity of the resource nodes participating in the unloading is beta = max { alpha, beta multiplied by A }, A belongs to [0,1], the percentage of sampling from the online resource nodes is represented, the default is set to 0.5, and the method can be customized through configuration;

In the step 4, the unloading pre-binding is performed through an intelligent contract, and the intelligent contract is specifically introduced as follows: task t _i Predecessor task t _j Task result data mu of _ij Must be transmitted to t _i Local edge server, t _i Can be executed, assuming task t _i All the predecessor tasks are executed and completed, and task result data are transmitted to the current task t _i At the computing resource node, then the predecessor task F _i To t _i Is calculated as

R _ij Representing the transmission rate from the resource node i to the resource node j, and assuming that no data transmission delay exists when the precursor task and the current task are on the same resource node;

pre-binding ordering is by priority, priority

Wherein it is present>

Representing the execution delay of a task on a resource node i, f _i Representing the computing power of resource node i.

(2) Each uploaded task is atomicity after being segmented and cannot be further segmented;

(3) A task is assumed to be offloaded to only one online resource node;

(4) Queue insert operations also preserve priority constraints.

In the step 5 and the step 6, the time delay of the batch task completion and the energy consumption of the edge server are always accumulated, and after all the tasks are completed, the task completion rate is calculated by statistics:

Ttotal＝Tfinal，T _total the total time delay for completing the batch tasks is shown, and the completion time of the last completed task in the batch tasks is the final completion time of the batch tasks;

E _ij representing the unloading task j uploaded by the user i;

the task completion rate is defined as:

The invention has the beneficial effects that: (1) A data cooperation method for on-chain evidence storage and under-chain data transmission is provided, and the block chain storage overhead is reduced. (2) The clustering method is utilized to classify the tasks and the resources, so that similar tasks can be better unloaded to proper resource nodes, the transmission waiting time between the tasks is reduced through an unloading scheduling strategy, and the time delay of the completion of the whole task is prolonged. (3) Data storage and unloading strategies are automatically executed through intelligent contracts, and the non-tampering, visibility and traceability of the whole unloading process are guaranteed.

The detailed embodiment provides a block chain-based trusted distributed computing task unloading method, which comprises the following steps:

1) Establishing a decentralized distributed alliance chain architecture facing the unloading of edge computing tasks;

2) Providing a clustering task unloading method aiming at the problem that the complex computing task subtasks have internal association;

3) And reducing the influence of the block chain on the task unloading delay requirement in a chain uplink and downlink data cooperation mode.

As shown in fig. 1, is a system model consisting of a user (task off-load demander), an edge computing facilitator (resource provider), and a federation chain. And the edge end and the user side perform data transmission through the Internet.

The user is an agent party which needs to carry out task unloading requests at the current moment, one user may be composed of a plurality of Internet of things devices, a plurality of unloading tasks are provided, and time sequence relevance may exist among the tasks. The user side utilizes various sensors to collect task data, and the task data comprises various sensors, a camera, collection hardware and the like. The devices of the internet of things often have limited computing power and cannot process collected data. It is therefore necessary to offload tasks to devices with computing power and a certain amount of storage space.

The edge computing service provider is an agent of the edge computing server with idle resources at the current moment, one edge computing service provider can provide a plurality of edge computing servers, various services are carried on the servers, and the servers of the same edge computing service provider can be distributed in different regions. A federation chain and distributed mongoDB database are also deployed on server nodes provided by the edge computing service.

The alliance chain is responsible for processing and recording unloading requests of users and resource distribution conditions of edge computing service providers, and distributed services including registration, authentication, unloading requests and resource distribution are provided through deployed intelligent contracts. Because the alliance chain is only carried on the resource node, the operation performance of the block chain can be greatly guaranteed. The user side accesses the system through the front-end interaction interface, and the user side equipment does not carry the alliance chain, so that the use experience of the user can be greatly guaranteed.

As shown in fig. 2, the task offloading decision framework is a task offloading decision framework based on a federation chain, which is composed of a plurality of modules such as an interactive interface, user management, resource node management, task management, a timer, offloading policy management, a scheduler, and a database. The following is a detailed description of each module:

1. interaction interface

The user can conveniently and visually issue the task unloading request, upload the task information and inquire the task unloading result. A user uploads complete information of a task on an interactive interface;

due to the existence of the interactive interface, a user does not need to build a block chain at the equipment end with great effort under the condition that the equipment capacity is limited, and the task unloading request is carried out through the interface, so that the equipment can only concentrate on the data acquisition process of the user. As long as a network exists, the user can share the block chain unloading service;

2. user management

The invention only has basic user management functions, including user registration, login verification and logout. After the user is registered and authenticated, the system stores a user unique identifier UserId, a user name UserName, a user password UserPwd, a user private key UserSK and a user public key UserPK in a down-link mode, and stores a user unique identifier in an up-link mode;

the user management module can be conveniently expanded according to actual use scenes, such as adding a role-permission corresponding relation and the like; 3. timer

The intelligent contract storage system is a callback function, the function is called once every other time, and the function is used for automatically calling the intelligent contract and the storage API program at intervals. The method comprises program modules for updating resources, updating tasks, collecting tasks, updating unloading results and the like. The time of each calling interval is millisecond, the default setting is 1000ms, and the calling interval can be customized. When the task flow in the system is too large, an ultra-low interval, for example 10ms, can be set;

4. resource node management

The invention only has basic resource node management functions, including registration, login verification and logout of the edge computing service provider, registration, online verification and logout of the resource node. After the edge computing service provider is registered and authenticated, the system stores the user unique identifier, the user name, the user password, the user private key and the user public key under a link, and stores the user unique identifier on a link. After the resource node of the edge computing service provider is registered and authenticated, the system stores the unique identifier of the edge computing service provider, the unique identifier NodeId of the resource node, a public private key NodeSK/NodePK of the resource node, the longitude NodeLon of the resource node, the latitude NodeLat of the resource node, the CPU period NodeResource of the resource node, the transmission rate NodeDTR of the resource node, the computing processing capability NodeMIPS of the resource node and the energy consumption NodeEnergy of the resource node under a link, and stores the unique identifier of the resource node in a uplink manner;

an online resource node table is maintained in the federation chain, wherein the online resource node unique identifier is stored in the table, and the resource information is obtained from the MongoDB. In addition, a network graph is maintained, the concept of the introduced edge stores the edge network structure, and the edge set { EdgeId, srcId, dstId, length, delay, bandwidth } is used to describe the attribute of the edge. The EdgeId represents the unique identifier of the edge, srcId represents the source of the edge, dstId represents the end point of the edge, srcId and DstId are both resource node IDs, and an edge represents that two resource nodes can directly communicate with each other. Length represents the actual physical distance of the edge, delay represents the transmission Delay between SrcId and DstId, delay is not always available at the beginning, the average value is kept through continuous test updating, the approximate transmission Delay can be estimated through Delay, and the calculation in the pre-binding process is greatly facilitated. Bandwidth represents the transmission Bandwidth of the edge, and likewise, bandwidth is not originally available, but is updated through constant testing, and the average value is kept;

5. task management

The method supports the creation and management of the computing task, and supports the starting and stopping of the computing task. The task information includes: task unique identifier TaskId, the number of CPU cycles requested by the task tasksource, task size TaskSize, task maximum tolerable time tasdeadline, task type TaskType (whether the task is a dependent task), task predecessor TaskFathers of the task, link taskfurl of the task file. The task unique identifier and the user unique identifier will be stored uplink (stored in the federation chain ledger) while other information is stored downlink (stored in the distributed MongoDB database); before creating the task, algorithm information needs to be created, and the part is explained in unloading policy management. After the task is established, the task state switch is closed by default, and the running task cannot be edited or deleted. The task includes the following three states:

(1) the task is ready: the predecessor tasks of the tasks are completed, but the tasks are not operated yet and wait for operation;

(2) and (3) task operation: in the process of task execution;

(3) task retry: when the task execution fails, the task is added into a retry queue;

in particular, task dependencies are represented by a DAG graph, a priori. The task dependence needs to be obtained in advance by a user through other task segmentation tools, and the task segmentation is not carried out;

6. policy management

And the acquisition of an algorithm and the selection of a clustering mode are supported. The workflow of policy management is as follows:

(1) acquiring a clustering algorithm according to the preference of a user, judging the cluster number of task clusters, and classifying the tasks;

(2) and judging whether to classify the resource nodes according to the task classification condition, and if so, sampling and adjusting according to classification constraint. If the number of tasks in one task cluster is far larger than that of the resource nodes after the resource nodes are classified, the task unloading effect is very poor, so that the number between classes needs to be adjusted, each resource node cluster serves a proper task cluster, and the overall completion delay of batch tasks can be reduced to the greatest extent. The method for adjusting the number between classes is illustrated as follows:

suppose 10 tasks are classified into 3 clusters, maxshiew =1, and the classification case is <2,3,5>, i.e. 2 tasks in the first cluster, 3 tasks in the second cluster, and 5 tasks in the third cluster. And calculating the value of the actualskaew as (0, 1, 3), if the constraint condition of the quantity among the classes is not met, adjusting the redundant tasks of the third cluster to the first cluster, wherein the value of the actualskaew is (0, 1), and the constraint condition is met. The adjustment of the quantity among the classes is finished, and the adjustment of the resource nodes is the same;

7. scheduler

And (3) carrying out actual scheduling, including the working flows of sequencing, filtering, pre-binding, binding and the like:

(1) sorting

The intelligent contracts are classified and ordered by a clustering method Kmeans according to the batch task information to be unloaded, so that the internal relevance among the tasks is ensured to the maximum extent, and the unloading sequence of the batch tasks is determined. When Kmeans performs clustering, multidimensional clustering is performed, and task attribute { S }must be set _i ，C _i ，D _i ，T _i ，F _i ，VM _i S in (b) } _i 、C _i 、D _i 、T _i And constructing a space system by taking the space system and the UserId as vectors at the same time, and calculating the distance. Because the representation methods of the attribute columns are different, before clustering, normalization processing needs to be carried out on the attributes, the weight of each index needs to be considered during normalization, namely the parameter used during actual normalization is W ₁ ×S _i 、W ₂ ×C _i And by analogy, unifying all attributes participating in clustering. In addition, outlier or noise data has a large influence on the mean, resulting in a center shift, and therefore we also need outlier detection of the data. In the invention, the data of the point closest to the detected abnormal point is selected for substitution, and the method can be improved; when the data are preprocessed, obtaining the cluster number K by using an elbow method, and randomly selecting K samples as initial clustering centers a = a ₁ ，a ₂ ，…a _k . For each sample X in the dataset _i And calculating the distances from the cluster centers to the K cluster centers and classifying the cluster centers into the class corresponding to the cluster center with the minimum distance. For each class a _j Recalculating its cluster centers by

C _i Is the ith cluster. And repeating the operation of calculating the cluster center until the minimum error is less than 0.1, namely stopping clustering.

The result obtained after clustering is shown as the task cluster in fig. 3, green represents a ready task, and orange represents a waiting task;

(2) filtration

Traversing the tasks according to the unloading sequence, adding the ready tasks into the ready queue, adding other tasks which are not ready into the waiting queue, and performing filtering processes like three queues shown in FIG. 3, inserting new tasks from the tail of the queue according to the operation of the queues, and taking out the ready tasks from the head of the queue. Red indicating that a task that failed execution is waiting for retry;

task readiness must satisfy two conditions: the task data is acquired, predecessor tasks of the task are completed, and task result data of the predecessor tasks are confirmed. After the scheduler judges that the task is ready, the task state is changed to be started;

(3) pre-binding

Traversing the running queue for the task t in the queue _i And finding an edge server node which meets the computing resources needed by the task and enables the time delay of the edge server node to be minimum, and carrying out unloading pre-binding. And calculating to obtain priority level through an intelligent contract, and sequencing according to the ascending order of the level to obtain a pre-binding unloading order. level is essentially the task t _i And the complete unloading time of the node k of the pre-bound resource comprises the steps of acquiring task data, executing the task and returning a task result. The method is a pre-reasoning method which is carried out according to the existing resource condition (a data matrix D and a communication cost matrix B);

the data matrix D is a matrix of α × α, D _ij For representing tasks t _i And task t _j The amount of data that needs to be transmitted. Task t _i And task t _j And when the same online resource node is used, the data volume needing to be transmitted is 0. The communication cost matrix B is a matrix of beta x beta, B _ij For representing the communication cost of online resource node i to online resource node j. In the pre-binding stage, the communication cost between tasks is marked by average communication cost;

it should be noted that all task data are acquired in parallel, and the documents of the calculation results are often much smaller than the original calculation data, so the return time of the calculation result data can be relatively ignored during modeling, and thus the task t is executed _i Task data communication cost calculation method required by readinessIs composed of

(4) Binding

And unloading the tasks according to the pre-unloading sequence, and adding the tasks which fail to be unloaded into a retry queue. And when the ready queue is empty, adding the ready tasks in the waiting queue into the ready queue. Tasks in the retry queue start retry when the ready queue and the waiting queue are both empty;

the binding phase must follow the constraint of fragmentation rate: for example, when MemFragmentationRatio<1, assuming that available CPU resources of the prebinding resource node are 1000, task t _i In fact, the applied CPU resource is 1200, which is equivalent to that the current node is in overload operation, and the actual speed of task execution is reduced. When MemFragmentationRatio > 1.6, assume that the available CPU resource of the prebinding resource node is 1000, task t _i Actually, the applied CPU resource is 600, and because the current node is exclusively operated, the utilization rate of the CPU resource is low, which reduces the operating efficiency of the whole system;

when the intelligent contract judges that the unloading scheduling meets the constraint of the fragment rate, the task is unloaded, the system time when the current task starts and the system time when the task ends are recorded, and the completion time of the task is recorded as the system time when the task ends. The task execution time is recorded as the difference t between the system time when the task ends and the system time when the task starts _f 。t _f The method is used for calculating the energy consumption of the whole system, and because the energy consumption of the server during operation is difficult to measure, a great deal of research is carried out on the work of measuring the energy consumption of the operation;

indicating that the edge compute server is performing task t _i In the process of (1), energy is supplied to the operation, and>

representing the average power consumption of the CPU in operation, counted by the edgeThe calculation service provider provides the information when uploading the resource node information; />

Indicating that the edge compute server is performing task t _i Idle energy consumption in process, based on the number of hours or hours>

The average energy consumption in the idle state of the CPU is represented and is provided by the edge computing service provider when uploading the resource node information;

the execution cost of the system to complete a tasks can be expressed as:

U＝max{λ ₀ ×T _total ×Cost ₀ ， ₁ λ×E _total ×Cost ₁ }; wherein λ is ₀ ，λ ₁ Respectively, scalar weights are expressed and represent the preference of unloading, namely the relativity of the system which tends to reduce the task execution delay of a user or reduce the energy consumption of an edge computing service provider. Cost ₀ Representing a unit price, cost, of the rented edge server ₁ Representing the unit price of energy purchased by the edge computing service.

8. Database with a plurality of databases

The method is used for realizing the data storage on the chain and the data storage under the chain, and establishing a table for all data structures and storing the table in the MongoDB. At least seven data tables are needed to maintain the uplink-downlink cooperative MongoDB of the system chain: users, tasks, nodes, edges, algorithms, overflowing-Orders, task-List. In the alliance chain account book, only five tables of Users, tasks, nodes, algorithms and Offloating-Orders need to be correspondingly maintained, and the unique identifiers are stored in a key-value group. The mongoDB database and the ledger book can be expanded according to the actual situation of the system.

The invention relates to a block chain-based trusted distributed computing unloading method, which comprises two parts of unloading decision and resource allocation, which are collectively called task unloading decision, wherein the unloading decision comprises three modes of local execution, complete unloading and partial unloading, and the unloading in the method refers to complete unloading; the resource allocation mainly selects which edge resource node to offload, and more specifically, may also specify which processor to execute, which is included in the task offloading in the present invention.

As shown in fig. 4, the process of task offload issue and offload decision making by the method of the present invention is as follows:

and task unloading request issuing: including 1-6 in fig. 4. And (3) the user issues a specific task unloading request to the alliance chain, and the alliance chain checks whether the unloading request can be responded or not, which corresponds to 1-3. If yes, the user uploads the task information, and the database updates the task list, corresponding to 5-6. Any user can be in the role of a task offload requester.

Task offloading decision: including 7-9 in fig. 4. For each task, the alliance chain requires that the online resource node return the residual resource amount after executing the task in advance before executing the task, so as to make the next task unloading decision, which corresponds to 7-8. And then, executing an intelligent contract for task unloading by the online resource node, giving a result range to the alliance chain by the online resource node which obtains a pre-unloading result most quickly, and starting to actually execute task unloading, wherein the result range corresponds to 8-9.

Task deployment: including 10 in fig. 4. After the pre-unloading result is determined, the task unloading request user sends data required by task computing to the designated task unloading computing execution node, and task deployment is directly executed by the user in contact with the designated task unloading computing execution node chain without passing through the alliance chain.

And (3) task unloading execution: including 10-12 in fig. 4. After the task is deployed, the task unloading calculation execution node completes the task, can calculate the actual execution time delay and execution energy consumption based on the current performance parameters of the node, and returns the result to the alliance chain and the task unloading request user.

Task offload result uplink: including 13-14 in fig. 4. And after the user checks the executed task result, the alliance chain records information such as the unloading order id, the task id, the user id, the online resource node id and the like to the chain.

And completely storing the task unloading result: including 15-16 in fig. 4. And after the task unloading result is linked up, the database records information such as the unloading order id, the task id, the user id, the online resource node id, the task starting time, the task ending time, the algorithm used by the task, the task execution cost and the like into the down-link distributed database, so that the down-link distributed database is convenient to search in the future.

In conclusion: the invention discloses a block chain-based trusted distributed computing unloading method which comprises the steps of firstly, establishing a decentralized distributed alliance chain architecture facing edge computing task unloading, secondly, providing a clustering task unloading method aiming at internal relevance among subtasks of complex computing tasks, and finally, flexibly using a data cooperation mode under a chain uplink to reduce the influence of a block chain on task unloading time delay requirements.

Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still make modifications to the technical solutions described in the foregoing embodiments, or make equivalent substitutions and improvements to part of the technical features of the foregoing embodiments, and any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A block chain-based trusted distributed computing offloading method is characterized in that:

the user or the task unloading demand side is an agent side which needs to carry out task unloading request at the current moment; at the current moment, a plurality of users issue task unloading requests, or one user provides a plurality of unloading tasks, and the tasks may have data relevance, namely the calculation result of the first task is the input data of the second task; the edge computing service provider or the resource provider is an agent of the edge computing server with idle resources at the current moment; a plurality of edge computing service providers exist at the current moment, or one edge computing service provider provides a plurality of edge computing servers, and all the servers are distributed in different areas;

the data store on the chain holds only a summary of important data, including: user indexes, online node indexes, transaction lists or task unloading records, digital fingerprints MD5 of computed files, signatures of users and signatures of resource nodes; the data is stored in a block chain account book of the alliance chain, called as Ledger for short;

2. The block chain-based trusted distributed computing offload method according to claim 1, wherein:

intelligent contracts deployed by federation chains are used to constrain the registration and initialization of users or edge computing service providers, including:

the edge computing service provider downloads the latest algorithm code to the specified URL to ensure the normal operation of task unloading; and the intelligent contract respectively performs uplink of submitted records and abstract indexes, and stores complete user information and resource node information in a downlink database.

3. The block chain-based trusted distributed computing offload method according to claim 2, wherein:

a user initiates a task unloading request to a alliance chain;

when the judgment result is yes, the intelligent contract in the alliance chain authorizes the user uploading task data, and the block chain link points call an authorization interface defined in the intelligent contract in the alliance chain according to authorization confirmation transaction submitted by the user to enable a task data user to obtain authorization; storing the authorized task data in a linked mode; then sending the unloading request to a dispatcher to wait for task unloading, and performing down-link storage on other main information such as user behavior;

if the judgment result is negative, rejecting the task unloading request;

calling an unloading algorithm in a scheduler to obtain a pre-unloading result, and performing actual unloading scheduling;

4. The block chain-based trusted distributed computing offloading method of claim 1, wherein the task offloading decision method on the federation chain comprises the following steps:

step 1, data acquisition, namely acquiring task data through Internet of things equipment by a user, uploading and unloading task information to an intelligent contract of a union chain, and actively updating resource conditions of all edge computing service providers by the intelligent contract;

step 5, binding: and (5) unloading the tasks according to the pre-unloading result obtained in the step (4), and adding the tasks which fail to be unloaded into a retry queue. And when the ready queue is empty, adding the ready tasks in the waiting queue into the ready queue. Tasks in the retry queue start retry when the ready queue and the waiting queue are both empty;

and 6, repeating the step 4 and the step 5 until all the queues are empty, completing unloading of the task, and recording the unloading result by the intelligent contract.

5. The block chain-based trusted distributed computing offloading method of claim 4, wherein in step 1, the offloading task request only uploads the task summary information, a small data volume of the account book on the chain is written once between the user and the federation chain, the transmission time is so short that it can be ignored, and the updating of the resource table maintained in the system and the uploading of the task offloading request summary information are performed simultaneously;

distributed computation offload research commonly uses Directed Acyclic Graph (DAG), which refers to a loop-free Directed Graph to represent dependency relationships between tasks, and is denoted as G = (V, E), where V represents a set of nodes in the DAG, and each node i belongs to V and represents a subtask in a running complex computation task; e represents a set of edges in the DAG, each edge E (i, j) belongs to E and represents the dependency relationship between the tasks i and j, namely the task j needs the output data of the task i as the input of the task j, the task i is called as the precursor of j, the task j is the successor of i, and the execution of one task can be started only after all the precursor tasks of the task are completed;

set of users expressed as U = { U = { [ U ] ₁ ，u ₂ ，…，u _n U, each user u _i Uploaded task collections expressed as T = { T = ₁ ,t ₂ ，…，t _m That can represent a set of different types of computational tasks, each t _i Are all composed of a set of vectors S _i ，C _i ，D _i ，T _i ，F _i ，VM _i Structure of describing the unloading task t _i (ii) an attribute of (d); wherein S is _i Representing a task t _i Size of uploaded data, C _i Indicating completion of task t _i Number of CPU cycles required, D _i Representing a task t _i Tolerable maximum delay, T _i Representing a task t _i Whether there is dependency, fi denotes task t _i From a set of vectors t _i1 ,t _i2 ,…,t _im Is composed of (i) }, t _ij =0or1, to characterize the computation offload task t _j Whether or not it is t _i Predecessor task of (1), VM _i For recording tasks t _i The result of the pre-unloading.

6. The block chain-based trusted distributed computing offloading method of claim 4, wherein in step 2, it is assumed that the task probabilities submitted by the same user are related, the intelligent contract determines the initial category K of task classification according to the elbow method, determines whether to classify the resource node according to the K value, and classifies the task attribute T _i The intelligent contract is specifically introduced as follows as an important attribute participating in clustering:

when the temperature is higher than the set temperature

the inter-class quantity adjustment has the following constraints: actualskaew = count [ allocated ] -min (count [ allocated ]), actualskaew < maxskaew. And the MaxSkaew is determined by configuration, the task classification defaults to 5, and the resource node classification defaults to 2.

7. The method according to claim 4, wherein in step 3, by determining task t, the trusted distributed computing offload method based on block chain is performed _i If there are not completed predecessor tasks, ti must wait for the task data of the predecessor task to be able to execute its own computing task.

8. The method for offloading distributed computing based on block chaining according to claim 4, wherein in step 4, the offloading is pre-bound by an intelligent contract, and the intelligent contract is specifically introduced as follows:

task t _i Predecessor task t _j Task result data mu of _ij Must be transmitted to t _i Local edge server, t _i Can be executed, assuming task t _i All the precursor tasks are executed and completed, and the task result data is transmitted to the current task t _i At the computing resource node, then the predecessor task F _i To t _i Is calculated as

pre-binding ordering is by priority, priority

Wherein it is present>

9. The method according to claim 4, wherein in step 5, the fragmentation rate of the current resource node is calculated, and the scheduling is actually performed according to the fragmentation rate and the node condition allocated in the pre-binding stage,

MemFragmentationRatio＝MemAllocatable/MemRequest，

when MemFragmentationRatio is more than or equal to 1 and less than 1.6, the current node is schedulable, and the actual unloading process has the following constraints:

(1) All tasks submitted by the user should be offloaded to the side for execution;

(3) A task is assumed to be offloaded to only one online resource node;

(4) Queue insert operations also preserve priority constraints.

10. The method according to claim 4, wherein in steps 5 and 6, the time delay for completing the batch task and the energy consumption of the edge server are accumulated all the time, and after all tasks are completed, the task completion rate is calculated by statistics:

T _total ＝T _final ，T _total the total time delay for completing the batch tasks is shown, and the completion time of the last completed task in the batch tasks is the final completion time of the batch tasks;

E _ij representing the unloading task j uploaded by the user i;

the task completion rate is defined as:

F _ij indicating whether the unloading task j uploaded by the user i is in D _i Internal completion of F _ij And =0or1, wherein α represents the total number of tasks requiring task unloading. />