CN111328107B - Multi-cloud heterogeneous mobile edge computing system architecture and energy optimization design method - Google Patents

Multi-cloud heterogeneous mobile edge computing system architecture and energy optimization design method Download PDF

Info

Publication number
CN111328107B
CN111328107B CN202010069937.XA CN202010069937A CN111328107B CN 111328107 B CN111328107 B CN 111328107B CN 202010069937 A CN202010069937 A CN 202010069937A CN 111328107 B CN111328107 B CN 111328107B
Authority
CN
China
Prior art keywords
action
cloud
mobile edge
edge computing
task
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010069937.XA
Other languages
Chinese (zh)
Other versions
CN111328107A (en
Inventor
宋令阳
张雨童
邸博雅
边凯归
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peking University
Original Assignee
Peking University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peking University filed Critical Peking University
Priority to CN202010069937.XA priority Critical patent/CN111328107B/en
Publication of CN111328107A publication Critical patent/CN111328107A/en
Application granted granted Critical
Publication of CN111328107B publication Critical patent/CN111328107B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W28/00—Network traffic management; Network resource management
    • H04W28/02—Traffic management, e.g. flow control or congestion control
    • H04W28/08—Load balancing or load distribution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5072—Grid computing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/502—Proximity
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

The application provides a design method, equipment and a storage medium for a multi-cloud heterogeneous mobile edge computing system architecture and energy optimization, relates to the technical field of cloud data, and aims to solve the problem that a plurality of cloud platforms compete for computing resources and transmission resources of edge equipment and heterogeneous mobile edge servers in the multi-cloud heterogeneous mobile edge computing system. The multi-cloud heterogeneous mobile edge computing system architecture is a three-layer network structure consisting of an edge device layer, a mobile edge computing server layer and a cloud platform layer. Determining a plurality of joint action pairs containing the most probable action in the joint action pair matrix; determining an optimal combined action pair according to the accumulated reward values of the plurality of combined action pairs; determining the coupling action to be executed by the target cloud platform in the optimal combined action pair as a coupling action to be executed; acquiring an energy value consumed by the target cloud platform to execute the coupling action to be executed, and acquiring the coupling actions simultaneously executed by other cloud platforms; the cumulative prize value of the joint action pair is performed.

Description

Multi-cloud heterogeneous mobile edge computing system architecture and energy optimization design method
Technical Field
The application relates to the technical field of cloud data, in particular to a method, equipment and storage medium for designing a multi-cloud heterogeneous mobile edge computing system architecture and energy optimization.
Background
With the development of the internet of things, the core network computing faces huge network flow pressure and computing pressure, so that the mobile edge computing can provide multiple purposes and reduce the core network flow pressure and the computing pressure.
Existing mobile edge computing networks are broadly divided into: the edge device and the mobile edge computing server (MEC server) form a two-layer network structure, and the single cloud platform, the edge device and the mobile edge computing server (MEC server) form a three-layer network.
Wherein, the computing resources of the two-layer network structure are not fully utilized, and the computing power of the mobile edge computing server (MEC server) is not enough to support the computing intensive tasks; the three-layer network structure does not consider an actual application scenario, that is, the mobile application is generally distributed and deployed on a plurality of mutually independent cloud platforms, the plurality of mutually independent cloud platforms are respectively connected with the mobile edge computing server (MEC server), and mutually rob computing resources and transmission resources of the mobile edge computing server (MEC server) and edge devices, so that resource allocation in the multi-cloud heterogeneous mobile edge computing system is uneven, and unnecessary energy consumption is caused by resource redundant allocation.
Disclosure of Invention
The embodiment of the application provides a method, equipment and a storage medium for designing a multi-cloud heterogeneous mobile edge computing system architecture and energy optimization, which aim to solve the problem that resource utilization by mobile edge computing is insufficient, solve the problem that a plurality of cloud platforms compete for computing resources and transmission resources of edge equipment and a mobile edge computing server in the multi-cloud mobile edge computing system, and reasonably distribute the computing resources and the transmission resources in the multi-cloud heterogeneous mobile edge computing system.
A first aspect of the present application provides a multi-cloud heterogeneous mobile edge computing system architecture, comprising: the mobile edge computing system comprises an edge device layer, a mobile edge computing server layer and a cloud platform layer;
the cloud platform layer consists of a plurality of cloud platforms respectively associated with independent service providers;
any one of the plurality of cloud platforms is connected with a plurality of mutually independent mobile edge computing servers through a wired transmission link;
the cloud platform is used for deciding the distribution of tasks issued by the independent service providers related to the tasks, and issuing the distribution decisions and the tasks to the mobile edge computing server layer and the edge device layer;
the mobile edge computing server layer is formed by the plurality of mutually independent mobile edge computing servers;
the mobile edge computing server layer is used for computing the task according to the distribution decision;
the mobile edge computing server is respectively connected with a plurality of edge devices through wireless transmission links;
the plurality of edge devices connected to different mobile edge computing servers form the constituent edge device layer;
the edge device layer is used for generating raw data and calculating the task according to the distribution decision.
Optionally, the specific step of the cloud platform for deciding the distribution of the tasks issued by the associated independent service providers includes:
the cloud platform decision task is unloaded; the task unloading is to unload the task to different mobile edge computing servers and different distribution ratios of edge devices;
the cloud platform decides a target mobile edge computing server or a target edge device to complete resource allocation of the task unloading; the resource allocation is a ratio of resources to be allocated for the task offloading completed by the target mobile edge computing server or the target edge device.
A second aspect of the embodiments of the present application provides a method for designing energy optimization based on a multi-cloud heterogeneous mobile edge computing system architecture, which is applied to a target cloud platform, where the target cloud platform is any one of a plurality of cloud platforms located in the multi-cloud heterogeneous mobile edge computing system architecture, and the method includes:
acquiring current state parameters of the multi-cloud heterogeneous mobile edge computing system;
acquiring the maximum probability action of other cloud platforms in the multi-cloud heterogeneous mobile edge computing system under the current state parameter;
determining a plurality of joint action pairs comprising the most probable action in a joint action pair matrix;
determining an optimal joint action pair among the plurality of joint action pairs according to the accumulated reward values of the plurality of joint action pairs;
determining the coupling action to be executed by the target cloud platform in the optimal combined action pair as a coupling action to be executed; the coupling action to be executed is the combination of task unloading and resource allocation of the current task;
obtaining an energy value consumed by the target cloud platform to execute the coupling action to be executed, and obtaining the coupling actions simultaneously executed by the other cloud platforms;
determining the coupling action to be executed and the coupling action simultaneously executed by the other cloud platforms as an execution combined action pair;
updating a cumulative reward value of the performing joint-action pair in the joint-action pair matrix with the energy value.
Optionally, the multi-cloud heterogeneous mobile edge computing system comprises a plurality of cloud platforms, a plurality of mobile edge computing servers, and a plurality of edge devices;
the plurality of mobile edge computing servers are independent of each other;
each mobile edge computing server is connected with the plurality of cloud platforms through a wired transmission link;
each edge device is connected with a target mobile edge computing server through a wireless transmission link, and the target mobile edge computing server is any one of the mobile edge computing servers.
Optionally, after obtaining the current state parameters of the cloudy heterogeneous mobile edge computing system, the method further comprises:
generating an arbitrary number;
under the condition that any number generated locally is smaller than a preset exploration probability, selecting any combination of task unloading and resource allocation in a preset action space as the coupling action to be executed;
obtaining a maximum probability action of other cloud platforms in the multi-cloud heterogeneous mobile edge computing system under the current state parameter, including:
and under the condition that any number generated locally is larger than a preset exploration probability, acquiring the maximum probability action of other cloud platforms in the multi-cloud heterogeneous mobile edge computing system under the current state parameter.
Optionally, the obtaining a maximum probability action of other cloud platforms in the multi-cloud heterogeneous mobile edge computing system under the current state parameter includes:
counting all historical coupling actions of the other cloud platforms;
taking the historical coupling action with the largest occurrence frequency under the current state parameter as the maximum probability action;
and acquiring the maximum probability action.
Optionally, the method further comprises:
forming a state space by using all state parameters in the multi-cloud heterogeneous mobile edge computing system;
and establishing the joint action pair matrix according to the state space and the preset action space.
Optionally, the method further comprises:
determining a total number N of the plurality of mutually independent mobile edge computing servers and the plurality of edge devices connected with the target cloud platform in the multi-cloud heterogeneous mobile edge computing system;
in the task unloading and resource allocation relation space, combining the task unloading and resource allocation for N +1 times to obtain f (N +1) task unloading and resource allocation combinations;
and obtaining the preset action space according to the combination of the f (N +1) task unloading and resource allocation.
Optionally, obtaining a preset action space according to the combination of the f (N +1) task offloads and resource allocations, including:
forming a set with the combination of f (N +1) task offloads and resource allocations;
deleting a combination of task offloading and resource allocation in the set that is a task offloading and resource allocation conflict;
deleting combinations of task offloading and resource allocation that violate constraints in the set;
and taking a set obtained by deleting the combination of task unloading and resource allocation which conflicts with the task unloading and resource allocation and the combination of task unloading and resource allocation which violates the constraint condition as a preset action space.
Optionally, determining a best joint action pair among the plurality of joint action pairs according to the cumulative prize value of the plurality of joint action pairs comprises:
determining respective cumulative prize values for the plurality of joint action pairs at the current state parameter;
and taking the combined action pair with the maximum accumulated reward value as the optimal combined action pair.
A third aspect of the embodiments of the present application provides an energy optimization design apparatus based on a multi-cloud heterogeneous mobile edge computing system architecture, where the apparatus includes:
the state parameter acquisition module is used for acquiring the current state parameter of the multi-cloud heterogeneous mobile edge computing system;
the maximum probability action acquisition module is used for acquiring the maximum probability action of other cloud platforms in the multi-cloud heterogeneous mobile edge computing system under the current state parameter;
a joint action pair determination module for determining a plurality of joint action pairs containing the most probable action in a joint action pair matrix;
a best joint action pair determining module, configured to determine a best joint action pair among the plurality of joint action pairs according to the accumulated reward values of the plurality of joint action pairs;
the first execution coupling action determining module is used for determining the coupling action needing to be executed by the target cloud platform in the optimal combined action pair as a coupling action to be executed; the coupling action to be executed is the combination of task unloading and resource allocation of the current task;
the energy value obtaining module is used for obtaining an energy value consumed by the target cloud platform to execute the coupling action to be executed and obtaining the coupling actions simultaneously executed by the other cloud platforms;
the execution joint action pair determining module is used for determining the coupling action to be executed and the coupling action simultaneously executed by the other cloud platforms as an execution joint action pair;
an update module to update a cumulative reward value of the performing joint action pair in the joint action pair matrix with the energy value.
In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps in the method according to the second aspect of the present application.
In a fifth aspect, embodiments of the present application further provide an electronic device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the method according to the second aspect of the present application is implemented.
By adopting the cloud heterogeneous mobile edge computing system architecture and the energy optimization design method based on the cloud heterogeneous mobile edge computing system architecture, a plurality of cloud platforms, a plurality of mobile edge computing servers and a plurality of edge devices form a three-layer cloud heterogeneous mobile edge computing system, each layer participates in task computing, and computing resources of the cloud heterogeneous mobile edge computing system are fully utilized. Each cloud platform is used as an agent, and the condition that tasks are distributed to each device is decided as an unloading strategy; deciding the allocation of computing resources and transmission resources of each device as resource allocation; taking an unloading strategy and resource allocation as the coupling actions of an intelligent agent, the intelligent agent (cloud platform) learns in a trial and error mode, interacting with the environment of the multi-cloud heterogeneous mobile edge computing system by making the coupling actions (the combination of the unloading strategy and the resource allocation), obtaining an energy value consumed by the intelligent agent (cloud platform) for executing the coupling actions (the combination of the unloading strategy and the resource allocation) in the multi-cloud heterogeneous mobile edge computing system, feeding back the energy value to the intelligent agent (cloud platform) as a reward, evaluating the joint action pairs executed by the cloud platform and other cloud platforms in the joint action pair established by the intelligent agent (cloud platform) and related to the coupling actions executed by other cloud platforms so as to enable the intelligent agent (cloud platform) to select the next coupling action (the combination of the unloading strategy and the resource allocation) according to the accumulated evaluation after multiple actions, and further, the probability that the multi-cloud heterogeneous mobile edge computing system is subjected to positive reinforcement (rewarding) is increased, so that when the task unloading, computing resource distribution and transmission resource distribution modes obtained after learning are used by all the cloud platforms, the energy value consumed by the system for processing the tasks on the cloud platforms is minimized.
The method for optimizing the multi-cloud heterogeneous mobile edge computing system combines the coupling actions (combination of unloading strategies and resource allocation) executed by the multiple cloud platforms and the state parameters corresponding to the coupling actions (combination of unloading strategies and resource allocation) to form a combined action pair matrix, the accumulated reward value of each combined action pair in the matrix is subjected to the combined action, whether the multi-cloud heterogeneous mobile edge computing system is reasonable or not through the multi-cloud actions executed by the multiple cloud platforms is objectively evaluated, the energy value consumed by the multi-cloud heterogeneous mobile edge computing system is taken as a reference, and the problem that resources of each device are contended by the multiple clouds is solved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments of the present application will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive exercise.
Fig. 1 is a diagram illustrating an exemplary structure of a multi-cloud heterogeneous mobile edge computing system architecture according to an embodiment of the present application;
FIG. 2 is a flowchart of steps related to obtaining a preset motion space according to an embodiment of the present application;
FIG. 3 is a space diagram of task offloading and resource allocation relationships according to an embodiment of the present application;
FIG. 4 is a flow chart of steps of a method of energy optimized design according to an embodiment of the present application;
FIG. 5 is a flow chart of steps of a method of obtaining a most probable action in an embodiment of the application;
fig. 6 is a schematic diagram of an energy-optimized design apparatus according to an embodiment of the present disclosure.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Existing mobile edge computing networks are broadly divided into: the edge device and the mobile edge computing server (MEC server) form a two-layer network structure, and the single cloud three-layer network comprises a single cloud platform, the edge device and the mobile edge computing server (MEC server).
In a two-layer network structure composed of an edge device and a mobile edge computing server (MEC server), or a three-layer network composed of a single cloud platform, an edge device and a mobile edge computing server (MEC server), when a computing task is specifically executed, only the edge device and the mobile edge computing server are utilized for computing, or only the cloud platform and the mobile edge computing server are utilized for computing, and the computing resources of the cloud platform, the edge device and the mobile edge computing server are not fully utilized.
To overcome the above problems, the present application provides a multi-cloud heterogeneous mobile edge computing system architecture, including: the mobile edge computing system comprises an edge device layer, a mobile edge computing server layer and a cloud platform layer;
the edge device layer is located at the bottommost layer, the mobile edge computing server layer is located at the middle layer, the cloud platform layer is located at the uppermost layer, and a heterogeneous three-layer mobile edge computing network structure is formed jointly. The edge device layer, the mobile edge computing server layer and the cloud platform layer all participate in computing, and computing resources of all devices in the multi-cloud heterogeneous mobile edge computing system architecture are fully utilized.
The cloud platform layer consists of a plurality of cloud platforms respectively associated with independent service providers;
the service provider releases tasks on the cloud platform, which may generally be tasks requiring multipoint sampling, such as finding missing people, collecting ground map, and the like.
Any one of the plurality of cloud platforms is connected with a plurality of mutually independent mobile edge computing servers through a wired transmission link;
the cloud platform is used for deciding the distribution of tasks issued by the independent service providers related to the tasks, and issuing the distribution decisions and the tasks to the mobile edge computing server layer and the edge device layer;
illustratively, the cloud platform 1 responsible for the task of communication and the cloud platform 2 responsible for acquiring the image belong to the same heterogeneous mobile edge computing network. The information interaction between the cloud platform 1 and the cloud platform 2 is incomplete, and in order to reduce the energy consumption of tasks of the cloud platform 1 and the cloud platform 2, the cloud platform 1 and the cloud platform 2 which are independent of each other compete for computing resources and transmission resources in a mobile edge computing server (MEC server) and edge equipment. Therefore, there is a need for a method that can reasonably allocate computing and transmission resources of a cloud platform, a mobile edge computing server (MEC server) and edge devices in a mobile edge computing network, thereby reducing system energy consumption.
The mobile edge computing server layer is formed by the plurality of mutually independent mobile edge computing servers;
the mobile edge computing server layer is used for computing the task according to the distribution decision;
the mobile edge computing server is respectively connected with a plurality of edge devices through wireless transmission links;
the plurality of edge devices connected to different mobile edge computing servers form the constituent edge device layer;
the edge device layer is used for generating raw data and calculating the task according to the distribution decision.
Referring to fig. 1, fig. 1 is a diagram illustrating a structure example of a multi-cloud heterogeneous mobile edge computing system architecture according to an embodiment of the present application.
The mobile edge computing server, i.e., the MEC server, is a base station, wireless access point, etc. with moderate computing power, located between the cloud platform and the edge devices. The wireless access network is utilized to provide services required by IT of telecommunication users and cloud computing functions nearby, so that a telecommunication service environment with high performance, low delay and high bandwidth is created, the rapid downloading of various contents, services and applications in the network is accelerated, and consumers can enjoy uninterrupted high-quality network experience.
The cloud platform is used for processing part of computing tasks issued by the cloud platform by utilizing self limited computing and transmission resources.
The edge device is a common mobile device or an internet of things device, such as a mobile phone, a camera, a sensor, a notebook computer, and the like, and has a weak computing capability. The system is used for generating and uploading original data required by the tasks issued by the cloud platform, and can also partially calculate the tasks issued by the cloud platform.
The cloud platform has strong computing power. Each cloud platform is associated with an independent service provider, and information interaction between the cloud platforms is incomplete; each cloud platform issues a computing task that is distributed to a plurality of mobile edge computing servers and edge devices. The cloud platform is used for integrating and calculating data uploaded by the edge devices and the mobile edge computing edge server and also performing partial processing on computing tasks issued by the cloud platform.
Exemplarily, in fig. 1, a cloud platform 1 connects an MEC server 1 and an MEC server 2 simultaneously through a wired transmission link, the MEC server 1 connects the cloud platform 1 and the cloud platform 2 simultaneously through a wired transmission link, and connects an edge device 1 and an edge device 2 through a wireless transmission link. For the edge device 1 or the edge device 2, there is and only is a connection with the MEC server 1 via a wireless transmission link.
Fig. 1 is only an example of a multi-cloud heterogeneous mobile edge computing system, the cloud platform is not limited to the cloud platform 1 and the cloud platform 2, the mobile edge computing servers are not limited to the MEC server 1 and the MEC server 2, and the edge devices are not limited to the edge device 1, the edge device 2, the edge device 3, and the edge device 4. For example, the multi-cloud heterogeneous mobile edge computing system may have three cloud platforms, four MEC servers, and fifteen edge devices, which is not limited in this embodiment of the present application.
The specific steps of the cloud platform for deciding the assignment of the associated tasks published by the independent service providers include:
the cloud platform decision task is unloaded; the task unloading is to unload the task to different mobile edge computing servers and different distribution ratios of edge devices;
the cloud platform decides a target mobile edge computing server or a target edge device to complete resource allocation of the task unloading; the resource allocation is a ratio of resources to be allocated for the task offloading completed by the target mobile edge computing server or the target edge device.
The allocation decision of the cloud platform comprises task unloading and resource allocation. The resource allocation includes a computational resource allocation and a transmission resource allocation.
First, the cloud platform decides allocation of each task and allocation of computing and transmission resources of each device (cloud platform, MEC server or edge device) in the multi-cloud heterogeneous mobile edge computing system.
Considering the allocation decision of the cloud platform on the task as task offloading, refer to fig. 1, for example: the cloud platform 1 allocates 6% of a communication task to the edge device 1, 9% of the task to the edge device 2, 6% of the task to the edge device 3, 9% of the task to the edge device 4, 20% of the task to the MEC server 1, 20% of the task to the MEC server 2, and 30% of the task to the cloud platform 1 itself. Then the task is offloaded as: [6, 9, 20, 30 ].
The allocation of computational resources and transmission resources is considered as a resource allocation. For example, the cloud platform 1 determines that 6% of the communication tasks need to occupy 12% of the computing resources of the edge device 1; 9% of the communication tasks need to occupy 30% of the computing resources of the edge device 2; 6% of the communication tasks need to be completed by occupying 20% of the computing resources of the edge device 3; 9% of the communication tasks need to occupy 30% of the computing resources of the edge device 4; 20% of the communication task needs to occupy 10% of the computing resources of the MEC server 1; 20% of the communication tasks need to occupy 8% of the computing resources of the MEC server 2; and occupies 30% of the computing resources of the system. Then the resource allocation is 12, 30,20, 30,10, 8, 30.
The specific policy regarding task offloading and resource allocation determined by the cloud platform 1 is a coupling action according to the embodiment of the present application.
In practical application, the cloud platform can make decisions on resource allocation and task offloading according to the specific allocation possibility of tasks and the occupation possibility of computing resources of specific equipment.
In the multi-cloud heterogeneous mobile edge computing system architecture, a plurality of clouds share transmission resources and computing resources of the system, and the computing tasks are issued to a plurality of MEC servers and a plurality of edge devices connected with the same. The coupling actions of the cloud platform 1 are [6, 9, 20, 30], [12, 30,20, 30,10, 8,30 ], other cloud platforms are independent of the cloud platform 1, other coupling actions can be decided, and a plurality of cloud platforms compete for computing resources or transmission resources of the MEC server and the edge device to reduce energy consumption of the cloud platforms. For example, 30% of the computing resources of the edge device 2 are utilized in the coupling action of the cloud platform 1 decision, and 20% of the computing resources of the edge device 2 are utilized in the coupling action of the cloud platform 2 decision, and assuming that the edge device only provides 50% of the available resources in the current state, the cloud platform 1 and the cloud platform 2 compete for 50% of the computing resources of the edge device with each other.
In view of this, the embodiment of the present application provides a method for optimizing heterogeneous mobile edge computing, which evaluates historical coupling actions of each cloud platform in a multi-cloud heterogeneous mobile edge computing system architecture according to an energy value consumed by a system, where the coupling actions pay attention to task allocation from a task on one hand, and pay attention to computing capabilities of devices (edge devices and MEC servers) from devices (edge devices and MEC servers) on the other hand, so as to achieve coupling of system energy consumption, tasks, and device computing capabilities. The multiple cloud platforms can work cooperatively in multiple times of learning. The method and the system jointly calculate a task by combining the MEC server and the edge device with strong calculation capacity, and improve the efficiency of calculating the task.
Illustratively, the service provider 1 arranges the communicated tasks on the cloud platform 1, and the service provider 2 arranges the tasks of acquiring images on the cloud platform 2.
In the embodiment of the application, the cloud platform decision coupling action is based on a preset action space.
Referring to fig. 2, fig. 2 is a flowchart illustrating steps of obtaining a preset motion space according to an embodiment of the present application.
Step S201: determining a total number N of the plurality of mutually independent mobile edge computing servers and the plurality of edge devices connected with the target cloud platform in the multi-cloud heterogeneous mobile edge computing system;
step S202: in the task unloading and resource allocation relation space, combining the task unloading and resource allocation for N +1 times to obtain f (N +1) task unloading and resource allocation combinations;
step S203: and obtaining the preset action space according to the combination of the f (N +1) task unloading and resource allocation.
And forming a task unloading and resource allocation relation space in advance according to the system properties of the multi-cloud heterogeneous mobile edge computing system.
The possibilities of all devices with respect to resource allocation are integrated. For example, an edge device may partition computing resources into [1, … … 2], or may be allocated as [10, 20, 40, 30], where there are multiple possibilities, and thus multiple possibilities for edge device resource allocation need to be aggregated together. [1, 3 … 30, 32, … 20,15, … 50, 52, 53 … … ], and similarly, task offloading is also a possibility for a variety of possible task assignments. Based on the above analysis, table a is established to show the task offloading and resource allocation relationship space:
Figure BDA0002376499250000111
Figure BDA0002376499250000121
the cloud platform 1 may select any blank space in table a as a combination of task offloading and resource allocation, and take the selected combination of task offloading and resource allocation as an action of the current task.
And under the condition that the cloud platform is directly or indirectly connected with a plurality of devices, the cloud platform selects a combination of a plurality of task uninstallations and resource allocation according to the upper table, and focuses on one device at a time.
Taking the multi-cloud heterogeneous mobile edge computing system shown in fig. 1 as an example, a service provider needs to complete a task of establishing a live view through the cloud platform 1, the cloud platform 1 issues a task of acquiring an image of a place to the system in order to complete the task of establishing the live view, and a selection about the edge device 1, the edge device 2, the MEC server 1, and other devices is made in the table a.
The cloud platform 1 selects a point P in the table a for the edge device 1, representing that 6% of the tasks of acquiring the image of the place are allocated to the edge device 1, and the task of calculating the 6% occupies 12% of the computing resources of the edge device 1. The cloud platform 1 needs to make a selection of 7 devices, including the cloud platform 1, in the above-described manner to determine a selection of a combination of resource allocations of the edge device 1, the edge device 2, the edge device 3, the edge device 4, the MEC server 1, the MEC server 2, and the cloud platform 1. The cloud platform first needs to determine its connected mobile edge compute server and the total number N of edge devices connected through the mobile edge compute server. And then making N +1 times of selection in the task unloading and resource allocation relation space.
For a particular device, if there are M possibilities for both resource allocation and task offloading, then it is determined that there are M × M possibilities for the combination of task offloading and resource allocation. For N +1 devices, it then takes (N +1) times to determine the current policy among M × M possibilities.
f(N+1)＝M2(N+1)
Where N is the sum of the number of mobile edge compute servers connected by the cloud platform and the number of edge devices connected through the mobile edge compute servers. And M is the size of the value range of the task unloading and resource allocation relation space. Taking table a as an example, the resource allocation and task offloading is 100.
In another embodiment of the present application, the resource allocation may be divided into transmission resources and computing resources, the task offloading and resource allocation relationship space may also be a cube as shown in fig. 3, and fig. 3 is a task offloading and resource allocation relationship space diagram according to an embodiment of the present application. The X-axis coordinate of the cube represents the specific allocation percentage of the task offload, the Y-axis coordinate represents the percentage of the computing resources of the device, and the Z-axis coordinate represents the percentage of the transmission resources of the device.
Following the above example, assume a task offload is [6, 9, 20, 30], a computing resource is [12, 30,20, 30,10, 8,30 ], and a transmission resource is [12, 10, 5, 9, 20,8, 36 ]. The coupling action of the cloud platform 1 may be [6, 9, 20, 30], [12, 30,20, 30,10, 8,30 ], [12, 10, 5, 9, 20,8 ].
It can be understood that the edge device processes the data and uploads the processed data to the mobile edge computing server, the mobile edge computing server recalculates the data and uploads the recalculated data to the cloud platform, and the cloud platform completes the final data computation and integrates the data to complete the task, and the data is not transmitted upwards any more.
In this embodiment, when the resource allocation includes transmission resources and computation resources, the combination of task offloading and resource allocation is M × M possible. For N +1 devices, it then takes (N +1) times to determine the current policy among M × M possibilities.
f(N+1)＝M3(N+1)
The preset action space is made by the cloud platform according to the task unloading and resource allocation relation space, and is related to the set of the combination of the task unloading and resource allocation of all the connected devices and the cloud platform. For example, in the multi-cloud heterogeneous mobile edge computing system shown in fig. 1, one element of the preset motion space of the cloud platform 1 may be [6, 12,12], [9,30,10], [6,20,5], [9,30,9], [20,10,20], [20,8,8], [30,30], [30 ] or the like. The preset motion space is therefore a huge space.
To describe [6, 12,12], [9,30,10], [6,20,5], [9,30,9], [20,10,20], [20,8,8], [30,30] by [6, 12,12] represents that [6, 12,12] denotes that the cloud platform 1 allocates 6% of the tasks of acquiring the image of the site to the edge device 1, and that the tasks of calculating the 6% occupy 12% of the computing resources of the edge device 1, and the data at the edge device 1 is transmitted to the MEC server 1 using 12% of the wireless transmission resources of the MEC server 1. Other devices have similar or same meanings as [6, 12,12], and are not described herein.
For a huge preset action space, in order to reduce the calculation amount of the system, the embodiment of the application firstly deletes actions which do not have practical significance, namely the combination of task unloading and resource allocation.
Obtaining a preset action space according to the combination of the f (N +1) task uninstallations and resource allocations, including:
forming a set with the combination of f (N +1) task offloads and resource allocations;
deleting a combination of task offloading and resource allocation in the set that is a task offloading and resource allocation conflict;
deleting combinations of task offloading and resource allocation that violate constraints in the set;
and taking a set obtained by deleting the combination of task unloading and resource allocation which conflicts with the task unloading and resource allocation and the combination of task unloading and resource allocation which violates the constraint condition as a preset action space.
The task offloading conflicts with resource allocation in the case that a task does not perform any computation at a certain device, but a certain device allocates certain computation resources exclusively for the computation of the task. For example, in a task of searching for a missing person, the task does not perform any calculation on a (edge device) sensor, only needs the sensor to upload acquired sensing data, and deletes actions occupying sensor calculation resources in a task space.
The constraint condition means that two or more cloud platforms do not occupy more than 100% of the computing resources or transmission resources of the same device. Or that a device is not allowed to allocate more computing or transmission resources than its own upper limit.
After the actions meeting the conditions are deleted, the calculation complexity of the preset action space is correspondingly reduced, and the calculation is more accurate.
In the embodiment of the present application, a cloud platform is used as an agent, and actually the cloud platform has a strong computing capability and is sufficient to be used as an agent for reinforcement learning, a combination of task offloading and resource allocation explained in the above embodiments is used as an action of the agent, feedback of edge devices or mobile edge computing servers in a multi-cloud heterogeneous mobile edge computing system is used as a state parameter, a value of consumed energy of the whole system of the multi-cloud heterogeneous mobile edge computing system is used as a reward, and actions performed by each agent in the same iteration are regarded as a joint action pair, so that reinforcement learning is performed on the cloud platform.
The status parameter may be a data generation rate of any edge device, and may also be a channel resource or a channel gain of the MEC server.
The cloud platform discretizes a state space and a preset action space before reinforcement learning. The state space is composed of all state parameters. Illustratively, with the data generation rate as the state parameter, the data generation rate of the edge device may be [2-15], and then the discretized state space is [2, 3,4,5,6,7,8,9,10,11,12,13,14,15 ].
The method further comprises the following steps:
forming a state space by using all state parameters in the multi-cloud heterogeneous mobile edge computing system;
and establishing the joint action pair matrix according to the state space and the preset action space.
The federated action pair is a combination of actions made by multiple cloud platforms. The joint action pair matrix is a collection of combinations of actions made by multiple cloud platforms. Alternatively, the joint action pair matrix may be regarded as a set of preset action spaces of the plurality of cloud platforms under the respective state parameters. The joint action pairs of any of the joint action pair matrices have corresponding Q values as the cumulative prize values. The amount of energy expended by the system after each iteration is weighted by the cumulative prize value.
The joint action pair at least comprises the coupling action of the two cloud platforms, namely at least comprises the strategies of the two cloud platforms about task unloading and resource allocation, the joint action pair is evaluated by the accumulated reward value, and the response condition of the specific device to the tasks allocated by the two cloud platforms can be objectively known when the two cloud platforms allocate the specific task to the specific device.
The iteration is a complete process that the intelligent agent (cloud platform) selects an action, executes the action to obtain an award value and updates the system state (the matrix is combined with the action) under the current state parameter.
Therefore, the embodiment of the application represents the current state of the system by a combined action pair matrix.
Referring to fig. 4, fig. 4 is a flowchart illustrating steps of a design method for energy optimization according to an embodiment of the present disclosure.
The method is applied to a target cloud platform, wherein the target cloud platform is any one of a plurality of cloud platforms in a multi-cloud heterogeneous mobile edge computing system; the method comprises the following steps:
step S401: acquiring current state parameters of the multi-cloud heterogeneous mobile edge computing system;
taking the multi-cloud heterogeneous mobile edge computing system shown in fig. 1 as an example, the target cloud platform is a cloud platform 1 that currently issues a task of acquiring an image of a place.
After the cloud platform 1 is initialized, a state space after discrete processing and a preset action space after discrete processing, deletion of actions without realistic significance and violation of constraint conditions are obtained. And obtaining a state parameter, namely a data generation rate of the edge device 12K from the multi-cloud heterogeneous mobile edge computing system.
In the multi-cloud heterogeneous mobile edge computing system in the embodiment of the application, the state parameter-data generation rate of the same task is the same on each device, so that the state parameters obtained by different cloud platforms are the same.
Step S402: acquiring the maximum probability action of other cloud platforms in the multi-cloud heterogeneous mobile edge computing system under the current state parameter;
incomplete information interaction can be carried out between the cloud platforms. Illustratively, cloud platform 1 observes historical actions of cloud platform 2 and observes a probability distribution of its historical actions.
In another embodiment of the present application, a method for obtaining a maximum probability action is specifically proposed:
referring to fig. 5, fig. 5 is a flowchart of steps of a method for obtaining a maximum probability action according to an embodiment of the present application:
step S402-1: counting all historical coupling actions of the other cloud platforms;
step S402-2: taking the historical coupling action with the largest occurrence frequency under the current state parameter as the maximum probability action;
step S402-3: and acquiring the maximum probability action.
In specific application, the cloud platform 1 may establish a mathematical model for the cloud platform 2 or other multiple cloud platforms to obtain a maximum probability action of the cloud platform, and the maximum probability action is used as an action of the cloud platform 2 in the current iteration, that is, a task allocation strategy made by the cloud platform 2.
In the case that the data generation rate of the cloud platform 2 is 12K, the action with the largest occurrence history is taken as the most probable action. The maximum probability action of the cloud platform 2 is assumed as: [6, 12,10], [8,30,20], [6,20,15], [10,30,15], [20,10,12], [20,8,16], [30,40]
Step S403: determining a plurality of joint action pairs comprising the most probable action in a joint action pair matrix;
the cloud platform 1 finds a plurality of joint action pairs containing the most probable action in the local joint action pair matrix. It is apparent that the multiple combined actions found are for the corresponding state parameter the data generation rate 12K. The elements in the joint action pair matrix are joint action pairs, and the prize values corresponding to the joint actions.
Continuing with the above example, the plurality of joint action pairs are all joint action pairs including [6, 12,10], [8,30,20], [6,20,15], [10,30,15], [20,10,12], [20,8,16], [30,40] corresponding to a data generation rate of 12K.
Step S404: determining an optimal joint action pair among the plurality of joint action pairs according to the accumulated reward values of the plurality of joint action pairs;
determining respective cumulative prize values for the plurality of joint action pairs at the current state parameter;
and taking the combined action pair with the maximum accumulated reward value as the optimal combined action pair.
The combined action pair with the largest accumulated reward value in all the combined actions including [6, 12,10], [8,30,20], [6,20,15], [10,30,15], [20,10,12], [20,8,16], [30,40] is the optimal combined action pair.
And the corresponding combined action pair with the maximum accumulated reward value is regarded as the best combined action pair after the system learns the current time.
According to the above embodiment, the joint action pair is a combination of actions made by each cloud platform; under the condition that the accumulated reward value of the combined action pair is the maximum, the tasks are distributed to all the devices by all the cloud platforms, and the situation that the resources of all the devices are occupied is the most reasonable. Therefore, the cloud platform 1 selects the joint action pair of the cumulative prize value as the most reasonable action matching the action that the cloud platform 2 is most likely to make in the state of the data generation rate of 12K.
Step S405: determining the coupling action to be executed by the target cloud platform in the optimal combined action pair as a coupling action to be executed; the coupling action to be executed is the combination of task unloading and resource allocation of the current task;
assuming that the optimal joint action pair including [6, 12,10], [8,30,20], [6,20,15], [10,30,15], [20,10,12], [20,8,16], [30,40] requires the coupling action performed by the cloud platform 1: [6, 12,12], [9,30,10], [6,20,5], [9,30,9], [20,10,20], [20,8,8], [30,30], [6, 12,12], [9,30,10], [6,20,5], [9,30,9], [20,10,20], [20,8,8], [30,30] are to be performed coupling actions in the iteration of the cloud platform 1.
The method for optimizing the multi-cloud heterogeneous mobile edge computing system provided by the embodiment of the application takes the accumulated reward value as the evaluation of the combination of the task uninstalling and the resource allocation of the plurality of cloud platforms in the same state in the multi-cloud heterogeneous mobile edge computing system, the higher the accumulated reward value is, the more reasonable the decision of the task uninstalling and the resource allocation of each corresponding cloud platform under the accumulated reward value is, and the significance is that in the multi-cloud heterogeneous mobile edge computing system, the efficiency of each cloud platform uninstalling the task to the equipment and the efficiency of the single equipment completing the allocated task are coupled as much as possible.
Step S406: obtaining an energy value consumed by the target cloud platform to execute the coupling action to be executed, and obtaining the coupling actions simultaneously executed by the other cloud platforms;
the energy value consumed by the target cloud platform to execute the coupling action to be executed comprises: the MEC server, the edge device, and the cloud platform may be configured to assign the assigned task to the edge device.
The coupling action executed at the same time is the coupling action executed by other cloud platforms under the current state parameter. For example, the coupling action simultaneously performed by the cloud platform 2 is a coupling action performed by the cloud platform 2 when the state parameter is 12K and the cloud platform 1 performs [6, 12,12], [9,30,10], [6,20,5], [9,30,9], [20,10,20], [20,8,8], [30,30], [6, 12] or [9,30 ] respectively.
The coupling actions performed by the cloud platform 2 at the same time may be [6, 12,10], [8,30,20], [6,20,15], [10,30,15], [20,10,12], [20,8,16], [30,40], or may be other coupling actions, which are based on the combination of task offloading and resource allocation selected by the cloud platform 2 in real time.
It is understood that the energy value is generated when the device in the multi-cloud heterogeneous mobile edge computing system simultaneously executes tasks published by other cloud platforms.
Step S407: determining the coupling action to be executed and the coupling action simultaneously executed by the other cloud platforms as an execution combined action pair;
the execution of the union action pair is: the coupling action to be executed and the coupling action simultaneously executed by other cloud platforms under the current state parameter are the coupling actions really executed by the target cloud platform and the other cloud platforms in the current iteration.
Step S408: updating a cumulative reward value of the performing joint-action pair in the joint-action pair matrix with the energy value.
The cumulative prize value of each co-action pair deposited in the matrix of co-action pairs represents its value in a different state, i.e., the weighted sum of the prizes for that co-action pair performed each time in that state. And updating a joint action pair consisting of the coupling actions really executed by the target cloud platform and other cloud platforms in the current iteration by using the energy value, and evaluating the action selection of the iteration target cloud platform and other cloud platforms.
When the energy consumption is large, the coupling action of the target cloud platform and other cloud platforms is selected unreasonably, and the large energy consumption value also makes a negative correlation weighted sum on the accumulated reward value, so that the multi-cloud heterogeneous mobile edge computing system avoids the currently selected coupling action in the next selection as much as possible. When the energy consumption is small, the coupling action of the target cloud platform and other cloud platforms is reasonably selected, and the small energy consumption value is used for making a positive correlation weighted sum on the accumulated reward value, so that the multi-cloud heterogeneous mobile edge computing system is close to the currently selected coupling action in the next selection. By the method, the preset iteration times are carried out, so that the multi-cloud heterogeneous mobile edge computing system learns more optimal selection, computing tasks, computing resources and transmission resources in a network are reasonably distributed by each cloud platform, and the system energy consumption is minimized.
In another embodiment of the present application, the cloud platform may also generate any number to decide the selection way of selecting the coupling action.
Under the condition that any number generated locally is smaller than a preset exploration probability, selecting any combination of task unloading and resource allocation in a preset action space as the coupling action to be executed;
the preset exploration probability can be set at will, and the situation that the multi-cloud heterogeneous mobile edge computing system framework selects the coupling action to be executed by using a method in the learning process is avoided. The preset exploration probability may represent a probability that the system selects the coupling action to be performed in another manner.
Obtaining a maximum probability action of other cloud platforms in the multi-cloud heterogeneous mobile edge computing system under the current state parameter, including:
and under the condition that any number generated locally is larger than a preset exploration probability, acquiring the maximum probability action of other cloud platforms in the multi-cloud heterogeneous mobile edge computing system under the current state parameter.
Selecting an optimal joint action pair from the joint action pair matrix according to the obtained maximum probability action;
further, selecting the optimal joint action pair in a preset action space to determine the coupling action to be executed by the target cloud platform as the coupling action to be executed. The specific selection process is the same as the method in the above embodiment, and will not be described herein.
And under the condition that any number is smaller than the preset exploration probability, selecting any combination of task unloading and resource allocation in the preset action space so as to enrich the samples in the matrix of the joint action pair, namely, more joint action pairs have corresponding accumulated reward values.
When the cloud platform (agent) obtains stable distribution of the accumulated reward values in the combined action pair matrix after multiple iterations, other choices need to be tried, namely any coupling action is selected in a preset action space, and an energy value when any coupling action is executed is obtained, so that the system is prevented from selecting and learning based on the limited combined action pair.
Based on the same inventive concept, the embodiment of the application provides a device for designing energy optimization based on a multi-cloud heterogeneous mobile edge computing system architecture. Referring to fig. 6, fig. 6 is a schematic diagram of an energy-optimized design apparatus according to an embodiment of the present disclosure. As shown in fig. 6, the apparatus includes:
a state parameter obtaining module 601, configured to obtain a current state parameter of the cloudy heterogeneous mobile edge computing system;
a maximum probability action obtaining module 602, configured to obtain a maximum probability action of other cloud platforms in the multi-cloud heterogeneous mobile edge computing system under the current state parameter;
a joint action pair determining module 603, configured to determine a plurality of joint action pairs including the most probable action in a joint action pair matrix;
a best joint action pair determination module 604, configured to determine a best joint action pair among the plurality of joint action pairs according to the accumulated reward values of the plurality of joint action pairs;
a first executing coupling action determining module 605, configured to determine, as a to-be-executed coupling action, a coupling action that needs to be executed by the target cloud platform in the optimal joint action pair; the coupling action to be executed is the combination of task unloading and resource allocation of the current task;
an energy value obtaining module 606, configured to obtain an energy value consumed by the target cloud platform to execute the coupling action to be executed, and obtain coupling actions executed by the other cloud platforms at the same time;
an execution joint action pair determining module 607, configured to determine the coupling action to be executed and the coupling action executed by the other cloud platform at the same time as an execution joint action pair;
an update module 608 for updating a cumulative reward value of the performing joint-action pair in the matrix of joint-action pairs with the energy value.
Optionally, the apparatus further comprises:
a generation module for generating an arbitrary number;
the second execution coupling action determining module is used for selecting any combination of task unloading and resource allocation as the coupling action to be executed in a preset action space under the condition that any number generated locally is smaller than a preset exploration probability;
the first execution coupling action determination module includes:
and the probability obtaining submodule is used for obtaining the maximum probability action of other cloud platforms in the multi-cloud heterogeneous mobile edge computing system under the current state parameter under the condition that the locally generated arbitrary number is larger than the preset exploration probability.
Optionally, the maximum probability action obtaining module includes:
the statistic submodule is used for counting all historical coupling actions of the other cloud platforms;
a maximum probability action determining submodule, configured to use the historical coupling action with the largest occurrence frequency in the current state parameter as the maximum probability action;
and the obtaining submodule is used for obtaining the maximum probability action.
Optionally, the apparatus further comprises:
the state space forming module is used for forming a state space by using all state parameters in the multi-cloud heterogeneous mobile edge computing system;
and the joint action pair matrix forming module is used for establishing the joint action pair matrix according to the state space and the preset action space.
Optionally, the apparatus further comprises:
a device shortage module, configured to determine a total number N of the plurality of independent mobile edge computing servers and the plurality of edge devices connected to the target cloud platform in the multi-cloud heterogeneous mobile edge computing system;
the action quantity determining module is used for combining the task unloading and the resource allocation for N +1 times in the task unloading and resource allocation relation space to obtain f (N +1) task unloading and resource allocation combinations;
a preset action space obtaining module, configured to obtain the preset action space according to the combination of the f (N +1) task offloads and resource allocations.
Optionally, the preset motion space obtaining module includes:
a set submodule for forming a set with the combination of the f (N +1) task offloads and resource allocations;
a first deletion submodule, configured to delete a combination of task offloading and resource allocation in the set, where the task offloading and resource allocation conflict;
a second deletion submodule for deleting combinations of task offloading and resource allocation that violate constraints from the set;
and the preset action space obtaining submodule is used for taking a set obtained by deleting the combination of task unloading and resource allocation which conflicts with the task unloading and resource allocation and the combination of task unloading and resource allocation which violates the constraint condition as a preset action space.
Optionally, the optimal combined action pair determining module includes:
a cumulative reward value determination submodule for determining respective cumulative reward values of the plurality of joint action pairs under the current state parameter;
and the optimal combined action pair determining submodule is used for determining the combined action pair with the maximum accumulated reward value as the optimal combined action pair.
For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
In a third aspect, embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps in the method according to the first aspect of the present application.
In a fourth aspect, embodiments of the present application further provide an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the steps in the method according to the first aspect of the present application when executed.
The embodiments in the specification are described in a progressive or illustrative manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
As will be appreciated by one of skill in the art, embodiments of the present application may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Embodiments of the present application are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus, and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present application have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the true scope of the embodiments of the application.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.
The detailed description is given above to the design method, device and storage medium for the architecture and energy optimization of the multi-cloud heterogeneous mobile edge computing system provided by the present application, and the description of the above embodiments is only used to help understand the method and the core idea of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims (10)

1. A multi-cloud heterogeneous mobile edge computing system architecture, comprising: the mobile edge computing system comprises an edge device layer, a mobile edge computing server layer and a cloud platform layer;
the cloud platform layer consists of a plurality of cloud platforms respectively associated with independent service providers;
any one of the plurality of cloud platforms is connected with a plurality of mutually independent mobile edge computing servers through a wired transmission link;
the cloud platform is used for deciding the distribution of tasks issued by the independent service providers related to the tasks, and issuing the distribution decisions and the tasks to the mobile edge computing server layer and the edge device layer;
the mobile edge computing server layer is formed by the plurality of mutually independent mobile edge computing servers;
the mobile edge computing server layer is used for computing the task according to the distribution decision;
the mobile edge computing server is respectively connected with a plurality of edge devices through wireless transmission links;
the edge devices connected with different mobile edge computing servers form the edge device layer;
the edge device layer is used for generating original data and calculating the task according to the distribution decision;
any one of the plurality of cloud platforms is configured to publish a computing task and distribute the computing task to the plurality of mobile edge computing servers and the plurality of edge devices.
2. The multi-cloud heterogeneous mobile edge computing system architecture of claim 1,
the specific steps of the cloud platform for deciding the assignment of the associated tasks published by the independent service providers include:
the cloud platform decision task is unloaded; the task unloading is to unload the task to different mobile edge computing servers and different distribution ratios of edge devices;
the cloud platform decides a target mobile edge computing server or a target edge device to complete resource allocation of the task unloading; the resource allocation is a ratio of resources to be allocated for the task offloading completed by the target mobile edge computing server or the target edge device.
3. A design method for energy optimization based on a multi-cloud heterogeneous mobile edge computing system architecture is characterized by being applied to a target cloud platform, wherein the target cloud platform is any one of the multi-cloud heterogeneous mobile edge computing system architecture; the method comprises the following steps:
acquiring current state parameters of the multi-cloud heterogeneous mobile edge computing system;
acquiring the maximum probability action of other cloud platforms in the multi-cloud heterogeneous mobile edge computing system in the historical state with the same current state parameters;
determining a plurality of joint action pairs comprising the most probable action in a joint action pair matrix;
determining an optimal joint action pair among the plurality of joint action pairs according to the accumulated reward values of the plurality of joint action pairs;
determining the coupling action to be executed by the target cloud platform in the optimal combined action pair as a coupling action to be executed; the coupling action to be executed is the combination of task unloading and resource allocation of the current task;
obtaining an energy value consumed by the target cloud platform to execute the coupling action to be executed, and obtaining the coupling actions simultaneously executed by the other cloud platforms;
determining the coupling action to be executed and the coupling action simultaneously executed by the other cloud platforms as an execution combined action pair;
updating a cumulative reward value of the performing joint-action pair in the joint-action pair matrix with the energy value.
4. The method of claim 3, wherein after obtaining current state parameters of the cloudy, heterogeneous mobile edge computing system, the method further comprises:
generating an arbitrary number;
under the condition that any number generated locally is smaller than a preset exploration probability, selecting any combination of task unloading and resource allocation in a preset action space as the coupling action to be executed;
obtaining a maximum probability action of other cloud platforms in the multi-cloud heterogeneous mobile edge computing system under the current state parameter, including:
and under the condition that any number generated locally is larger than a preset exploration probability, acquiring the maximum probability action of other cloud platforms in the multi-cloud heterogeneous mobile edge computing system under the current state parameter.
5. The method of claim 3, wherein obtaining a maximum probability action for other cloud platforms in the multi-cloud heterogeneous mobile edge computing system at the current state parameter comprises:
counting all historical coupling actions of the other cloud platforms;
taking the history coupling action with the largest occurrence frequency in the history state with the same current state parameters as the maximum probability action;
and acquiring the maximum probability action.
6. The method of claim 4, further comprising:
forming a state space by using all state parameters in the multi-cloud heterogeneous mobile edge computing system;
and establishing the joint action pair matrix according to the state space and the preset action space.
7. The method of claim 4, further comprising:
determining a total number N of the plurality of mutually independent mobile edge computing servers and a plurality of edge devices connected with the target cloud platform in the multi-cloud heterogeneous mobile edge computing system;
in the task unloading and resource allocation relation space, combining the task unloading and resource allocation for N +1 times to obtain f (N +1) task unloading and resource allocation combinations; wherein f (N +1) ═ M2(N+1)M is the size of the value range of the task unloading and resource allocation relation space;
and obtaining the preset action space according to the combination of the f (N +1) task unloading and resource allocation.
8. The method of claim 7, wherein obtaining a preset action space according to the combination of f (N +1) task offloads and resource allocations comprises:
forming a set with the combination of f (N +1) task offloads and resource allocations;
deleting a combination of task offloading and resource allocation in the set that is a task offloading and resource allocation conflict;
deleting combinations of task offloading and resource allocation that violate constraints in the set;
and taking a set obtained by deleting the combination of task unloading and resource allocation which conflicts with the task unloading and resource allocation and the combination of task unloading and resource allocation which violates the constraint condition as a preset action space.
9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 3 to 8.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing performs the steps of the method according to any of claims 3 to 8.
CN202010069937.XA 2020-01-20 2020-01-20 Multi-cloud heterogeneous mobile edge computing system architecture and energy optimization design method Active CN111328107B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010069937.XA CN111328107B (en) 2020-01-20 2020-01-20 Multi-cloud heterogeneous mobile edge computing system architecture and energy optimization design method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010069937.XA CN111328107B (en) 2020-01-20 2020-01-20 Multi-cloud heterogeneous mobile edge computing system architecture and energy optimization design method

Publications (2)

Publication Number Publication Date
CN111328107A CN111328107A (en) 2020-06-23
CN111328107B true CN111328107B (en) 2021-06-18

Family

ID=71170964

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010069937.XA Active CN111328107B (en) 2020-01-20 2020-01-20 Multi-cloud heterogeneous mobile edge computing system architecture and energy optimization design method

Country Status (1)

Country Link
CN (1) CN111328107B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112148464B (en) * 2020-10-30 2023-07-07 深圳供电局有限公司 Method and system for unloading mobile edge computing task

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170195247A1 (en) * 2015-12-31 2017-07-06 EMC IP Holding Company LLC Method and apparatus for cloud system
CN109548155A (en) * 2018-03-01 2019-03-29 重庆大学 A kind of non-equilibrium edge cloud network access of distribution and resource allocation mechanism
CN109688596A (en) * 2018-12-07 2019-04-26 南京邮电大学 A kind of mobile edge calculations system constituting method based on NOMA
CN109788069A (en) * 2019-02-27 2019-05-21 电子科技大学 Calculating discharging method based on mobile edge calculations in Internet of Things
CN110418416A (en) * 2019-07-26 2019-11-05 东南大学 Resource allocation methods based on multiple agent intensified learning in mobile edge calculations system

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110113140B (en) * 2019-03-08 2020-08-11 北京邮电大学 Calculation unloading method in fog calculation wireless network

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170195247A1 (en) * 2015-12-31 2017-07-06 EMC IP Holding Company LLC Method and apparatus for cloud system
CN109548155A (en) * 2018-03-01 2019-03-29 重庆大学 A kind of non-equilibrium edge cloud network access of distribution and resource allocation mechanism
CN109688596A (en) * 2018-12-07 2019-04-26 南京邮电大学 A kind of mobile edge calculations system constituting method based on NOMA
CN109788069A (en) * 2019-02-27 2019-05-21 电子科技大学 Calculating discharging method based on mobile edge calculations in Internet of Things
CN110418416A (en) * 2019-07-26 2019-11-05 东南大学 Resource allocation methods based on multiple agent intensified learning in mobile edge calculations system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"移动边缘计算任务卸载和基站关联协同决策问题研究";于博文等;《计算机研究与发展》;20180315;全文 *

Also Published As

Publication number Publication date
CN111328107A (en) 2020-06-23

Similar Documents

Publication Publication Date Title
CN113055489B (en) Implementation method of satellite-ground converged network resource allocation strategy based on Q learning
CN109600798B (en) Multi-domain resource allocation method and device in network slice
CN111614754B (en) Fog-calculation-oriented cost-efficiency optimized dynamic self-adaptive task scheduling method
CN109787915A (en) Flow control methods, device, electronic equipment and the storage medium of network access
CN113037877A (en) Optimization method for time-space data and resource scheduling under cloud edge architecture
CN115421930B (en) Task processing method, system, device, equipment and computer readable storage medium
WO2023222061A1 (en) Intent-driven wireless network resource conflict resolution method and apparatus
CN110402567A (en) Central caching is based in network centered on information
CN113687875A (en) Vehicle task unloading method and device in Internet of vehicles
CN111328107B (en) Multi-cloud heterogeneous mobile edge computing system architecture and energy optimization design method
CN115297008A (en) Intelligent computing network-based collaborative training method and device, terminal and storage medium
CN113032149B (en) Edge computing service placement and request distribution method and system based on evolution game
CN114116209A (en) Spectrum map construction and distribution method and system based on deep reinforcement learning
CN111885551B (en) Selection and allocation mechanism of high-influence users in multi-mobile social network based on edge cloud collaborative mode
CN110224861A (en) The implementation method of adaptive dynamic heterogeneous network selection policies based on study
CN116843016A (en) Federal learning method, system and medium based on reinforcement learning under mobile edge computing network
CN114461299B (en) Unloading decision determining method and device, electronic equipment and storage medium
CN115051999B (en) Energy consumption optimal task unloading method, device and system based on cloud edge cooperation
CN117172627B (en) Service execution method, device, equipment and storage medium
CN113485718B (en) Context-aware AIoT application program deployment method in edge cloud cooperative system
CN115551105B (en) Task scheduling method, device and storage medium based on 5G network edge calculation
CN116633932B (en) Dynamic scheduling system for cloud computing resource pool
CN115623366B (en) Route wavelength distribution method, device, terminal and medium of satellite all-optical network
CN117251276B (en) Flexible scheduling method and device for collaborative learning platform
CN113923223B (en) User allocation method with low time cost in edge environment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant