CN115658230A - Method and system for arranging high-performance containers in cloud data center - Google Patents
Method and system for arranging high-performance containers in cloud data center Download PDFInfo
- Publication number
- CN115658230A CN115658230A CN202211339950.8A CN202211339950A CN115658230A CN 115658230 A CN115658230 A CN 115658230A CN 202211339950 A CN202211339950 A CN 202211339950A CN 115658230 A CN115658230 A CN 115658230A
- Authority
- CN
- China
- Prior art keywords
- container
- time slot
- decision
- data center
- cloud data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 230000005012 migration Effects 0.000 claims abstract description 57
- 238000013508 migration Methods 0.000 claims abstract description 57
- 238000005457 optimization Methods 0.000 claims abstract description 51
- 238000005265 energy consumption Methods 0.000 claims abstract description 39
- 230000015654 memory Effects 0.000 claims description 25
- 230000000694 effects Effects 0.000 claims description 13
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000010276 construction Methods 0.000 claims description 6
- 230000014509 gene expression Effects 0.000 claims description 5
- LFQSCWFLJHTTHZ-UHFFFAOYSA-N Ethanol Chemical compound CCO LFQSCWFLJHTTHZ-UHFFFAOYSA-N 0.000 claims description 3
- 238000012512 characterization method Methods 0.000 claims description 3
- 238000009499 grossing Methods 0.000 claims description 3
- 230000007704 transition Effects 0.000 claims description 3
- 230000007774 longterm Effects 0.000 claims description 2
- 238000007726 management method Methods 0.000 claims description 2
- 230000001131 transforming effect Effects 0.000 claims 1
- 238000004422 calculation algorithm Methods 0.000 description 13
- 238000010586 diagram Methods 0.000 description 5
- 238000004590 computer program Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 101100313471 Streptomyces sp getA gene Proteins 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 238000003064 k means clustering Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and a system for arranging high-performance containers of a cloud data center, which comprises the steps of obtaining overall information of the cloud data center, wherein the overall information comprises the number of servers, the upper limit of resources of each server, the minimum power of each server, the maximum power of each server and the bandwidth budget of an average migration network provided by the cloud data center; acquiring information of all containers in a section of continuous time slot, wherein the information comprises the number of the containers, the resource request quantity of each container, the mirror image size of each container, the running time of each container and the migration attribute of each container; and according to the information of all containers in the continuous time slot and the overall information of the cloud data center, constructing and solving an optimization problem with minimized average energy consumption of the cloud data center as a target, and obtaining a container arrangement decision of the continuous time slot. The invention can obtain the current time slot container arrangement decision under the condition of meeting various resource constraint conditions under the condition that the number of future container creation requests and container specification parameters cannot be accurately obtained, and is efficient, energy-saving and environment-friendly.
Description
Technical Field
The invention relates to the field of cloud computing, in particular to a container arrangement method and system for a cloud data center.
Background
Cloud computing is a novel business computing model, and resources of servers in a data center are pooled through a virtualization technology, so that computing resources are provided in the form of on-demand services through the internet. Enterprises or individual users deploy the business in a cloud mode according to self requirements, do not need to purchase, configure or manage resources by themselves, and only need to pay according to the amount and time of actually used resources. Cloud computing makes it easier for enterprises and individuals to deploy applications and reduces various maintenance costs. The container is one of virtualization technology, compares in traditional virtual machine, and the container has opens and stops advantage such as fast, lightweight, expansibility is strong, isolation is good, therefore the container becomes the alternative technology of virtual machine, is widely used in the cloud data center. The container creates container instances through mirroring, and each container instance has a resource request, a running time and a migration attribute of the container instance.
With the continuous increase of the demand of cloud computing, the scale of the cloud data center is also rapidly increased, and the problem of high energy consumption of the cloud data center is exposed. Energy consumption of a cloud data center is composed of a number of aspects, including server energy consumption. The energy consumption of a single server is related to the utilization rate of computing resources of the server and the energy consumption attribute of the server, and the no-load server can enter a dormant state to reduce the energy consumption. In the cloud data center, as the number of containers is continuously increased and a proper high-performance container arrangement strategy is lacked, the containers are dispersed on different servers, and a large number of servers are in a low resource utilization state, so that the energy consumption of the data center is increased. A container arrangement method for reducing energy consumption is urgently needed in a cloud data center.
However, the number of container creation requests changes continuously with time, and it is difficult to find the optimal high-performance container arrangement strategy in a long time because the actual inferred number of requests of future users cannot be accurately known in advance, and the running time, resource request parameters, and migration attribute characteristics between containers are different. The existing cloud data center container arrangement strategy is simple to move according to the virtual machine arrangement strategy, and the migration attribute of the container is not considered, so that the energy-saving effect of the arrangement strategy cannot reach an ideal condition, and the energy consumption of the cloud data center cannot be obviously reduced. The above problems need to be solved.
Disclosure of Invention
The purpose of the invention is as follows: the invention provides a method for arranging high-performance containers of a cloud data center and a system for arranging the high-performance containers of the cloud data center, aiming at solving the defects of the prior art.
The technical scheme is as follows: in order to solve the technical problem, the invention provides a method for arranging a high-performance container in a cloud data center, which comprises the following steps:
acquiring overall information of a cloud data center, wherein the overall information comprises the number of servers, the resource upper limit of each server, the minimum power of each server, the maximum power of each server and the average migration network bandwidth budget provided by the cloud data center;
acquiring information of all containers in a section of continuous time slot, wherein the information comprises the number of the containers, the resource request quantity of each container, the mirror image size of each container, the running time of each container and the migration attribute of each container; the section of continuous time slot is from time slot 1 to time slot T, T >1;
according to the information of all containers in the continuous time slot and the overall information of the cloud data center, constructing and solving an optimization problem with minimized average energy consumption of the cloud data center as a target, and taking a solving result as a container arrangement decision of the continuous time slot; the container scheduling decision is the scheduling deployment position of each container in each time slot from the starting time slot 1 to the time slot T.
The invention also provides a high-performance container arrangement system of the cloud data center, which comprises the following components:
a system initialization unit configured to initialize a dynamic parameter raw value;
an input acquisition unit configured to acquire information as an input of the other module;
a decision storage unit configured to obtain and store a container arrangement decision for each time slot;
the arrangement decision feedback unit is configured to return a deployment effect corresponding to a certain container arrangement decision;
and the scheduling decision generating unit is configured to acquire the container scheduling decision in the time slot.
Preferably, the scheduling decision generating unit includes an optimization problem constructing module and an optimization problem solving module:
wherein the optimization problem construction module: the method is configured to realize the construction of the scheduling optimization problem of the time slot container by adopting any one of the methods according to the container request of the arrival of the time slot and the scheduling decision of the cloud data center container of the previous time slot;
wherein the optimization problem solving module: is configured to implement the solution of the container scheduling optimization problem in the time slot by using any one of the methods described above, and obtain the final container scheduling decision of the time slot.
The invention also provides a computer device, which comprises:
one or more processors; and
one or more memories; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, which when executed by the processors, implement the steps of the cloud datacenter high performance container orchestration method as recited in any one of the above.
The invention also provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the cloud data center high-performance container orchestration method according to any one of the above.
Has the advantages that: the invention provides a chain type service entity placing method taking minimized user response time as a target through an edge computing environment, which obtains a placing scheme of a chain type service entity through constructing a network model, a time delay model and a cost model of the edge network environment, combining an objective function and constraint conditions of a chain type service entity placing problem in edge computing and through a heuristic algorithm based on a K-Means clustering algorithm and a greedy algorithm, and can obtain an approximately optimal result in lower time complexity.
Compared with the prior art, the method and the system for arranging the high-performance container of the cloud data center have the following beneficial effects: under the condition of meeting various resource constraint conditions and limitations in various scenes, the method establishes an optimization problem by taking the average energy consumption of the minimized cloud data center as a target, converts and solves the optimization problem, and obtains a container arrangement decision in each time slot. The high-efficiency container arrangement method and system for the cloud data center can obtain the current time slot container arrangement decision according to the container creation request and the previous time slot container arrangement condition at the beginning of each time slot under the condition that the number of future container creation requests and the container specification parameters cannot be accurately obtained, arrange newly arrived containers and operated containers, achieve the effect of minimizing the average energy consumption in the cloud data center scene under the condition that various resource constraint conditions are met, effectively reduce the average energy consumption of the cloud data center, and are efficient, energy-saving and environment-friendly.
Drawings
Fig. 1 is a schematic flowchart illustrating an arrangement method of a cloud data center high performance container according to an embodiment;
fig. 2 is a schematic structural diagram of an exemplary high performance container orchestration system;
FIG. 3 is a schematic diagram of the orchestration decision feedback unit of FIG. 2;
FIG. 4 is a schematic diagram of the orchestration decision generating unit of FIG. 2;
fig. 5 is a comparison diagram of simulation of average energy consumption of the cloud data center high-performance container arrangement method and other arrangement algorithms provided in the embodiment.
Detailed Description
The present invention will be described in further detail with reference to examples, which are not intended to limit the present invention.
As shown in fig. 1, the method for arranging a high performance container in a cloud data center according to this embodiment includes the following steps:
(S1) acquiring/collecting overall information of a cloud data center, wherein the overall information comprises the number of servers, the resource upper limit of each server (including the computing resource upper limit of each server and the memory resource upper limit of each server), the minimum power of each server, the maximum power of each server and the average migration network bandwidth budget provided by the cloud data center;
(S2) acquiring information of all containers in a section of continuous time slot, wherein the information comprises the number of the containers, the resource request quantity of each container, the mirror image size of each container, the running time of each container and the migration attribute of each container; the section of continuous time slot is from time slot 1 to time slot T, T >1;
(S3) according to the information of all containers in the continuous time slot and the overall information of the cloud data center, constructing and solving an optimization problem (1) with the minimum average energy consumption of the cloud data center as a target, and taking a solving result as a container arrangement decision of the continuous time slot; the container scheduling decision is the scheduling deployment position of each container in each time slot from the starting time slot 1 to the time slot T;
in the method provided by this embodiment, after the optimization problem (1) with a goal of minimizing the average energy consumption of the cloud data center is constructed in the step (S3), the method further includes:
(S31) converting the constructed optimization problem (1) with the aim of minimizing the average energy consumption of the cloud data center, and decomposing the optimization problem into an optimization problem (2) of each time slot;
(S32) when each time slot starts, actually solving the optimization problem (2), namely solving the optimization problem (2) aiming at the container arrangement in the time slot to obtain the final container arrangement decision of the time slot (namely the time slot).
Specifically, the step (S32) of solving the optimization problem (2) of arranging containers in the time slot to obtain a final container arrangement decision of the time slot specifically includes the following steps:
l1) an initial container scheduling decision S to obtain this time slot 0 : at the beginning of the time slot, a container arrangement decision of the previous time slot (referring to the final container arrangement decision of the previous time slot) is obtained, and an initialization container arrangement decision S in the time slot is generated on the basis of the decision S 0 ；
L2) making decisions S in the initialization container by loop iteration 0 Updating the time slot to obtain the final container arrangement decision S of the time slot f (i.e. make a decision S on the initial container 0 Performing f times of loop iteration, and continuously updating the container arrangement decision until the final container arrangement decision of the time slot is obtainedPlan S f )；
L3) updating the dynamic parameters according to the final container arrangement decision in the time slot, that is: final container arrangement decision S according to the obtained time slot f Updating a virtual queue variable q t (ii) a In this embodiment, it can be said that the final container scheduling decision S is made according to the obtained current time slot f And final container arrangement decision of the last time slot, and updating the virtual queue variable.
Specifically, the method aims to achieve the effect of minimizing the energy consumption of the average server of the cloud data center under the condition of being limited by various resource constraints of the cloud data center: the established optimization problem (1) aiming at minimizing the average energy consumption of the cloud data center is as follows:
optimizing the target:
constraint conditions are as follows:
1.1 For any container i, the decision constraints are programmed:
1.2 For any container i, the runtime orchestrates decision constraints:
1.3 For any one container i): scheduling decision constraints outside the run time:
1.4 For any server j, compute resource constraints:
1.5 For any server j, memory resource constraints:
1.6 For any container i, the migration property constraint:
1.7 Long term migration network bandwidth constraints:
wherein [ x, y ]]Represents a set from integer x to integer y; t represents the total time slot number of the time coordinate axis; m is the total number of the servers in the cloud data center; n is the total number of the container running examples from the time slot 1 to the time slot T; e i,j,t A decision whether the container i in the time slot t is arranged to a server j or not; p t The sum of the power consumption of all servers in the cloud data center in the time slot t is obtained; t is t i,a The first time slot to run for container i; t is t i,e The last time slot run for container i; r is i,c Computing a resource request amount for container i; r is i,m The memory resource request quantity of the container i is set; l is j,c Is the total amount of computing resources of server j; l is a radical of an alcohol j,m The total amount of memory resources of the server j; m i Characterizing variables for the migration attributes of container i; c t Migration network bandwidth overhead caused by scheduling containers for a time slot t; c b Providing an average migration network bandwidth budget for the cloud data center;
wherein the time slot t is in the cloud data centerSum of all server power consumptions P t The calculation formula of (2) is as follows:
in the formula, mu t,j Represents the computing resource usage of server j at time slot t; p t,j Representing the energy consumption of the server j in the time slot t; p j,min Represents the lowest power consumption of server j; p j,max Represents the maximum power consumption of server j;
wherein the migration network bandwidth overhead C caused by the container scheduling within a time slot t t The calculation formula of (2) is as follows:
in the formula, H t,i A characterizing variable for whether a container I migrates in a time slot t, I i Representing the mirror size of container i.
In fact, based on a current time slot, e.g., a time slot before time slot 1, information of all containers in a next consecutive time slot (time slot 1 to time slot T) is difficult to obtain and predict, even if only information of all containers in time slot 1 to time slot T is available based on a current time slot T after time slot 1, container information in future time slots (i.e., time slot T +1 to time slot T) is also difficult to obtain and predict. Therefore, in order to obtain an effective solution according to the actual situation, the method needs to convert the optimization problem of a section of continuous time slots into the optimization problem of each time slot. Further, the transformation of the optimization problem (1) comprises:
decomposing the optimization problem (1) into each time slot, and actually solving the following optimization problem (2) at the beginning of each time slot:
optimizing the target: min (q) t *C t +V*P t )
Constraint conditions are as follows: including constraints 1.1) to 1.6)
In the formula, q t For continuously updated virtual queue variables, its initial value q 1 =0, representing the reserve amount of the quota of the migration network bandwidth expenditure caused by container arrangement in each time slot at the time slot t from the beginning time slot 1 of the continuous time slot to the previous time slot t-1 of the current time slot t, which exceeds the average migration network bandwidth budget (representing the degree of importance for dynamic adjustment of the migration network bandwidth expenditure); v is a preset weight adjustment parameter (indicating the degree of importance to energy consumption).
Meanwhile, the embodiment provides a method for arranging a high-performance container in a cloud data center, which further comprises the following steps:
storing/reading a final container arrangement decision of a certain time slot; the container arrangement decision is the arrangement and deployment positions of all containers in the time slot;
calculating/obtaining the deployment effect of a certain container arrangement decision in a certain time slot; the deployment effect comprises the sum P of the power consumptions of all servers in the time slot of the cloud data center under the certain container arrangement decision t And the bandwidth expense C of the migration network caused by arranging a decision on the certain container in the time slot of the cloud data center t (ii) a The deployment effect also includes the corresponding calculation result (also referred to as score) U = q t *C t +V*P t 。
In the above step L1), at the beginning of the time slot, the container scheduling decision of the previous time slot is obtained, and based on this, an initialization container scheduling decision S in the time slot is generated 0 (i.e., at the beginning of each timeslot, a feasible initial strategy is obtained for the optimization problem (2)Omitted), specifically including:
l11) obtaining a container arrangement decision of the previous time slot;
l12) on the basis of the container arrangement decision of the previous time slot, keeping the arrangement position of the operated container unchanged, and for each newly arrived container in the time slot, arranging the newly arrived containers to a server with equal probability that both available computing resources and available memory resources meet the container resource requirement, thereby obtaining an initial container arrangement decision S of the time slot 0 。
The step L2) is implemented by loop iteration, and the decision S is arranged in the initialization container 0 Updating the time slot to obtain the final container arrangement decision S of the time slot f Including making decisions S on the initial container 0 Performing f times of loop iteration (wherein f is the number of loop iteration), and continuously updating the container arrangement decision until a final container arrangement decision S is obtained f Wherein the loop iteration specifically comprises:
after the kth round of loop iteration is finished, the obtained container arrangement decision is S k ；
Making a decision S based on the container k In a newly arrived container or a container which is running and has a migration attribute characterization variable of 1, randomly selecting a container i' with equal probability;
selecting a server j' from all servers with available computing resources and available memory resources meeting container requirements at equal probability;
setting the arrangement position of the container i 'as a server j', thereby obtaining the arrangement position S k Is obtained a container arrangement decision S' k ；
Make a decision S for a container k And S' k The calculation expressions related to the optimization objectives are respectively substituted into the optimization problem (2) to obtain the results U and U' (it can also be said that the decision S is made on the container layout respectively) k And S' k Carrying out quantitative scoring to respectively obtain corresponding calculation results, namely scores U and U'); the method also corresponds to a step of calculating/obtaining a deployment effect of a certain container scheduling decision in a certain time slot (here, the time slot) in the method; excellence of the above optimization problem (2)Calculating expressions related to the targets, namely calculating expressions corresponding to corresponding calculation results in the step of the method, namely scores; wherein U = q t *C t +V*P t ，U′＝q t *C′ t +V*P′ t Here: c t 、C′ t Respectively representing the arrangement decision S of the containers in the time slot of the cloud data center k 、S′ k Resulting migration network bandwidth expenditure, P t 、P′ t Respectively representing container arrangement decision S in the time slot of the cloud data center k 、S′ k Sum of all server power consumptions.
Order to
Wherein tau is a strategy transition smoothing parameter, eta represents selection S' k As S k+1 Such that with a probability of η, S k+1 Is S' k With a probability of 1-eta, S k+1 Is S k ；
After each loop iteration, k is incremented by 1 to enter the next loop iteration, i.e. so that: k = k +1, wherein k is greater than or equal to 1 and less than or equal to f; after f times of cycle iteration are completed, the final container arrangement decision S of the time slot is obtained f 。
The step L3) is carried out according to the obtained final container arrangement decision S of the time slot f Updating a virtual queue variable q t (to make decision S by arranging the resulting final containers of the present time slot f Performing a virtual queue variable q as feedback t Update of (2) specifically including:
final container arrangement decision S according to the obtained time slot f And the final container arrangement decision of the previous time slot, and calculating the final container arrangement decision S of the current time slot according to the final container arrangement decision S of the current time slot f ) Migration network bandwidth expenditure C resulting from container orchestration t Updating the virtual queue variable q t+1 ＝[q t +C t -C b ] + Wherein [. X] + Refers to max {. 0} (meaning that the representation is compared to 0, taking the larger value).
Meanwhile, the embodiment provides a cloud data center high performance container arrangement system, as shown in fig. 2, including:
a system initialization unit configured to initialize a dynamic parameter raw value;
an input acquisition unit configured to acquire information as an input to the other module;
a decision storage unit configured to obtain and store a container arrangement decision for each time slot;
the scheduling decision feedback unit is configured to return a deployment effect corresponding to a certain container scheduling decision; (may also be said to be configured to quantitatively score container scheduling decisions, corresponding to the steps of the above method of calculating/obtaining the deployment effect of a certain container scheduling decision in a certain time slot);
and the scheduling decision generating unit is configured to acquire the container scheduling decision in the time slot.
In this embodiment, the input acquiring unit includes a data center information acquiring module and a container information acquiring module;
wherein the data center information acquisition module: the cloud data center management method comprises the steps that overall information of the cloud data center is obtained, wherein the overall information comprises the number of servers, resource upper limits of the servers, minimum power of the servers, maximum power of the servers and average migration network bandwidth budget provided by the cloud data center;
the system comprises a container information acquisition module, a data processing module and a data processing module, wherein the container information acquisition module is configured to acquire all container information of a section of continuous time slot from a starting time slot to a current time slot, and the container information comprises the number of containers, the resource request amount of each container, the mirror image size of each container, the running time of each container and the migration attribute of each container;
in this embodiment, an operation flow of the arrangement decision feedback unit is shown in fig. 3, and includes an arrangement decision energy consumption feedback module and an arrangement decision migration expense feedback module;
wherein the arrangement decision energy consumption feedback module: the method is configured to return the sum P of the power consumptions of all servers in the cloud data center under the container arrangement decision in a time slot according to the overall information of the cloud data center aiming at the container arrangement decision in the time slot t Is (can also be)Called the energy consumption expenditure of all servers in the cloud data center under the scheduling decision of the container in the time slot), the formula is as follows:
in the formula, mu t,j Representing the computing resource utilization rate of the server j in the time slot t; p t,j Representing the energy consumption of the server j in the time slot t; p j,min Represents the lowest power consumption of server j; p j,max Represents the maximum power consumption of server j; m is the total number of servers in the cloud data center; n is the total number of running instances in the container from the time slot 1 to the time slot T; e i,j,t A decision whether the container i is arranged to the server j or not in the time slot t; (P) t The sum of the power consumption of all servers in the cloud data center in the time slot t; ) R i,c Computing a resource request amount for container i; l is j,c Is the total amount of computing resources of server j;
wherein the orchestration decision migration expense feedback module: configured to return all migration network bandwidth expenses C caused by a certain container arrangement decision in a time slot according to final container arrangement decision information of a previous time slot for the certain container arrangement decision in the time slot t The formula is as follows:
wherein [ x, y [ ]]Represents fromA set of integers x through y; i is i Represents the mirror size of container i; t is t i,a The first time slot to run for container i; t is t i,e The last time slot run for container i; r is i,m Is the memory resource request amount of container i. H t,i A characterizing variable, C, for whether a container i migrates during a time slot t t Migration network bandwidth overhead resulting from the container scheduling within time slot t.
The scheduling decision generating unit comprises an optimization problem constructing module and an optimization problem solving module, and the operation flow is shown in fig. 4:
wherein the optimization problem construction module: the method is configured to implement model construction of the scheduling optimization problem of the time slot container by adopting any one of the above cloud data center high-performance container scheduling methods according to the container request arriving at the time slot and the scheduling decision of the cloud data center container at the previous time slot, that is, the optimization problem (2) is constructed according to the container request arriving at the time slot and the current scheduling situation of the cloud data center container, and is as follows:
optimizing the target: min (q) t *C t +V*P t )
Constraint conditions are as follows:
1.1 For any container i, the decision constraints are programmed:
1.2 For any container i, the run-time orchestration decision constraints:
1.3 For any one container i): scheduling decision constraints outside the runtime:
1.4 For any server j, compute resource constraints:
1.5 For any server j, memory resource constraints:
1.6 For any container i, the migration property constrains:
in the formula, M i Characterizing variables for the migration attributes of container i; q. q.s t For continuously updated virtual queue variables, its initial value q 1 =0, representing the reserve amount of the quota of the migration network bandwidth expenditure caused by container arrangement in each time slot at the time slot t from the beginning time slot 1 of the continuous time slot to the previous time slot t-1 of the current time slot t, which exceeds the average migration network bandwidth budget (representing the degree of importance for dynamic adjustment of the migration network bandwidth expenditure); v is a preset weight adjustment parameter (indicating the degree of importance to energy consumption).
Wherein the optimization problem solving module: the method is configured to implement solution of an optimization problem of container arrangement in the time slot by using any one of the cloud data center high-performance container arrangement methods, so as to obtain a final container arrangement decision of the time slot. In this embodiment, the problem of optimizing container arrangement in the time slot is solved to obtain a final container arrangement decision of the time slot, and the specific steps are as follows:
l1) at the beginning of this time slot, getA time slot container scheduling decision, on the basis of which an initialization container scheduling decision S within the time slot is generated 0 The method comprises the following steps:
l11) obtaining a container arrangement decision of the previous time slot;
l12) on the basis of the container arrangement decision of the previous time slot, keeping the arrangement position of the operated container unchanged, and for each newly arrived container in the time slot, arranging the newly arrived containers to a server with equal probability that the available computing resource and the available memory resource all meet the container resource requirement, thereby obtaining an initial container arrangement decision S of the time slot 0 ；
L2) making decisions S in the initialization container by loop iteration 0 Updating the time slot to obtain the final container arrangement decision S of the time slot f Namely: make a decision S for the initial container 0 F times of loop iteration are carried out, the container arrangement decision is continuously updated until the final container arrangement decision S of the time slot is obtained f Wherein the loop iteration specifically comprises:
l21) after the k round of loop iteration is finished, the obtained container arrangement decision is S k ；
L22) decision S on the basis of container arrangement k In a newly arrived container or a container which is running and has a migration attribute characterization variable of 1, randomly selecting a container i' with equal probability;
l23) selecting a server j' at equal probability randomly from all servers with available computing resources and available memory resources meeting the container requirement;
l24) setting the arrangement position of the container i 'to the server j', thereby at S k Obtaining a container arrangement decision S' k ；
L25) make decision S for container arrangement k And S' k Respectively bringing into calculation expressions related to the optimization target of the optimization problem (2) to respectively obtain results U and U'; here, the method corresponds to the step of calculating/obtaining the deployment effect of a certain container scheduling decision in a certain time slot (here, this time slot) in the above method, and corresponds to the step of calculating/returning a certain container scheduling decision in the above system through the scheduling decision feedback unitA deployment effect corresponding to the container arrangement decision (which may also be said to be a quantitative score for a certain container arrangement decision);
l26) order
Wherein tau is a strategy transition smoothing parameter, eta represents selected S' k As S k+1 Such that with a probability of η, S k+1 Is S' k With a probability of 1-eta, S k+1 Is S k ；
Loop iteration steps L21) to L26), after each loop iteration, k is incremented by 1 to enter the next loop iteration, i.e. such that: k = k +1, wherein k is greater than or equal to 1 and less than or equal to f; after f times of cycle iteration are completed, the final container arrangement decision S of the time slot is obtained f 。
L3) updating the dynamic parameters according to the final container scheduling decision in the time slot, that is: final container arrangement decision S based on the obtained time slot f Updating a virtual queue variable q t (ii) a In this embodiment, it can be said that the final container arrangement decision S according to the obtained local time slot f And final container arrangement decision of the last time slot, and updating the virtual queue variable.
Fig. 5 shows comparison simulation data of average energy consumption of the high-performance container arrangement method for the cloud data center and other arrangement algorithms (the ordinate is average energy consumption), which shows average energy consumption of the cloud data center caused by different container arrangement methods. The cloud data center high-performance container arrangement method provided by the embodiment is abbreviated as "LM", and the compared container arrangement method comprises the following steps: greedy scheduling algorithm and container energy-saving Migration algorithm (abbreviated as "Greedy & Migration"); the NextFit scheduling algorithm plus the container energy-saving Migration algorithm (referred to as NF & Migration for short); a FirstFitDestructing scheduling algorithm plus a container energy-saving Migration algorithm (abbreviated as 'FFD & Migration'); the container migration zone is divided into a first FitDecreating scheduling algorithm and a container energy-saving migration algorithm (referred to as 'Segmentation' for short). As can be seen from fig. 5, compared with other existing container arrangement methods, the high-performance container arrangement method for the cloud data center provided by the embodiment reduces the average energy consumption of the server by more than 16.29% and even reaches 31.86%, can effectively reduce the average energy consumption of the cloud data center, and is efficient, energy-saving and environment-friendly.
The invention also provides a computer device, which comprises: one or more processors; and one or more memories; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, which when executed by the processors, implement the steps of the cloud data center high performance container orchestration method according to any of the above.
The invention also provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the cloud data center high-performance container orchestration method according to any one of the above.
The container orchestration decision described herein may also be referred to as a container orchestration policy. A container arrangement decision as described herein may also be referred to as a container arrangement decision. The method for arranging the high-performance containers in the cloud data center can also be referred to as a method for arranging the high-performance containers in the cloud computing center. The cloud data center high-performance container arrangement system may also be referred to as a cloud computing center high-performance container arrangement system.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium. In the context of the present invention, the computer-readable storage medium may be considered tangible and non-transitory. Non-limiting examples of a non-transitory tangible computer-readable storage medium include a non-volatile memory circuit (e.g., a flash memory circuit, an erasable programmable read-only memory circuit, or a mask read-only memory circuit), a volatile memory circuit (e.g., a static random access memory circuit or a dynamic random access memory circuit), a magnetic storage medium (e.g., an analog or digital tape or hard drive), and an optical storage medium (e.g., a CD, DVD, or blu-ray disc), among others.
Program code for implementing the methods of the present invention may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
Further, while operations are depicted in a particular order, this should be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the invention. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation can also be implemented in multiple implementations separately or in any suitable subcombination.
The above is only a preferred embodiment of the present invention, it should be noted that the above embodiment does not limit the present invention, and equivalent changes, various changes and modifications made by workers within the scope not departing from the technical spirit of the present invention are within the protection scope of the present invention.
Claims (10)
1. A method for arranging high-performance containers in a cloud data center is characterized by comprising the following steps:
acquiring total information of a cloud data center, wherein the total information comprises the number of servers, the resource upper limit of each server, the minimum power of each server, the maximum power of each server and the average migration network bandwidth budget provided by the cloud data center;
acquiring information of all containers in a section of continuous time slot, wherein the information comprises the number of the containers, the resource request quantity of each container, the mirror image size of each container, the running time of each container and the migration attribute of each container; the section of continuous time slot is from time slot 1 to time slot T, and T is more than 1;
according to the information of all containers in the continuous time slot and the overall information of the cloud data center, constructing and solving an optimization problem with minimized average energy consumption of the cloud data center as a target, and taking a solving result as a container arrangement decision of the continuous time slot; the container arrangement decision is the arrangement deployment position of each container in each time slot from the initial time slot 1 to the time slot T.
2. The cloud data center high-performance container arrangement method according to claim 1, wherein:
the optimization problem (1) which is constructed by taking minimum average energy consumption of the cloud data center as a target is as follows:
optimizing the target:
constraint conditions are as follows:
1.1 For any container i, the decision constraints are programmed:
1.2 For any container i, the runtime orchestrates decision constraints:
1.3 For any one container i): scheduling decision constraints outside the runtime:
1.4 For any server j, compute resource constraints:
1.5 For any server j, memory resource constraints:
1.6 For any container i, the migration property constraint:
1.7 Long term migration network bandwidth constraints:
wherein [ x, y ]]Represents a set from integer x to integer y; t represents the total time slot number of the time coordinate axis; m is the total number of servers in the cloud data center; n is the total number of the container running examples from the time slot 1 to the time slot T; e i，j，t A decision whether the container i is arranged to the server j or not in the time slot t; p is t The sum of the power consumption of all servers in the cloud data center in the time slot t; t is t i，a The first time slot to run for container i; t is t i，e The last time slot run for container i; r i，c Computing resource request amounts for container i; r i，m The memory resource request quantity of the container i; l is a radical of an alcohol j，c Total amount of computing resources for server j; l is a radical of an alcohol j，m The total amount of memory resources of the server j; m is a group of i Characterizing variables for the migration attributes of container i; c t Migration network bandwidth overhead caused by scheduling containers for a time slot t; c b Providing an average migration network bandwidth budget for the cloud data center;
wherein the sum P of the power consumptions of all servers in the cloud data center in the time slot t t Comprises the following steps:
wherein:
wherein:
in the formula, mu t，j Representing the computing resource utilization rate of the server j in the time slot t; p t，j Representing the energy consumption of the server j in the time slot t; p is j，min Represents the lowest power consumption of server j; p is j，max Represents the maximum power consumption of server j;
wherein the migration network bandwidth cost C caused by the arrangement of containers within a time slot t t Comprises the following steps:
wherein H t，i A characterizing variable for whether container i migrates during time slot t:
in the formula I i Representing the mirror size of container i.
3. The cloud data center high-performance container orchestration method according to claim 2, wherein transforming the optimization problem (1) that is constructed with the goal of minimizing the average energy consumption of the cloud data center comprises:
decomposing the optimization problem (1) into each time slot, and actually solving the following optimization problem (2) at the beginning of each time slot:
optimizing the target: min (q) t *C t +V*P t )
Constraint conditions are as follows: including constraints 1.1) to 1.6)
In the formula, q t Is a virtual queue variable, whose initial value q is 1 =0; v is a preset weight adjustment parameter.
4. The cloud data center high-performance container arrangement method according to claim 3, wherein solving a container arrangement optimization problem (2) in the time slot to obtain a final container arrangement decision of the time slot specifically comprises:
l1) an initial container scheduling decision S to obtain this time slot 0 ；
L2) making decisions S in the initialization container by loop iteration 0 Updating the time slot to obtain the final container arrangement decision S of the time slot f ；
L3) Final Container scheduling decision S based on the obtained time slot f Updating a virtual queue variable q t 。
5. The cloud data center high-performance container orchestration method according to claim 4,
an initial container arrangement decision S for the time slot obtained in the step L1) 0 The method comprises the following steps:
l11) obtaining a container arrangement decision of the previous time slot;
l12) on the basis of the container arrangement decision of the previous time slot, keeping the arrangement position of the operated containers unchanged, and for each newly arrived container in the time slot, arranging the newly arrived containers into a table at random with equal probabilityA server for satisfying the container resource demand by computing resources and available memory resources, thereby obtaining an initial container arrangement decision S of the time slot 0 。
6. The cloud data center high-performance container orchestration method according to claim 4,
the step L2) arranges a decision S in the initialization container through loop iteration 0 Updating the time slot to obtain the final container arrangement decision S of the time slot f Including making decisions S on the initial container 0 Performing f times of loop iteration, and continuously updating the container arrangement decision until a final container arrangement decision S is obtained f Wherein the loop iteration specifically comprises:
after the kth round of loop iteration is finished, the obtained container arrangement decision is S k ；
Making a decision S based on the container k In a newly arrived container or a container which is running and has a migration attribute characterization variable of 1, randomly selecting a container i' with equal probability;
randomly selecting a server j' in equal probability from all servers with available computing resources and available memory resources meeting the requirements of the container;
setting the arrangement position of the container i 'as a server j', thereby obtaining the arrangement position S k Is obtained a container arrangement decision S' k ；
Make a decision S for a container k And S' k Respectively substituting calculation expressions related to the optimization target of the optimization problem (2) to respectively obtain results U and U';
order to
Wherein tau is a strategy transition smoothing parameter, eta represents selected S' k As S k+1 Such that with a probability of η, S k+1 Is S' k With a probability of 1-eta, S k+1 Is S k ；
After f times of cycle iteration are completed, the final container arrangement decision S of the time slot is obtained f 。
7. The cloud data center high-performance container orchestration method according to claim 4,
said step L3) is based on the final container arrangement decision S of this time slot f Updating a virtual queue variable q t The method specifically comprises the following steps:
final container arrangement decision S based on the obtained time slot f Calculating the bandwidth expense C of the migration network caused by arranging the containers in the time slot according to the bandwidth expense C t Updating the virtual queue variable q t+1 ＝[q t +C t -C b ] + Wherein [. X] + Refers to max {, 0}.
8. A cloud data center high-performance container orchestration system, comprising:
a system initialization unit configured to initialize original values of the dynamic parameters;
an input acquisition unit configured to acquire information as an input to the other module;
a decision storage unit configured to obtain and store a container arrangement decision for each time slot;
the arrangement decision feedback unit is configured to return a deployment effect corresponding to a certain container arrangement decision;
and the scheduling decision generating unit is configured to acquire the container scheduling decision in the time slot.
9. The cloud data center high performance container orchestration system according to claim 8, wherein the input acquisition unit comprises: the system comprises a data center information acquisition module and a container information acquisition module;
wherein the data center information acquisition module: the cloud data center management method comprises the steps that overall information of the cloud data center is obtained, wherein the overall information comprises the number of servers, resource upper limits of the servers, minimum power of the servers, maximum power of the servers and average migration network bandwidth budget provided by the cloud data center;
the system comprises a container information acquisition module, a resource request module, a storage module and a migration module, wherein the container information acquisition module is configured to acquire all container information from a starting time slot to a current time slot, and the container information comprises the number of containers, the resource request amount of each container, the mirror image size of each container, the running time of each container and the migration attribute of each container;
the arrangement decision feedback unit comprises an arrangement decision energy consumption feedback module and an arrangement decision migration expense feedback module;
wherein the arrangement decision energy consumption feedback module: the method is configured to return the sum P of the power consumptions of all servers in the cloud data center under the container arrangement decision in a time slot according to the overall information of the cloud data center aiming at the container arrangement decision in the time slot t Comprises the following steps:
wherein:
wherein:
in the formula, mu t，j Representing the computing resource utilization rate of the server j in the time slot t; p is t，j Representing the energy consumption of the server j in the time slot t; p is j，min Represents the lowest power consumption of server j; p j，max Represents the maximum power consumption of server j; m is the total number of servers in the cloud data center; n is the total number of running instances in the container from the time slot 1 to the time slot T; e i，j，t A decision whether the container i in the time slot t is arranged to a server j or not; r is i，c Computing resource request amounts for container i; l is j，c Is the total amount of computing resources of server j;
wherein the orchestration decision migration expense feedback module: is configured to return all the container arrangement decisions in a time slot according to the final container arrangement decision information of the previous time slot aiming at a certain container arrangement decision in the time slotMigration network bandwidth expenditure C t Comprises the following steps:
wherein H t，i A characterizing variable for whether container i migrates during time slot t:
wherein [ x, y [ ]]Represents a set from integer x to integer y; I.C. A i Represents the mirror size of container i; t is t i，a The first time slot to run for container i; t is t i，e The last time slot run for container i; r i，m Is the memory resource request amount of container i.
10. The cloud data center high-performance container orchestration system of claim 8, wherein the orchestration decision generation unit comprises an optimization problem building module and an optimization problem solving module:
wherein the optimization problem construction module: the method is configured to implement construction of the scheduling optimization problem of the time slot container according to the container request arriving at the time slot and the scheduling decision of the cloud data center container at the last time slot by adopting the method as claimed in claim 2 or 3;
wherein the optimization problem solving module: is configured to implement the solution of the container scheduling optimization problem in the time slot by using the method according to any one of claims 4 to 7, resulting in the final container scheduling decision of the time slot.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211339950.8A CN115658230A (en) | 2022-10-27 | 2022-10-27 | Method and system for arranging high-performance containers in cloud data center |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211339950.8A CN115658230A (en) | 2022-10-27 | 2022-10-27 | Method and system for arranging high-performance containers in cloud data center |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115658230A true CN115658230A (en) | 2023-01-31 |
Family
ID=84992388
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211339950.8A Pending CN115658230A (en) | 2022-10-27 | 2022-10-27 | Method and system for arranging high-performance containers in cloud data center |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115658230A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116028193A (en) * | 2023-03-29 | 2023-04-28 | 南京大学 | Big data task dynamic high-energy-efficiency scheduling method and system for mixed part cluster |
| CN117573374A (en) * | 2024-01-15 | 2024-02-20 | 北京大学 | System and method for server to have no perceived resource allocation |
-
2022
- 2022-10-27 CN CN202211339950.8A patent/CN115658230A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116028193A (en) * | 2023-03-29 | 2023-04-28 | 南京大学 | Big data task dynamic high-energy-efficiency scheduling method and system for mixed part cluster |
| CN117573374A (en) * | 2024-01-15 | 2024-02-20 | 北京大学 | System and method for server to have no perceived resource allocation |
| CN117573374B (en) * | 2024-01-15 | 2024-04-05 | 北京大学 | System and method for server to have no perceived resource allocation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109491790B (en) | Container-based industrial Internet of things edge computing resource allocation method and system | |
| CN115658230A (en) | Method and system for arranging high-performance containers in cloud data center | |
| Guo et al. | Cloud resource scheduling with deep reinforcement learning and imitation learning | |
| CN106055395B (en) | Deadline constrains workflow schedule method in a kind of cloud environment based on ant colony optimization algorithm | |
| Shi et al. | MDP and machine learning-based cost-optimization of dynamic resource allocation for network function virtualization | |
| CN110069341B (en) | Method for scheduling tasks with dependency relationship configured according to needs by combining functions in edge computing | |
| CN110058924B (en) | Multi-objective optimized container scheduling method | |
| CN104168318A (en) | Resource service system and resource distribution method thereof | |
| CN112052071B (en) | Cloud software service resource allocation method combining reinforcement learning and machine learning | |
| WO2021259246A1 (en) | Resource scheduling method and apparatus, electronic device, and computer-readable storage medium | |
| CN111274036A (en) | Deep learning task scheduling method based on speed prediction | |
| CN112416585A (en) | GPU resource management and intelligent scheduling method for deep learning | |
| CN111209095B (en) | Pruning method based on tree search in DAG parallel task scheduling | |
| WO2021109588A1 (en) | Data processing method, apparatus, electronic device, and readable storage medium | |
| CN109361750B (en) | Resource allocation method, device, electronic equipment and storage medium | |
| WO2020186872A1 (en) | Expense optimization scheduling method for deadline constraint under cloud scientific workflow | |
| TW201447763A (en) | System and method for controlling virtual machine | |
| CN108762927A (en) | The multiple target method for scheduling task of mobile cloud computing | |
| CN116069512B (en) | Serverless efficient resource allocation method and system based on reinforcement learning | |
| CN111913800B (en) | Resource allocation method for optimizing cost of micro-service in cloud based on L-ACO | |
| CN109710406A (en) | Data distribution and its model training method, device and computing cluster | |
| CN108415766B (en) | Rendering task dynamic scheduling method | |
| CN114968520B (en) | Task searching method and device, server and storage medium | |
| CN117331693A (en) | Task unloading method, device and equipment for edge cloud based on DAG | |
| CN115543626A (en) | Power defect image simulation method adopting heterogeneous computing resource load balancing scheduling |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |