CN107947983B - Fair distribution method for maximizing resource utilization rate in cloud platform - Google Patents

Fair distribution method for maximizing resource utilization rate in cloud platform Download PDF

Info

Publication number
CN107947983B
CN107947983B CN201711192325.4A CN201711192325A CN107947983B CN 107947983 B CN107947983 B CN 107947983B CN 201711192325 A CN201711192325 A CN 201711192325A CN 107947983 B CN107947983 B CN 107947983B
Authority
CN
China
Prior art keywords
resource
cloud platform
user
resources
management server
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201711192325.4A
Other languages
Chinese (zh)
Other versions
CN107947983A (en
Inventor
李伟东
张学杰
张骥先
李贵兵
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yunnan University YNU
Original Assignee
Yunnan University YNU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yunnan University YNU filed Critical Yunnan University YNU
Priority to CN201711192325.4A priority Critical patent/CN107947983B/en
Publication of CN107947983A publication Critical patent/CN107947983A/en
Application granted granted Critical
Publication of CN107947983B publication Critical patent/CN107947983B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/70—Admission control; Resource allocation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention discloses a fair allocation method for maximizing resource utilization rate in a cloud platform, which solves the virtual price of resources and the maximum task number of users under the condition of maximizing the resource utilization rate through a constructed optimization model on the basis of counting the resource requirements of all users and meeting constraint conditions, wherein the resource price is virtual. Therefore, on the basis of resource price virtualization, the maximum task number of the users under the equal constraint of resource allocation is obtained, the fairness of resource allocation is ensured, and the resource utilization rate is greatly improved.

Description

Fair distribution method for maximizing resource utilization rate in cloud platform
Technical Field
The invention belongs to the technical field of cloud computing, and particularly relates to a fair allocation method for maximizing resource utilization rate in a cloud platform.
Background
As the technology of cloud computing matures, more and more businesses or individuals submit computing and storage tasks to the cloud computing platform for execution. In 2011, the cloud computing resource allocation method mainly comprises a first-in first-out (FIFO) allocation method and a Proportional Fair (PF) allocation method. The first-in first-out allocation mode (FIFO) is that resources are allocated to users who submit calculation and storage tasks first according to the sequence of the calculation and storage tasks submitted by the users; the proportional fair allocation mode (PF) obtains a resource allocation scheme by solving a mathematical plan which meets resource constraints and aims at maximizing the sum of logarithms of the number of tasks of the users. The two methods may cause uneven distribution of resources and idle of part of resources, so that the overall utilization rate of the resources is low.
In 2011, american scholars proposed a fair allocation mechanism (DRF) for dominant resources, and rapidly applied a messs platform and a Yarn platform, which become the mainstream allocation mechanism for resource allocation and are applied to a plurality of fields.
The fair allocation of dominant resources mechanism (DRF) means that the resource allocation of a user submitting a computing or storage task should be determined by the dominant share of resources of the user, which is a kind of resources occupying the largest share among all the various resources already allocated to the user. In short, the fair allocation of dominant resources mechanism (DRF) attempts to maximize the smallest dominant share of resources among all users. However, the fair allocation mechanism of the occupied resources cannot solve the problem of low resource utilization rate in some cases.
Currently, because the working nature of the resource user depends heavily on some kind of resource, for example, the weather department depends on CPU in particular, and the media department depends on storage or bandwidth resources in particular, it is the case for the resource provider that the server CPU is fully loaded but a large amount of memory is idle or vice versa. Therefore, various virtual resources are managed in a unified mode, a high-performance computing platform is established, and the resource utilization rate can be effectively improved. But how to fairly and efficiently allocate computing resources is one of the major problems facing administrators.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a fair distribution method for maximizing the resource utilization rate in a cloud platform, so that the resource utilization rate can be greatly improved on the premise of ensuring fairness.
In order to achieve the above object, the fair allocation method for maximizing resource utilization in a cloud platform of the present invention is characterized by comprising the following steps:
(1) the user puts resource demands on a resource management server of the cloud platform (D)i1,Di2,…,Dim) Wherein, i is the user number, i is 1,2, …, n, n is the user number, m is the resource type number;
(2) the resource management server of the cloud platform acquires the total available amount C of each resource of the data center of the cloud platformr，r＝1,2,…,m；
The resource management server of the cloud platform acquires the resource requirements of the users and calculates the standardized resource requirements (d)i1,di2,…,dim) Wherein d isir＝Dir/Cr；
Let vrRepresenting the virtual price of a resource r, let xiIndicating the maximum number of tasks allocated to user i that can be processed, i.e. for resource dirMultiple of (3), on a cloud platformAn optimization model is constructed in the resource management server, namely, the resource utilization rate c is maximized, and the constraint conditions are as follows:
and (3) limiting the resource utilization rate:
Figure BDA0001481279870000021
for each resource r ═ 1,2, …, m (1);
and (4) limiting the total amount of resources:
Figure BDA0001481279870000022
for each resource r ═ 1,2, …, m (2);
resource allocation equal limitation:
Figure BDA0001481279870000023
v is the expected income of the cloud platform for all resources;
price non-negative limits:
Figure BDA0001481279870000024
vrr is 1,2, …, m (4) for each resource, which is more than or equal to 0;
the resource management server of the cloud platform solves the optimization model to obtain the virtual price v of the resource r of the optimal solutionrAnd the maximum number of tasks x of user iiThen according to the maximum number of tasks xiCalculating the resources (D) that user i should be allocatedi1xi,Di2xi,…,Dimxi) Finally, the virtual price v of the resource rrR 1,2, …, m, and the resource (D) to be allocatedi1xi,Di2xi,…,Dimxi) Transmitting to a user i;
(3) user i obtains virtual price v of resource rrAnd the resource (D) to be allocatedi1xi,Di2xi,…,Dimxi) Paying the corresponding resource, i.e. V/n, amount, cloudsAfter obtaining the user purchase information, a resource management server of the platform requests resources from a data center of the cloud platform;
(4) the data center of the cloud platform allocates resources and returns resource allocation information to the resource management server, such as an ip address, a user login name, a password and the like;
(5) after the resource management server obtains the resource allocation information, returning the resource allocation information to the user;
(6) and after the user obtains the resource allocation information, submitting the task to a data center for obtaining the cloud platform, and using the cloud platform resources.
The object of the invention is thus achieved.
According to the fair allocation method for maximizing the resource utilization rate in the cloud platform, on the basis of counting the resource requirements of all users, the virtual price of the resources and the maximum task number of the users under the condition of maximizing the resource utilization rate are solved through the constructed optimization model under the condition that the constraint conditions are met, wherein the resource price is virtual. Therefore, on the basis of resource price virtualization, the maximum task number of the users under the equal constraint of resource allocation is obtained, the fairness of resource allocation is ensured, and the resource utilization rate is greatly improved.
Drawings
Fig. 1 is a data flow diagram of a fair allocation method for maximizing resource utilization in a cloud platform according to a specific embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention is provided in order to better understand the present invention for those skilled in the art with reference to the accompanying drawings. It is to be expressly noted that in the following description, a detailed description of known functions and designs will be omitted when it may obscure the subject matter of the present invention.
Fig. 1 is a data flow diagram of a fair allocation method for maximizing resource utilization in a cloud platform according to a specific embodiment of the present invention.
In this embodiment, as shown in fig. 1, the data flow of the fair allocation method for maximizing resource utilization in the cloud platform of the present invention is as follows:
step S1: user puts resource demands on resource management server of cloud platform (D)i1,Di2,…,Dim) The resource management server of the cloud platform acquires the total available amount C of each resource of the data center of the cloud platformrThe constructed optimization model is used for solving the virtual price of the resources and the maximum task number of the user under the condition of meeting the constraint conditions; in this embodiment, the data center is a Hadoop server cluster; the solving process can be carried out by designing an algorithm, or by adopting special mathematical programming solving software CPLEX of IBM company;
with respect to standardized resource requirements, such as: there are two main types of resources in a data center, one is a CPU, one is a memory, the total available amount of the CPU is 12, the total available amount of the memory is 12, at a certain time, there are two users with computing requirements, the resource requirement of user 1 is 2 CPUs, 3 memories, denoted as (2, 3), the resource requirement of user 2 is 5 CPUs, 8 memories, denoted as (5, 8), here, for convenience of processing, by standardizing it, that is, the standardized resource requirement is the resource requirement/total resource amount of a certain user, so that the resource requirement of user 1 after standardization is (2/12,3/12), and the resource requirement of user 2 is (5/12, 8/12).
Step S2: the user obtains the virtual price of the resource and the resource (D) to be allocatedi1xi,Di2xi,…,Dimxi) Paying the cost of the corresponding resource, namely the sum of V/n;
step S3: after obtaining the user purchase information, a resource management server of the cloud platform requests resources from a data center of the cloud platform;
step S4: the data center of the cloud platform allocates resources and returns resource allocation information to the resource management server, such as an ip address, a user login name, a password and the like;
step S5: the resource management server returns the resource allocation information to the user after acquiring the resource allocation information;
step S6: after obtaining the resource allocation information, the user submits the task to a data center for obtaining the cloud platform and uses the cloud platform resources;
step S7: and the data center of the cloud platform executes tasks.
Specific examples
Considering that a data center of a certain cloud platform comprises 90 CPUs and 90GB memories, 3 users put forward requirements in total, 5 CPUs and 1GB memories are needed for users 1 and 2 to process one task, 1CPU and 2GB memories are needed for users 3 to process one task, and the expected profit of the platform on the resources is 90 yuan. According to the invention, 30 CPUs and 6GB memories can be respectively allocated to the users 1 and 2, 30 CPUs and 60GB memories can be allocated to the users 3, the utilization rate of the CPUs in the cloud platform is 1, and the utilization rate of the memories is 80%. In the example, the method provided by the invention has higher resource utilization rate than the existing fair allocation mechanism (such as proportional fairness and occupied resource fairness) on the premise of satisfying fairness. The calculation process is as follows:
1. standardizing the resource requirements proposed by the users to obtain that the resource requirements of the user 1 are (5/90,1/90), the resource requirements of the user 2 are (5/90,1/90) and the resource requirements of the user 3 are (1/90, 2/90);
2. setting the maximum utilization rate c of resources, and the resource allocation quantity of each user as x1,x2,x3Virtual price of CPU is v1Virtual price of memory is v2；
3. The following equation is derived from the present invention:
Figure BDA0001481279870000051
in this example, the optimal solution is calculated by solving the linear program poles.
By the limitation of
Figure BDA0001481279870000052
Get x1＝x2. Note that the utilization of at least one of the two types of resources is 1. Therefore, the following cases are discussed.
(1) When in
Figure BDA0001481279870000053
Then, get x1＝x2＝5,x340, but
Figure BDA0001481279870000054
Therefore, the algorithm constraint is not satisfied, and thus (5,5,40) is an infeasible solution;
(2) when in
Figure BDA0001481279870000055
And
Figure BDA0001481279870000056
when there is x3＝90-10x1If v is10, get
Figure BDA0001481279870000057
At this time have
Figure BDA0001481279870000058
If v is1Not equal to 0, for convenience of calculation, let
Figure BDA0001481279870000059
Thus, x1(5+ρ)＝(1+2ρ)x3To obtain
Figure BDA00014812798700000510
Figure BDA00014812798700000511
It is clear that when ρ is 0, i.e. v2When c is 0, the maximum value can be obtained
Figure BDA00014812798700000512
At this time, x1＝x2＝6,x3＝30；
(3) When in
Figure BDA00014812798700000513
And
Figure BDA00014812798700000514
when there is x1＝45-x3If v is1X is 0, d1＝2x3When it is equal to 30, there are
Figure BDA00014812798700000515
The algorithm constraints are not met and (30,30,15) is therefore an infeasible solution. If v is1Not equal to 0, for convenience of calculation, let
Figure BDA00014812798700000516
Thus, x1(5+ρ)＝(1+2ρ)x3To obtain
Figure BDA00014812798700000517
Therefore, the temperature of the molten metal is controlled,
Figure BDA00014812798700000518
is an infeasible solution.
(4) Determination of the price. From the constraint 3 and the last equation in the example, it can be determined that the platform expects a 90-dollar revenue for three users, and that the amount paid by each user is the same, i.e., 30-dollars, at which time v is known1＝90，v20. It should be noted that the resource price in the present invention is virtual, and the purpose is to allocate resources while ensuring fairness and improving resource utilization.
In summary, the virtual price-based shared computing resource allocation method obtains the final solution (6, 6, 30), and the resource utilization rate of the final solution is 80%, 30GB CPU and 6GB memory are allocated to the user 1 and the user 2, respectively, 30GB CPU and 60GB memory are allocated to the user 3, the utilization rate of the CPU in the data center is 100%, and the utilization rate of the memory is 80%.
In order to compare the effects of the invention, under the condition that the user resource needs are the same, the DRF (fair allocation of occupied resources) and the PF (fair allocation of proportion) are used for calculation.
When DRF (fair allocation of occupied resources) is used for the calculation. And finally, allocating 36 CPUs and 7.2GB memories to the user 1 and the user 2, and allocating 18GB CPUs and 36GB memories to the user 3, wherein the utilization rate of the CPUs is 100%, and the utilization rate of the system memories is 56%. When PF (proportional fair allocation mechanism) is used for calculation, 37.5 CPUs and 7.5GB memories are allocated to user 1 and user 2, and 15 CPUs and 30GB memories are allocated to user 3, at this time, the utilization rate of the system CPUs is 100%, and the utilization rate of the system memories is 50%.
By comparison, the method provided by the invention has higher resource utilization rate than proportional fairness and dominant resource fairness on the premise of meeting fairness, and solves the problem that the resource utilization rate of DRF (proportional fair resource allocation mechanism) and PF (proportional fair allocation mechanism) is too low
On the premise of fairness, the invention provides a brand-new resource allocation mechanism, which has higher resource utilization rate than the current popular resource allocation algorithms such as DRF (predominantly resource fair allocation mechanism) and PF (proportional fair allocation mechanism), and can be conveniently applied to the current server cluster platform such as hadoop and spark.
Although illustrative embodiments of the present invention have been described above to facilitate the understanding of the present invention by those skilled in the art, it should be understood that the present invention is not limited to the scope of the embodiments, and various changes may be made apparent to those skilled in the art as long as they are within the spirit and scope of the present invention as defined and defined by the appended claims, and all matters of the invention which utilize the inventive concepts are protected.

Claims (2)

1. A fair distribution method for maximizing resource utilization rate in a cloud platform is characterized by comprising the following steps:
(1) the user puts resource demands on a resource management server of the cloud platform (D)i1,Di2,…,Dim) Wherein, i is the user number, i is 1,2, …, n, n is the user number, m is the resource type number;
(2) the resource management server of the cloud platform acquires the total available amount C of each resource of the data center of the cloud platformr，r＝1,2,…,m；
The resource management server of the cloud platform acquires and counts the resource requirements of the userCalculating its standardized resource requirement (d)i1,di2,…,dim) Wherein d isir＝Dir/Cr；
Let vrRepresenting the virtual price of a resource r, let xiIndicating the maximum number of tasks allocated to user i that can be processed, i.e. for resource dirThe method comprises the following steps of (1) constructing an optimization model in a resource management server of the cloud platform, namely maximizing resource utilization rate c, wherein the constraint conditions are as follows:
and (3) limiting the resource utilization rate:
Figure FDA0002585551540000011
and (4) limiting the total amount of resources:
Figure FDA0002585551540000012
resource allocation equal limitation:
Figure FDA0002585551540000013
v is the expected income of the cloud platform for all resources;
price non-negative limits:
Figure FDA0002585551540000014
the resource management server of the cloud platform solves the optimization model to obtain the virtual price v of the resource r of the optimal solutionrAnd the maximum number of tasks x of user iiThen according to the maximum number of tasks xiCalculating the resources (D) that user i should be allocatedi1xi,Di2xi,…,Dimxi) Finally, the virtual price v of the resource rrR 1,2, …, m, and the resource (D) to be allocatedi1xi,Di2xi,…,Dimxi) Transmitting to a user i;
(3) user i obtains virtual price v of resource rrAnd the resource (D) to be allocatedi1xi,Di2xi,…,Dimxi) Paying the cost of the corresponding resources, namely the sum of V/n, and requesting the resources from a data center of the cloud platform after a resource management server of the cloud platform obtains user purchase information;
(4) the data center of the cloud platform allocates resources and returns resource allocation information to the resource management server;
(5) after the resource management server obtains the resource allocation information, returning the resource allocation information to the user;
(6) and after the user obtains the resource allocation information, submitting the task to a data center for obtaining the cloud platform, and using the cloud platform resources.
2. The fair allocation method of claim 1, wherein said solving the optimization model in step (2) is performed using IBM corporation specific mathematical programming solving software CPLEX.
CN201711192325.4A 2017-11-24 2017-11-24 Fair distribution method for maximizing resource utilization rate in cloud platform Active CN107947983B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711192325.4A CN107947983B (en) 2017-11-24 2017-11-24 Fair distribution method for maximizing resource utilization rate in cloud platform

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711192325.4A CN107947983B (en) 2017-11-24 2017-11-24 Fair distribution method for maximizing resource utilization rate in cloud platform

Publications (2)

Publication Number Publication Date
CN107947983A CN107947983A (en) 2018-04-20
CN107947983B true CN107947983B (en) 2020-10-16

Family

ID=61948703

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711192325.4A Active CN107947983B (en) 2017-11-24 2017-11-24 Fair distribution method for maximizing resource utilization rate in cloud platform

Country Status (1)

Country Link
CN (1) CN107947983B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109901929B (en) * 2019-03-04 2023-04-18 云南大学 Cloud computing task share fair distribution method under server level constraint

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8914511B1 (en) * 2009-06-26 2014-12-16 VMTurbo, Inc. Managing resources in virtualization systems
US9749208B2 (en) * 2014-06-30 2017-08-29 Microsoft Technology Licensing, Llc Integrated global resource allocation and load balancing
CN104618269B (en) * 2015-01-29 2017-08-29 南京理工大学 Cloud system utilization rate maximum resource distribution method based on horsepower requirements
CN106611277A (en) * 2016-05-13 2017-05-03 四川用联信息技术有限公司 Cloud resource scheduling method in mixed cloud model
CN106506607A (en) * 2016-10-19 2017-03-15 云南大学 Cloud computing resources distribution method based on fair credible two way auction mechanism
CN107193658A (en) * 2017-05-25 2017-09-22 重庆工程学院 Cloud computing resource scheduling method based on game theory

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
云环境中虚拟机资源分配算法研究;李孝天等;《互联网+应用》;20170915;第98-99页 *

Also Published As

Publication number Publication date
CN107947983A (en) 2018-04-20

Similar Documents

Publication Publication Date Title
US11201832B2 (en) Dynamic allocation of resources while considering resource reservations
US20190095249A1 (en) System, method, and medium for facilitating auction-based resource sharing for message queues in an on-demand services environment
US20150046279A1 (en) Auction-based resource sharing for message queues in an on-demand services environment
CN107688492B (en) Resource control method and device and cluster resource management system
US9256452B1 (en) Providing an instance availability estimate
US8756441B1 (en) Data center energy manager for monitoring power usage in a data storage environment having a power monitor and a monitor module for correlating associative information associated with power consumption
CN106789118B (en) Cloud computing charging method based on service level agreement
CN111026547A (en) Edge computing server resource allocation method based on auction mechanism
US20220121467A1 (en) A method and a system for managing the computing resources of a cloud computing platform
CN104092756A (en) Cloud storage system resource dynamic allocation method based on DHT mechanism
US20220229701A1 (en) Dynamic allocation of computing resources
Mihailescu et al. The impact of user rationality in federated clouds
Adrian et al. Analysis of K-means algorithm for VM allocation in cloud computing
CN107947983B (en) Fair distribution method for maximizing resource utilization rate in cloud platform
US11709707B2 (en) Low latency distributed counters for quotas
Jayanthi Literature review: Dynamic resource allocation mechanism in cloud computing environment
Khazaei Performance modeling of cloud computing centers
US10007559B1 (en) Virtual tiering
CN112805684A (en) Resource allocation using recovery borrowing
CN113535408B (en) Auction type optimization method for edge side computing resources
Begam et al. Time-sensitive virtual machines provisioning and resource allocation in clouds
Huo et al. Hybrid storage throughput allocation among multiple clients in heterogeneous data center
Yan et al. A Game‐Theoretic Based Resource Allocation Strategy for Cloud Computing Services
Hu et al. Virtual resource pricing scheme in cloud platforms
Xavier et al. A dynamic admission control scheme to manage contention on shared computing resources

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant