CN111367677A - Cluster resource allocation method and system - Google Patents
Cluster resource allocation method and system Download PDFInfo
- Publication number
- CN111367677A CN111367677A CN202010241226.6A CN202010241226A CN111367677A CN 111367677 A CN111367677 A CN 111367677A CN 202010241226 A CN202010241226 A CN 202010241226A CN 111367677 A CN111367677 A CN 111367677A
- Authority
- CN
- China
- Prior art keywords
- host
- container
- host machine
- cluster
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000013468 resource allocation Methods 0.000 title claims abstract description 40
- 239000006185 dispersion Substances 0.000 claims abstract description 105
- 238000012546 transfer Methods 0.000 claims abstract description 49
- 238000012545 processing Methods 0.000 claims description 42
- 238000004590 computer program Methods 0.000 claims description 16
- 238000003860 storage Methods 0.000 claims description 16
- 238000011084 recovery Methods 0.000 claims description 7
- 238000012423 maintenance Methods 0.000 abstract description 14
- 238000010586 diagram Methods 0.000 description 4
- 238000013467 fragmentation Methods 0.000 description 3
- 238000006062 fragmentation reaction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012163 sequencing technique Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/485—Task life-cycle, e.g. stopping, restarting, resuming execution
- G06F9/4856—Task life-cycle, e.g. stopping, restarting, resuming execution resumption being on a different machine, e.g. task migration, virtual machine migration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5021—Priority
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Hardware Redundancy (AREA)
Abstract
The invention provides a cluster resource allocation method and a cluster resource allocation system. The method comprises the following steps: determining the residual resources of the host machine according to the actual used resources of the host machine and the specification of the host machine which are obtained in advance; determining an optimal host machine according to the residual resources of the host machine and the specification of the container; deploying a container on the optimal host; determining host machine dispersion of a host machine and cluster dispersion of a cluster where the host machine is located according to a pre-obtained central processor weight ratio, actual use resources of the host machine and specifications of the host machine; when the cluster dispersion is smaller than a preset dispersion threshold value, determining a receiving container host machine and a transferring container host machine; transferring the containers in the transfer container host machine to the receiving container host machine, and recovering the transfer container host machine. The invention can make the cluster resources intensive, reduce the cost of operation and maintenance hardware and improve the utilization rate of the cluster.
Description
Technical Field
The present invention relates to the technical field of resource allocation, and in particular, to a cluster resource allocation method and system.
Background
Currently, cloud computing is widely applied in the internet and financial industry, with the gradual increase of cloud application, the cloud computing cluster scale is larger and larger, application resource clusters are numerous, and Central Processing Units (CPUs) and memory resources are huge, so that the distribution and use of cluster resources become important contents of attention. One of the main reasons for the low resource utilization rate is that cluster resources are unevenly distributed and the fragmentation of the cluster resources is severe, which increases unnecessary operation and maintenance hardware cost. How to improve the utilization waterline of the CPU and the memory of the cluster resources and improve the allocation rate and the utilization rate of the resources are key and difficult points of cloud computing resource management.
Disclosure of Invention
The embodiments of the present invention mainly aim to provide a method and a system for allocating cluster resources, so as to make the cluster resources intensive, reduce the cost of operation and maintenance hardware, and improve the utilization rate of the cluster.
In order to achieve the above object, an embodiment of the present invention provides a cluster resource allocation method, including:
determining the residual resources of the host machine according to the actual used resources of the host machine and the specification of the host machine which are obtained in advance;
determining an optimal host machine according to the residual resources of the host machine and the specification of the container;
deploying a container on the optimal host;
determining host machine dispersion of a host machine and cluster dispersion of a cluster where the host machine is located according to a pre-obtained central processor weight ratio, actual use resources of the host machine and specifications of the host machine;
when the cluster dispersion is smaller than a preset dispersion threshold value, determining a receiving container host machine and a transferring container host machine;
transferring the containers in the transfer container host machine to the receiving container host machine, and recovering the transfer container host machine.
An embodiment of the present invention further provides a cluster resource allocation system, including:
a residual resource determining unit, configured to determine a residual resource of the host according to a pre-obtained actually used resource of the host and a specification of the host;
the optimal host unit is used for determining the optimal host according to the residual resources of the host and the specification of the container;
the container deployment unit is used for deploying the container on the optimal host machine;
the dispersion determining unit is used for determining the host machine dispersion of the host machine and the cluster dispersion of the cluster where the host machine is located according to the pre-acquired central processor weight ratio, the actual used resource of the host machine and the specification of the host machine;
the host machine determining unit is used for determining the receiving container host machine and the transferring container host machine when the cluster dispersion is smaller than a preset dispersion threshold value;
and the host machine transfer and recovery unit is used for transferring the containers in the transfer container host machine to the receiving container host machine and recovering the transfer container host machine.
The embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the steps of the cluster resource allocation method when executing the computer program.
Embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the cluster resource allocation method.
The cluster resource allocation method and the system of the embodiment of the invention firstly determine the residual resources of the host machine, then determine the optimal host machine according to the residual resources of the host machine and the specification of the container, and deploy the container on the optimal host machine; then determining the host machine dispersion of the host machine and the cluster dispersion of the cluster where the host machine is located; when the cluster dispersion is smaller than a preset dispersion threshold value, the containers in the transfer container host machine are transferred to the receiving container host machine, and the transfer container host machine is recovered, so that cluster resources are concentrated, the operation and maintenance hardware cost is reduced, and the cluster utilization rate is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.
FIG. 1 is a flow chart of a cluster resource allocation method in an embodiment of the present invention;
FIG. 2 is a flow chart of determining similarity of a host and a container in an embodiment of the present invention;
FIG. 3 is a flow chart of a cluster resource allocation method in another embodiment of the present invention;
FIG. 4 is a flow chart of determining the number of remaining boot containers for a host in an embodiment of the present invention;
FIG. 5 is a block diagram of a cluster resource allocation system according to an embodiment of the present invention;
fig. 6 is a block diagram showing the structure of a computer device in the embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As will be appreciated by one skilled in the art, embodiments of the present invention may be embodied as a system, apparatus, device, method, or computer program product. Accordingly, the present disclosure may be embodied in the form of: entirely hardware, entirely software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.
In view of the serious fragmentation of cluster resources in the prior art and the increase of unnecessary operation and maintenance hardware cost, embodiments of the present invention provide a cluster resource allocation method, so as to centralize cluster resources, reduce the operation and maintenance hardware cost, and improve the cluster utilization rate. The present invention will be described in detail below with reference to the accompanying drawings.
Fig. 1 is a flowchart of a cluster resource allocation method in an embodiment of the present invention. As shown in fig. 1, the cluster resource allocation method includes:
s101: and determining the residual resources of the host machine according to the actual used resources of the host machine and the specification of the host machine which are obtained in advance.
The actual use resources of the host machine can be acquired through an open source monitoring component prometheus (prometheus) deployed on the host machine. The actual used resources of the host machine comprise central processing unit used resources and memory used resources. The host machine specification comprises a central processing unit specification and a memory specification. The residual resources of the host machine comprise the residual amount of the central processing unit and the residual amount of the internal memory.
For example, the cpu specification of the host is 8C (core), and the memory specification is 16GB (gigabytes); and if the used resource of the central processing unit of the host machine is 4.196C and the used resource of the memory is 9.6GB, the residual quantity of the central processing unit of the host machine is 3.804C and the residual quantity of the memory is 9.6 GB.
S102: and determining the optimal host according to the residual resources of the host and the specification of the container.
In one embodiment, before executing S102, the method further includes: and filtering the hosts which do not meet the container specification according to the residual resources of the hosts and the specification of the container.
The residual resources of the host machine comprise the residual quantity of the central processing unit and the residual quantity of the memory, and the specification of the container comprises the specification of the central processing unit of the container and the specification of the memory of the container.
For example, when the remaining amount of the cpu of the host is greater than the cpu specification of the container and the remaining amount of the memory of the host is greater than the memory specification of the container, the host is retained, otherwise the host does not meet the container specification, and the host is excluded.
In one embodiment, S102 includes: determining the similarity between the host and the container according to the residual resources of the host and the specification of the container; and taking the host with the similarity larger than a preset similarity threshold as an optimal host.
Wherein, the similarity threshold value can be 0.8, and the hosts with the similarity greater than 0.8 with the container are all the optimal hosts.
FIG. 2 is a flow chart of determining similarity of a host and a container in an embodiment of the present invention. As shown in fig. 2, determining the similarity between the host and the container includes:
s201: and creating a host machine vector according to the residual quantity of the central processing unit and the residual quantity of the memory of the host machine.
For example, the host vector is
x1Is the remaining amount of the CPU, y1The remaining amount of the memory.
S202: and creating a container vector according to the central processor specification and the memory specification of the container.
For example, the container vector is
x2Is the specification of the central processing unit, y2Is the memory specification.
S203: and taking the cosine value of the included angle between the vector of the host machine and the vector of the container as the similarity of the host machine and the container.
Wherein, the expression of the similarity is as follows:
s103: deploying the container on the optimal host.
S104: and determining the host machine dispersion of the host machine and the cluster dispersion of the cluster where the host machine is located according to the pre-obtained central processor weight ratio, the actual use resource of the host machine and the specification of the host machine.
The cluster dispersion is the quotient of the sum of the host dispersion of all the hosts of the cluster and the number of the host of the cluster.
Host dispersion can be determined by the following formula:
μ is the host dispersion, x 'is the CPU resource, y' is the memory resource, α is the CPU weight ratio, which can be 0.5.
S105: and when the cluster dispersion is smaller than a preset dispersion threshold value, determining the receiving container host machine and the transferring container host machine.
In one embodiment, the cluster spread is measured
Greater than or equal to a discrete threshold
And in time, the host machines of the idle resource pool are required to be added into the cluster, so that the redundancy rate of the cluster resources is ensured to be at a normal level, and the temporary elastic capacity expansion requirement of the application container is met.
In one embodiment, determining the receiving container host and the transferring container host comprises:
and taking the host with the host dispersion between the cluster dispersion and a preset dispersion threshold value as a receiving container host.
For example, when the host dispersion μ is larger than the cluster dispersion
And is less than a discrete threshold
Then, the host is used as the receiving container host. When the host machine dispersion mu of the host machine is greater than or equal to the dispersion threshold value
Time, it indicates that the host resources are centralized and no containers for transfer need to be received.
And taking the host machine with the host machine dispersion degree smaller than or equal to the cluster dispersion degree as the transfer container host machine.
For example, when the host dispersion μ is less than or equal to the cluster dispersion
Then, the host is used as the transfer container host.
S106: transferring the containers in the transfer container host machine to the receiving container host machine, and recovering the transfer container host machine.
In one embodiment, transferring a container in a transfer container host to a receiving container host comprises:
sequencing the similarity of the receiving container host machine and the containers in a descending order; and transferring the containers in the transfer container host machine to the corresponding receiving container host machines in sequence according to the sequence.
For example, if the similarity of the receiving container host J is the maximum value among all the receiving container hosts, and the similarity of the receiving container host K is the second largest value among all the receiving container hosts, the container in the transfer container host is transferred to the receiving container host J first, and when the host dispersion of the receiving container host J is greater than or equal to the dispersion threshold value
Then, the container in the transfer container host is transferred to the receiving container host K.
The containers in the transfer container host machine are transferred to the receiving container host machine according to the dispersion of the host machine, so that the container deployment condition can be adjusted in real time, the fragmentation of resources is effectively avoided, and the resource utilization rate is improved. The host of the transfer container is recovered to the free resource pool, and powerful basis is provided for elastic expansion and contraction of the cluster resources on the line.
The execution subject of the cluster resource allocation method shown in fig. 1 is a computer. As can be seen from the process shown in fig. 1, the cluster resource allocation method according to the embodiment of the present invention determines the remaining resources of the host, determines the optimal host according to the remaining resources of the host and the specification of the container, and deploys the container on the optimal host; then determining the host machine dispersion of the host machine and the cluster dispersion of the cluster where the host machine is located; when the cluster dispersion is smaller than a preset dispersion threshold value, the containers in the transfer container host machine are transferred to the receiving container host machine, and the transfer container host machine is recovered, so that cluster resources are concentrated, the operation and maintenance hardware cost is reduced, and the cluster utilization rate is improved.
The prior art cannot reflect the residual capacity condition of the cluster in real time and cannot meet the production operation and maintenance requirements. Fig. 3 is a flowchart of a cluster resource allocation method in another embodiment of the present invention. As shown in fig. 3, to solve the above technical problem, the cluster resource allocation method of the present invention further includes:
s301: and determining the number of the residual starting containers of the hosts and the number of the residual starting containers of the cluster where the hosts are located according to the residual resources of the hosts.
And the number of the residual starting containers of the cluster where the host is located is the sum of the number of the residual starting containers of all the hosts of the cluster.
FIG. 4 is a flow chart of determining the number of remaining boot containers for a host in an embodiment of the present invention. As shown in fig. 4, determining the remaining number of boot containers for the host includes:
s401: and determining the number of the containers of the central processing unit according to the residual quantity of the central processing unit of the host machine and the specification of the central processing unit of the container.
When the residual quantity of the central processing unit can be divided by the specification of the central processing unit of the container, the quotient of the residual quantity of the central processing unit and the specification of the central processing unit of the container is reduced by one to obtain the quantity of the central processing unit containers. When the residual quantity of the central processing unit cannot be divided by the specification of the central processing unit of the container, the quantity of the central processing unit containers can be obtained by rounding the quotient of the residual quantity of the central processing unit and the specification of the central processing unit of the container.
For example, if the cpu specification of the container is 2C, the remaining cpu amount is 3.804C, and the number of cpu containers is 1.
S402: and determining the quantity of the memory containers according to the memory surplus of the host and the memory specification of the containers.
When the memory surplus can be divided by the memory specification of the container, the quotient of the memory surplus and the memory specification of the container is reduced by one to obtain the number of the memory containers. And when the memory surplus cannot be divided by the memory specification of the container, rounding the quotient of the memory surplus and the memory specification of the container to obtain the number of the memory containers.
For example, if the memory specification of the container is 4GB and the remaining memory is 6.4GB, the number of memory containers is 1.
S403: and taking the minimum value of the number of the central processor containers and the number of the memory containers as the number of the residual starting containers.
For example, when the number of cpu containers is 1 and the number of memory containers is 2, the number of remaining boot containers is 1. Or when the number of the central processor containers is 1 and the number of the memory containers is 1, the number of the remaining starting containers is 1.
S302: and determining cluster redundancy according to the current container number of the cluster and the residual starting container number of the cluster.
Wherein the cluster redundancy may be determined by the following formula:
δ is the cluster redundancy, u is the remaining starting container number of the cluster, and v is the current container number of the cluster.
S303: and when the cluster redundancy is greater than the preset redundancy upper limit, recovering the host machines which do not deploy the containers.
Wherein the upper redundancy limit is 40%. And recovering the hosts without the deployed containers as capacity reduction operation, so that the resources of the cluster can be released.
S304: and when the cluster redundancy is less than the preset redundancy residual, adding the host machine into the cluster.
Wherein the lower redundancy limit is 30%. The host machine is added into the cluster for capacity expansion operation, so that the requirements of elastic expansion and capacity expansion of the current cluster application can be met.
In specific implementation, the current cluster includes 100 hosts of 8C4GB, and the number of the remaining boot containers of one host is 3 and the number of the remaining boot containers of the cluster is 300, calculated according to the container specification of 2C1GB standard. The current container number of the current cluster is 240, the remaining starting container number of the cluster is 60, the redundancy of the current cluster is 20%, and the redundancy is lower than the lower redundancy limit of 30%. If the lower limit requirement is required to be met, the host machine is required to be added into the cluster. If the number of host machines added to the cluster is n, the redundancy of the cluster after the host machines are added is as follows:
wherein, δ' is the cluster redundancy after adding the host, and w is the number of the remaining starting containers of the host. Then bringing specific values can be:
when the ratio of 30% < delta' < 40%, 14.28 < n < 33.33, therefore, adding 15-33 hosts to the cluster can meet the capacity expansion requirement of the current cluster application.
Similarly, if the number of the host recovery machines is m, the cluster redundancy after the host recovery machines is:
wherein, the delta is the cluster redundancy after the host machine is recovered, and is more than 30 percent and less than 40 percent.
The specific process of the embodiment of the invention is as follows:
1. and determining the residual resources of the host machine according to the actual used resources of the host machine and the specification of the host machine which are obtained in advance. The residual resources of the host machine comprise the residual quantity of the central processing unit and the residual quantity of the memory, and the specification of the container comprises the specification of the central processing unit of the container and the specification of the memory of the container.
2. And determining the number of the containers of the central processing unit according to the residual quantity of the central processing unit of the host machine and the specification of the central processing unit of the container.
3. And determining the quantity of the memory containers according to the memory surplus of the host and the memory specification of the containers.
4. And taking the minimum value of the number of the containers of the central processing unit and the number of the containers of the internal memory as the number of the remaining starting containers of the host machine.
5. And determining the number of the residual starting containers of the cluster where the host is located according to the number of the residual starting containers of the host.
6. And determining cluster redundancy according to the current container number of the cluster and the residual starting container number of the cluster.
7. And when the cluster redundancy is greater than the preset redundancy upper limit, recovering the host machines which do not deploy the containers, and when the cluster redundancy is less than the preset redundancy remaining limit, adding the host machines into the cluster.
8. And filtering the hosts which do not meet the container specification according to the residual resources of the hosts and the specification of the container.
9. Determining the similarity between the host and the container according to the residual resources of the host and the specification of the container; and taking the host with the similarity larger than a preset similarity threshold as an optimal host.
10. Deploying the container on the optimal host.
11. And determining the host machine dispersion of the host machine and the cluster dispersion of the cluster where the host machine is located according to the pre-obtained central processor weight ratio, the actual use resource of the host machine and the specification of the host machine.
12. When the cluster dispersion is smaller than a preset dispersion threshold value, a host machine with the host machine dispersion between the cluster dispersion and the preset dispersion threshold value is used as a receiving container host machine, and a host machine with the host machine dispersion smaller than or equal to the cluster dispersion is used as a transfer container host machine.
13. And sequencing the similarity of the receiving container host machine and the containers in a descending order.
14. And transferring the containers in the transfer container hosts to the corresponding receiving container hosts in sequence according to the sequence, and recovering the transfer container hosts.
To sum up, the cluster resource allocation method of the embodiment of the present invention determines the remaining resources of the host, determines the optimal host according to the remaining resources of the host and the specification of the container, and deploys the container on the optimal host; then determining the host machine dispersion of the host machine and the cluster dispersion of the cluster where the host machine is located; when the cluster dispersion is smaller than a preset dispersion threshold value, the containers in the transfer container host machine are transferred to the receiving container host machine, and the transfer container host machine is recovered, so that cluster resources are concentrated, the operation and maintenance hardware cost is reduced, and the cluster utilization rate is improved. The invention also recycles or adds host machines according to the cluster redundancy to dynamically and accurately reflect the residual capacity of resources and meet the production, operation and maintenance requirements.
Based on the same inventive concept, the embodiment of the present invention further provides a cluster resource allocation system, and because the principle of solving the problem of the system is similar to the cluster resource allocation method, the implementation of the system can refer to the implementation of the method, and repeated details are not repeated.
Fig. 5 is a block diagram of a cluster resource allocation system in an embodiment of the present invention. As shown in fig. 5, the cluster resource allocation system includes:
a residual resource determining unit, configured to determine a residual resource of the host according to a pre-obtained actually used resource of the host and a specification of the host;
the optimal host unit is used for determining the optimal host according to the residual resources of the host and the specification of the container;
the container deployment unit is used for deploying the container on the optimal host machine;
the dispersion determining unit is used for determining the host machine dispersion of the host machine and the cluster dispersion of the cluster where the host machine is located according to the pre-acquired central processor weight ratio, the actual used resource of the host machine and the specification of the host machine;
the host machine determining unit is used for determining the receiving container host machine and the transferring container host machine when the cluster dispersion is smaller than a preset dispersion threshold value;
and the host machine transfer and recovery unit is used for transferring the containers in the transfer container host machine to the receiving container host machine and recovering the transfer container host machine.
In one embodiment, the host determination unit is specifically configured to:
taking a host with the host dispersion between the cluster dispersion and a preset dispersion threshold value as a receiving container host;
and taking the host machine with the host machine dispersion degree smaller than or equal to the cluster dispersion degree as the transfer container host machine.
In one embodiment, the optimal host unit is specifically configured to:
determining the similarity between the host and the container according to the residual resources of the host and the specification of the container;
and taking the host with the similarity larger than a preset similarity threshold as an optimal host.
In one embodiment, the remaining resources include the remaining amount of the central processing unit and the remaining amount of the memory; the specification of the container comprises the specification of a central processing unit of the container and the specification of a memory of the container;
the optimal host unit is specifically configured to:
creating a host machine vector according to the residual quantity of a central processing unit and the residual quantity of a memory of the host machine;
creating a container vector according to the central processing unit specification and the memory specification of the container;
and taking the cosine value of the included angle between the vector of the host machine and the vector of the container as the similarity of the host machine and the container.
In one embodiment, the method further comprises the following steps:
the residual starting container determining unit is used for determining the residual starting container quantity of the host machine and the residual starting container quantity of the cluster where the host machine is located according to the residual resources of the host machine;
the cluster redundancy determining unit is used for determining cluster redundancy according to the current container number of the cluster and the residual starting container number of the cluster;
the host machine recovery unit is used for recovering the host machines which are not provided with containers when the cluster redundancy is greater than the preset redundancy upper limit;
and the host deployment unit is used for adding the host into the cluster when the cluster redundancy is less than the preset redundancy residual limit.
In one embodiment, the remaining starting container determining unit is specifically configured to:
determining the number of the containers of the central processing unit according to the residual amount of the central processing unit of the host machine and the specification of the central processing unit of the container;
determining the quantity of the memory containers according to the memory surplus of the host and the memory specification of the containers;
and taking the minimum value of the number of the central processor containers and the number of the memory containers as the number of the residual starting containers.
To sum up, the cluster resource allocation system of the embodiment of the present invention determines the remaining resources of the host first, then determines the optimal host according to the remaining resources of the host and the specification of the container, and deploys the container on the optimal host; then determining the host machine dispersion of the host machine and the cluster dispersion of the cluster where the host machine is located; when the cluster dispersion is smaller than a preset dispersion threshold value, the containers in the transfer container host machine are transferred to the receiving container host machine, and the transfer container host machine is recovered, so that cluster resources are concentrated, the operation and maintenance hardware cost is reduced, and the cluster utilization rate is improved. The invention also recycles or adds host machines according to the cluster redundancy to dynamically and accurately reflect the residual capacity of resources and meet the production, operation and maintenance requirements.
The embodiment of the present invention further provides a specific implementation manner of a computer device, which can implement all the steps in the cluster resource allocation method in the foregoing embodiment. Fig. 6 is a block diagram of a computer device in an embodiment of the present invention, and referring to fig. 6, the computer device specifically includes the following:
a processor (processor)601 and a memory (memory) 602.
The processor 601 is configured to call a computer program in the memory 602, and when the processor executes the computer program, the processor implements all the steps in the cluster resource allocation method in the foregoing embodiments, for example, when the processor executes the computer program, the processor implements the following steps:
determining the residual resources of the host machine according to the actual used resources of the host machine and the specification of the host machine which are obtained in advance;
determining an optimal host machine according to the residual resources of the host machine and the specification of the container;
deploying a container on the optimal host;
determining host machine dispersion of a host machine and cluster dispersion of a cluster where the host machine is located according to a pre-obtained central processor weight ratio, actual use resources of the host machine and specifications of the host machine;
when the cluster dispersion is smaller than a preset dispersion threshold value, determining a receiving container host machine and a transferring container host machine;
transferring the containers in the transfer container host machine to the receiving container host machine, and recovering the transfer container host machine.
To sum up, the computer device of the embodiment of the present invention determines the remaining resources of the host machine, determines the optimal host machine according to the remaining resources of the host machine and the specification of the container, and deploys the container on the optimal host machine; then determining the host machine dispersion of the host machine and the cluster dispersion of the cluster where the host machine is located; when the cluster dispersion is smaller than a preset dispersion threshold value, the containers in the transfer container host machine are transferred to the receiving container host machine, and the transfer container host machine is recovered, so that cluster resources are concentrated, the operation and maintenance hardware cost is reduced, and the cluster utilization rate is improved.
An embodiment of the present invention further provides a computer-readable storage medium capable of implementing all the steps in the cluster resource allocation method in the foregoing embodiment, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements all the steps in the cluster resource allocation method in the foregoing embodiment, for example, when the processor executes the computer program, the processor implements the following steps:
determining the residual resources of the host machine according to the actual used resources of the host machine and the specification of the host machine which are obtained in advance;
determining an optimal host machine according to the residual resources of the host machine and the specification of the container;
deploying a container on the optimal host;
determining host machine dispersion of a host machine and cluster dispersion of a cluster where the host machine is located according to a pre-obtained central processor weight ratio, actual use resources of the host machine and specifications of the host machine;
when the cluster dispersion is smaller than a preset dispersion threshold value, determining a receiving container host machine and a transferring container host machine;
transferring the containers in the transfer container host machine to the receiving container host machine, and recovering the transfer container host machine.
To sum up, the computer-readable storage medium of the embodiment of the present invention determines the remaining resources of the host first, then determines the optimal host according to the remaining resources of the host and the specification of the container, and deploys the container on the optimal host; then determining the host machine dispersion of the host machine and the cluster dispersion of the cluster where the host machine is located; when the cluster dispersion is smaller than a preset dispersion threshold value, the containers in the transfer container host machine are transferred to the receiving container host machine, and the transfer container host machine is recovered, so that cluster resources are concentrated, the operation and maintenance hardware cost is reduced, and the cluster utilization rate is improved.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Those of skill in the art will further appreciate that the various illustrative logical blocks, units, and steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate the interchangeability of hardware and software, various illustrative components, elements, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design requirements of the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present embodiments.
The various illustrative logical blocks, or elements, or devices described in connection with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a digital signal processor, an Application Specific Integrated Circuit (ASIC), a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other similar configuration.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. For example, a storage medium may be coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC, which may be located in a user terminal. In the alternative, the processor and the storage medium may reside in different components in a user terminal.
In one or more exemplary designs, the functions described above in connection with the embodiments of the invention may be implemented in hardware, software, firmware, or any combination of the three. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media that facilitate transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a general purpose or special purpose computer. For example, such computer-readable media can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store program code in the form of instructions or data structures and which can be read by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Additionally, any connection is properly termed a computer-readable medium, and, thus, is included if the software is transmitted from a website, server, or other remote source via a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wirelessly, e.g., infrared, radio, and microwave. Such discs (disk) and disks (disc) include compact disks, laser disks, optical disks, DVDs, floppy disks and blu-ray disks where disks usually reproduce data magnetically, while disks usually reproduce data optically with lasers. Combinations of the above may also be included in the computer-readable medium.
Claims (14)
1. A method for cluster resource allocation, comprising:
determining the residual resources of the host machine according to the actual used resources of the host machine and the specification of the host machine which are acquired in advance;
determining an optimal host machine according to the residual resources of the host machine and the specification of the container;
deploying the container on the optimal host;
determining host machine dispersion of the host machine and cluster dispersion of a cluster where the host machine is located according to a pre-obtained central processor weight ratio, actual use resources of the host machine and specifications of the host machine;
when the cluster dispersion is smaller than a preset dispersion threshold value, determining a receiving container host machine and a transferring container host machine;
and transferring the containers in the transfer container host machine to a receiving container host machine, and recovering the transfer container host machine.
2. The method of claim 1, wherein determining the receiving container host and the transferring container host comprises:
taking the host with the host dispersion between the cluster dispersion and a preset dispersion threshold as a receiving container host;
and taking the host with the host dispersion smaller than or equal to the cluster dispersion as a transfer container host.
3. The method of claim 1, wherein determining an optimal host according to the specifications of the container and the remaining resources of the host comprises:
determining the similarity between the host and the container according to the residual resources of the host and the specification of the container;
and taking the host with the similarity larger than a preset similarity threshold as an optimal host.
4. The method according to claim 3, wherein the remaining resources include remaining amount of central processing unit and remaining amount of memory; the specification of the container comprises the specification of a central processing unit of the container and the specification of a memory of the container;
determining the similarity of the host to the container comprises:
creating a host machine vector according to the residual amount of the central processing unit and the residual amount of the memory of the host machine;
creating a container vector according to the central processor specification and the memory specification of the container;
and taking the cosine value of the included angle between the vector of the host machine and the vector of the container as the similarity between the host machine and the container.
5. The method of claim 4, further comprising:
determining the number of the remaining starting containers of the host machine and the number of the remaining starting containers of the cluster where the host machine is located according to the remaining resources of the host machine;
determining cluster redundancy according to the current container number of the cluster and the residual starting container number of the cluster;
when the cluster redundancy is greater than a preset redundancy upper limit, recovering the host machines which do not deploy containers;
and when the cluster redundancy is less than the preset redundancy residual, adding the host machine into the cluster.
6. The method of claim 5, wherein determining the number of remaining boot containers for the host comprises:
determining the number of the central processor containers according to the residual amount of the central processors of the host machines and the specification of the central processors of the containers;
determining the quantity of the memory containers according to the memory surplus of the host machine and the memory specification of the containers;
and taking the minimum value of the number of the central processor containers and the number of the memory containers as the number of the residual starting containers.
7. A cluster resource allocation system, comprising:
a residual resource determining unit, configured to determine a residual resource of the host according to a pre-obtained actual used resource of the host and a specification of the host;
the optimal host unit is used for determining an optimal host according to the residual resources of the host and the specification of the container;
a container deployment unit for deploying the container on the optimal host;
the dispersion determining unit is used for determining the host machine dispersion of the host machine and the cluster dispersion of the cluster where the host machine is located according to the pre-acquired central processor weight ratio, the actual used resource of the host machine and the specification of the host machine;
a host determination unit, configured to determine a receiving container host and a transferring container host when the cluster dispersion is smaller than a preset dispersion threshold;
and the host machine transfer and recovery unit is used for transferring the containers in the transfer container host machine to the receiving container host machine and recovering the transfer container host machine.
8. The cluster resource allocation system of claim 7, wherein the host determination unit is specifically configured to:
taking the host with the host dispersion between the cluster dispersion and a preset dispersion threshold as a receiving container host;
and taking the host with the host dispersion smaller than or equal to the cluster dispersion as a transfer container host.
9. The cluster resource allocation system of claim 7, wherein the optimal host unit is specifically configured to:
determining the similarity between the host and the container according to the residual resources of the host and the specification of the container;
and taking the host with the similarity larger than a preset similarity threshold as an optimal host.
10. The cluster resource allocation system of claim 9, wherein the remaining resources comprise a remaining amount of central processing units and a remaining amount of memory; the specification of the container comprises the specification of a central processing unit of the container and the specification of a memory of the container;
the optimal host unit is specifically configured to:
creating a host machine vector according to the residual amount of the central processing unit and the residual amount of the memory of the host machine;
creating a container vector according to the central processor specification and the memory specification of the container;
and taking the cosine value of the included angle between the vector of the host machine and the vector of the container as the similarity between the host machine and the container.
11. The cluster resource allocation system of claim 10, further comprising:
a residual starting container determining unit, configured to determine, according to residual resources of the host, the residual starting container number of the host and the residual starting container number of the cluster where the host is located;
the cluster redundancy determining unit is used for determining cluster redundancy according to the current container number of the cluster and the residual starting container number of the cluster;
the host machine recovery unit is used for recovering the host machines which are not provided with containers when the cluster redundancy is greater than the preset redundancy upper limit;
and the host machine deployment unit is used for adding the host machine into the cluster when the redundancy of the cluster is less than the preset redundancy residual limit.
12. The cluster resource allocation system of claim 11, wherein the remaining start-up container determining unit is specifically configured to:
determining the number of the central processor containers according to the residual amount of the central processors of the host machines and the specification of the central processors of the containers;
determining the quantity of the memory containers according to the memory surplus of the host machine and the memory specification of the containers;
and taking the minimum value of the number of the central processor containers and the number of the memory containers as the number of the residual starting containers.
13. A computer arrangement comprising a memory, a processor and a computer program stored on the memory and running on the processor, characterized in that the processor implements the steps of the cluster resource allocation method according to any one of claims 1 to 6 when executing the computer program.
14. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the cluster resource allocation method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010241226.6A CN111367677A (en) | 2020-03-31 | 2020-03-31 | Cluster resource allocation method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010241226.6A CN111367677A (en) | 2020-03-31 | 2020-03-31 | Cluster resource allocation method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111367677A true CN111367677A (en) | 2020-07-03 |
Family
ID=71209278
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010241226.6A Pending CN111367677A (en) | 2020-03-31 | 2020-03-31 | Cluster resource allocation method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111367677A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112905350A (en) * | 2021-03-22 | 2021-06-04 | 北京市商汤科技开发有限公司 | Task scheduling method and device, electronic equipment and storage medium |
| CN114726734A (en) * | 2022-03-21 | 2022-07-08 | 平安壹账通云科技(深圳)有限公司 | Cloud service resource capacity expansion method and device, electronic equipment and storage medium |
| CN115086335A (en) * | 2022-07-27 | 2022-09-20 | 北京思和科创软件有限公司 | Container cloud node dynamic adding method and device, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102591443A (en) * | 2011-12-29 | 2012-07-18 | 华为技术有限公司 | Method, device and system for integrating virtual clusters |
| CN103677958A (en) * | 2013-12-13 | 2014-03-26 | 华为技术有限公司 | Virtualization cluster resource scheduling method and device |
| JP2015184965A (en) * | 2014-03-25 | 2015-10-22 | 日本電気株式会社 | Virtual machine management device, virtual machine management method, and virtual machine management system |
| CN105824686A (en) * | 2016-03-11 | 2016-08-03 | 中国联合网络通信集团有限公司 | Selecting method and selecting system of host machine of virtual machine |
| CN107819867A (en) * | 2017-11-18 | 2018-03-20 | 洛阳理工学院 | The load-balancing method and device of a kind of cluster network |
-
2020
- 2020-03-31 CN CN202010241226.6A patent/CN111367677A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102591443A (en) * | 2011-12-29 | 2012-07-18 | 华为技术有限公司 | Method, device and system for integrating virtual clusters |
| CN103677958A (en) * | 2013-12-13 | 2014-03-26 | 华为技术有限公司 | Virtualization cluster resource scheduling method and device |
| JP2015184965A (en) * | 2014-03-25 | 2015-10-22 | 日本電気株式会社 | Virtual machine management device, virtual machine management method, and virtual machine management system |
| CN105824686A (en) * | 2016-03-11 | 2016-08-03 | 中国联合网络通信集团有限公司 | Selecting method and selecting system of host machine of virtual machine |
| CN107819867A (en) * | 2017-11-18 | 2018-03-20 | 洛阳理工学院 | The load-balancing method and device of a kind of cluster network |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112905350A (en) * | 2021-03-22 | 2021-06-04 | 北京市商汤科技开发有限公司 | Task scheduling method and device, electronic equipment and storage medium |
| CN114726734A (en) * | 2022-03-21 | 2022-07-08 | 平安壹账通云科技(深圳)有限公司 | Cloud service resource capacity expansion method and device, electronic equipment and storage medium |
| CN115086335A (en) * | 2022-07-27 | 2022-09-20 | 北京思和科创软件有限公司 | Container cloud node dynamic adding method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111367677A (en) | Cluster resource allocation method and system | |
| EP3264275B1 (en) | Method, apparatus, and controller for managing storage array | |
| EP3036642B1 (en) | Hardware managed compressed cache | |
| CN102027453B (en) | System and method for optimizing interrupt processing in virtualized environments | |
| EP2557494A1 (en) | Storage apparatus and data copy method between thin-provisioning virtual volumes | |
| CN107533435B (en) | Storage space allocation method and storage equipment | |
| CN111309267B (en) | Storage space allocation method and device, storage equipment and storage medium | |
| CN110737924B (en) | Data protection method and equipment | |
| CN111475108A (en) | Distributed storage method, computer equipment and computer readable storage medium | |
| CN109521970B (en) | Data processing method and related equipment | |
| US8438338B2 (en) | Flexible use of extended cache using a partition cache footprint | |
| KR101198400B1 (en) | Memory management apparatus and method | |
| CN113126889A (en) | Method, apparatus and computer program product for managing storage space | |
| CN115185874B (en) | PCIE resource allocation method and related device | |
| WO2016029481A1 (en) | Method and device for isolating disk regions | |
| CN111367646A (en) | Container transfer method and system | |
| CN115480973A (en) | Server performance test method, device and medium | |
| US11307999B2 (en) | Data cache segregation for spectre mitigation | |
| CN111124275B (en) | Monitoring service optimization method and device of distributed block storage system | |
| CN115904795A (en) | Data storage method and device in storage system | |
| CN110007875B (en) | Method, device and equipment for migrating effective data | |
| CN111367678B (en) | Cluster resource management method and system | |
| CN108959517B (en) | File management method and device and electronic equipment | |
| US20170285956A1 (en) | Storage Space Management Method and Apparatus | |
| CN111338861B (en) | RSS-based network card performance test method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200703 |
|
| RJ01 | Rejection of invention patent application after publication |