CN116700966A - Pod resource allocation method of K8S cluster - Google Patents

Pod resource allocation method of K8S cluster Download PDF

Info

Publication number
CN116700966A
CN116700966A CN202310662313.2A CN202310662313A CN116700966A CN 116700966 A CN116700966 A CN 116700966A CN 202310662313 A CN202310662313 A CN 202310662313A CN 116700966 A CN116700966 A CN 116700966A
Authority
CN
China
Prior art keywords
value
pod
usage
data
cpu
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310662313.2A
Other languages
Chinese (zh)
Inventor
陈晓天
邓科
刘纯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Kege Iot Technology Co ltd
Original Assignee
Kege Iot Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kege Iot Technology Co ltd filed Critical Kege Iot Technology Co ltd
Priority to CN202310662313.2A priority Critical patent/CN116700966A/en
Publication of CN116700966A publication Critical patent/CN116700966A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061Partitioning or combining of resources
    • G06F9/5072Grid computing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5016Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The application discloses a pod resource allocation method of a K8S cluster, which belongs to the technical field of cloud computing, can greatly simplify the operation flow of the current common practice of users, can easily acquire data optimization suggestions for primary entrances, and can perform batch resource allocation modification of pod/containers without consuming a great deal of time to familiarize with K8S background commands, data acquisition, data cleaning and data analysis, and can provide analysis structures and suggestions without taking great care of editing and deploying yaml files for each pod/container.

Description

Pod resource allocation method of K8S cluster
Technical Field
The application relates to the technical field of cloud computing, in particular to a pod resource allocation method of a K8S cluster.
Background
In the use of k8s, an important operation that a user needs to do is to perform resource application allocation on the pod/container service that needs to be established and maintained in the daily work of the user, and perform resource analysis and redistribution after the pod/container service operates for a certain period;
the analysis and reassignment of resources can ensure the security of node/pod/container services, and meanwhile, the phenomenon that the pod/container services are not fully scheduled by a platform core resource scheduling mechanism due to insufficient resources of node applicable can be avoided as much as possible.
And analyzing whether the resource allocation is reasonable, safe, stable and efficient according to the pod/container resource, wherein the method involves a large amount of learning cost of the platform pod/container and also comprises the historical data reading and analysis workload of each pod. It may not be a major problem for a professional, but is not familiar with the native service platform node/pod/container resource application theory, and it takes time for an operator who is not professional in analyzing the visual data of the platform to learn how to grasp the data, read the data and analyze the data, so that a great deal of time is consumed and a lot of troublesome operation risks are not said.
Disclosure of Invention
The application provides a pod resource allocation method of a K8S cluster, which aims to relieve the trouble of users on K8S large-batch background operation, and can realize necessary resource management reallocation which is needed at any time according to project management requirements without being forgiving due to complex operation flow.
A pod resource allocation method of a K8S cluster comprises the following steps:
acquiring a pod CPU and storing a maximum value of the usage amount in a preset period;
acquiring a pod CPU and an average value of the use amount stored in a preset period;
taking a certain proportion of a request value in the current pod resource allocation value as a preset threshold value;
comparing the obtained maximum value and average value of the usage of the CPU and the memory with a preset threshold value;
if the maximum value and the average value of the usage are both larger than the preset threshold value, adding resource adjustment optimization;
if the maximum value of the usage amount and the average value of the usage amount are smaller than the preset threshold value, keeping unchanged;
if the maximum value of the usage amount and the average value of the usage amount are equal to the preset threshold value, the resource adjustment optimization is reduced.
The method for obtaining the maximum value of the usage amount in the preset period by the pod CPU is as follows:
and dividing the use amount in the preset period into a plurality of barrels on average, sequencing the data in each barrel, taking out the data according to the sequence of the barrels, and merging the data to obtain an ordered sequence of the use amount, thereby obtaining the pod CPU and the maximum use amount in the preset period.
The method for obtaining the average value of the usage amount in the preset period by the pod CPU is as follows:
based on the obtained ordered sequence of usage, all data are added and divided by the total number of data to obtain an average value of usage.
The preset threshold is 70%.
The added resource adjustment optimization specifically comprises the following steps: the preset threshold is increased, and the increase value is 10% -20% of the maximum value of the request.
The resource adjustment optimization is specifically as follows: the preset threshold value is reduced, the value range of the reduced value is 0- |n| and the value range of the reduced value is positive number of the difference value between the average value of the usage and the threshold value.
The preset period is 30 days.
A pod resource allocation apparatus of a K8S cluster, comprising:
the acquisition module is used for acquiring the resource utilization data of the pod/container in a preset period;
the calculation module is used for calculating based on the resource utilization data acquired by the acquisition module to obtain the maximum value and the average value of the usage of the CPU and the memory,
the judging module is used for comparing the maximum value and the average value of the use amount of the CPU and the memory with a preset threshold value to obtain a comparison result and giving a resource allocation suggestion;
and the allocation module is used for carrying out deployment based on the resource allocation proposal.
An electronic device, comprising:
at least one processor, and
a memory communicatively coupled to the at least one processor for storing executable instructions;
the processor is capable of executing executable instructions stored in the memory to implement a pod resource allocation method for the K8S cluster.
A computer readable storage medium having stored thereon a computer program which when executed by a processor implements a pod resource allocation method of a K8S cluster.
Compared with the prior art, the application has the beneficial effects that: the application can collect data of one pod, analyze and optimize deployment operation within 30 seconds, while the traditional method needs 10-30 minutes to be unequal, and can perform parallel batch analysis and optimization operation on a large number of pods in a cluster, thereby greatly saving efficiency and simultaneously ensuring the service stability and safety of nodes/pods/containers;
the method can greatly simplify the operation flow of the current common practice of the user, simultaneously can easily acquire data optimization suggestions for primary entrances, and can carry out batch resource allocation change of pod/container without consuming a great deal of time to familiarize k8s background commands, data acquisition, data cleaning and data analysis, and can provide analysis result suggestions without taking great care for editing and deploying yaml files by each pod/container.
Drawings
FIG. 1 is a graph of CPU utilization data for pod/container;
FIG. 2 is a Memory utilization data map of the pod/container;
FIG. 3 is a graph of CPU utilization peak marking data for pod/container;
FIG. 4 is a graph of Memory utilization peak tag data for pod/container;
fig. 5 is a flowchart of a resource allocation method according to the present application.
Reference numerals illustrate:
Detailed Description
One embodiment of the present application will be described in detail below with reference to the attached drawings, but it should be understood that the scope of the present application is not limited by the embodiment.
Example 1
As shown in fig. 1 to fig. 5, the method for allocating pod resources of a K8S cluster provided by the embodiment of the present application aims to alleviate the trouble of users on large-batch background operations of K8S, and is not required to be forgiving due to complex operation flows, so that necessary resource management reallocation can be performed at any time according to project management requirements, and the current general method is to obtain a period of time of each pod/container first, and arrange resource (CPU and Memory) utilization data for 30 days according to each project characteristic, as shown in fig. 1 and fig. 2;
the peak (maximum) value data of the resource utilization rate of pod/container in the manual recording data is a mark recording value, and fig. 3 and 4;
pod: redis-slave-redundant-0, CPU peak value reaches 0.153% within 30 days, and memory peak value reaches 0.033% within 30 days;
pod: redis-slave-master-0, the highest peak value of CPU reaches 0.101% within 30 days, and the highest peak value of memory reaches 0.034% within 30 days;
therefore, the existing mode requires operators to spend time learning how to grasp data, read the data, analyze the data and the like, which is very complicated;
the specific method of the application is as follows:
acquiring a pod CPU and storing a maximum value of the usage amount in a preset period;
acquiring a pod CPU and an average value of the use amount stored in a preset period;
the average value and the maximum value of the usage of the pod CPU and the Memory (Memory) in a preset period are as follows: the maximum usage amount of all pod CPUs and memories within 30 days (the preset period of the embodiment is 30 days) is identified by using a bucket ordering (bucket sort) algorithm, the core idea is that to-be-ordered data of the CPU memories are evenly distributed into some buckets, each bucket (namely, bucket) is ordered and collected, and finally, the data in each bucket are sequentially taken out in sequence, so that an ordered sequence which is arranged according to the size sequence is obtained.
Dividing the usage in 30 days into a certain number of barrels, for example, one barrel every 5 days, then sequencing the data in each barrel, and finally sequentially taking out the data from each barrel according to the sequence of the barrels for merging, so that an ordered sequence of the usage in 30 days can be obtained;
since bucket ordering does not require additional comparison operations, the time complexity is O (n 1 ) Wherein n is 1 Is the data volume to be discharged; bucket ordering is therefore a relatively low-time complexity ordering algorithm;
on the other hand, since additional barrel space needs to be created, the space complexity is relatively high, but when the data is uniformly distributed, the number of barrels is also small, and the occupied space is not too high;
obtaining an average value of 30 days of usage of the pod cpu memory, wherein the simplest algorithm is to directly add the 30 days of usage and then divide the added 30 days of usage; the temporal complexity of this algorithm is O (n 2 ) Wherein n is 2 Is the data volume, namely 30; in addition to the simple algorithms described above, more efficient algorithms may be used to calculate the average;
the most commonly used algorithm is the "cumulative average method", which is to add all the data and then divide the sum by the total number of data to obtain the average value. The accumulated average method can reduce the number of times of summation compared with a simple algorithm, thereby improving the calculation efficiency. By means of the cumulative average method, we can apply the method to the sum of the values of O (n 2 ) The average value calculation of the usage amount of 30 days is completed within the time complexity of (2); compared with a simple algorithm of directly adding and dividing by 30, the accumulated average method not only can improve the calculation efficiency, but also can avoid precision errors and obtain more accurate results;
taking a certain proportion of a request value in the current pod resource allocation value as a preset threshold value;
in this embodiment, 70% of the request value in the current pod resource allocation value is used as the threshold
Comparing the obtained maximum value and average value of the usage of the CPU and the memory with a preset threshold value; there are the following cases:
if the maximum value of the usage amount and the average value of the usage amount are both larger than a preset threshold value, adding resource adjustment optimization, wherein the optimization method is to increase the threshold value, and the range of the recommended added value is 10-20% of the maximum value;
if the maximum value of the usage amount and the average value of the usage amount are smaller than the preset threshold value, keeping unchanged;
if the maximum value of the usage amount and the average value of the usage amount are equal to the preset threshold value, reducing resource adjustment optimization, namely reducing the preset threshold value, suggesting that the value range of the reduced value is 0-n| and n is the difference value between the average value of the usage amount and the threshold value and n| is the absolute value of n.
In addition, there are two special cases, case one: the CPU or memory usage value is always within a range of 30 days below the threshold value, but within a short period of hours, the CPU or memory usage rate may suddenly increase and exceed the threshold value due to various reasons, in which case the usage value is: the maximum value is greater than the threshold and the average value is less than the threshold.
And a second case: within 30 days, the CPU or memory usage value is always above the threshold (depending on the specific application running on the pod, some memory-type database applications are characterized by extremely high memory usage), but within a short period of hours, the CPU or memory usage may suddenly drop due to various reasons, in which case the usage value is: the maximum value is larger than the threshold value, and the average value is larger than the threshold value;
the processing method of this embodiment is as follows: the threshold value can be independently increased according to the characteristic application, and the resource application value can not be adjusted;
the following are examples:
as shown in table 1:
wherein the peak value (maximum value) of the 24 requestcpus is only 0.036 cpu usage;
the peak value of 64Grequest memory is only 0.019G memory usage
Therefore, the current resource allocation value can be set according to the new application value and the new limit value of resource optimization according to project management planning;
in table 1, R and L are the newly proposed resource values, R represents the request value (application value), and L represents the limit value (limit value);
pod: the recommended resource values for Redis-slave-replicas-0 are: request:12 CPUs, limit:12 CPUs;
pod: the recommended resource value for Redis-slave-master-0 is the Request:12G memory, limit: a 12G memory;
the method can optimize and enable the whole node resource pool to acquire 0.1-63 CPUs and 0.1-63G memories again;
then, according to the background command line, the resource value in the yaml file of the pod/container is subjected to optimized numerical adjustment according to project requirements, and is stored and deployed;
combining yaml arrangement and one-piece deployment capability of a platform, combining suggested new resource values with one-key deployment through a page, and carrying out rapid deployment.
Example two
The embodiment provides a pod resource allocation device of a K8S cluster, including:
the acquisition module is used for acquiring the resource utilization data of the pod/container in a preset period;
the calculation module is used for calculating based on the resource utilization data acquired by the acquisition module to obtain the maximum value and the average value of the usage of the CPU and the memory,
the judging module is used for comparing the maximum value and the average value of the use amount of the CPU and the memory with a preset threshold value to obtain a comparison result and giving a resource allocation suggestion;
the allocation module is used for carrying out deployment based on the resource allocation suggestion;
the acquisition module sends the acquired data to the calculation module, the calculation module sends the calculation result to the judgment module, the judgment module sends the resource allocation suggestion to the allocation module, and the allocation module executes the resource allocation suggestion.
Example III
The present embodiment provides an electronic device capable of implementing the above embodiment, including:
at least one processor, and
a memory communicatively coupled to the at least one processor for storing executable instructions;
the processor is capable of executing executable instructions stored in the memory to implement a pod resource allocation method for the K8S cluster.
Example IV
The present embodiment provides a computer readable storage medium having a computer program stored thereon, which when executed by a processor implements the pod resource allocation method of the K8S cluster mentioned in the first embodiment.
It will be evident to those skilled in the art that the application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Furthermore, it should be understood that although the present disclosure describes embodiments, not every embodiment is provided with a separate embodiment, and that this description is provided for clarity only, and that the disclosure is not limited to the embodiments described in detail below, and that the embodiments described in the examples may be combined as appropriate to form other embodiments that will be apparent to those skilled in the art.

Claims (10)

1. A method for allocating pod resources of a K8S cluster, comprising:
acquiring a pod CPU and storing a maximum value of the usage amount in a preset period;
acquiring a pod CPU and an average value of the use amount stored in a preset period;
taking a certain proportion of a request value in the current pod resource allocation value as a preset threshold value;
comparing the obtained maximum value and average value of the usage of the CPU and the memory with a preset threshold value;
if the maximum value and the average value of the usage are both larger than the preset threshold value, adding resource adjustment optimization;
if the maximum value of the usage amount and the average value of the usage amount are smaller than the preset threshold value, keeping unchanged;
if the maximum value of the usage amount and the average value of the usage amount are equal to the preset threshold value, the resource adjustment optimization is reduced.
2. The method for allocating pod resources of a K8S cluster according to claim 1, wherein the method for obtaining the pod CPU and the maximum value of the usage in the preset period is as follows:
and dividing the use amount in the preset period into a plurality of barrels on average, sequencing the data in each barrel, taking out the data according to the sequence of the barrels, and merging the data to obtain an ordered sequence of the use amount, thereby obtaining the pod CPU and the maximum use amount in the preset period.
3. The method for allocating pod resources of a K8S cluster according to claim 2, wherein the method for obtaining the average value of the usage of pod CPU and memory within a preset period is as follows:
based on the obtained ordered sequence of usage, all data are added and divided by the total number of data to obtain an average value of usage.
4. The method for allocating pod resources of a K8S cluster of claim 1, wherein the preset threshold is 70%.
5. The method for allocating pod resources of a K8S cluster according to claim 1, wherein the adding resource adjustment optimization is specifically: the preset threshold is increased, and the increase value is 10% -20% of the maximum value of the request.
6. The method for allocating pod resources of a K8S cluster according to claim 1, wherein the reducing resource adjustment optimization is specifically: the preset threshold value is reduced, the value range of the reduced value is 0- |n| and the value range of the reduced value is positive number of the difference value between the average value of the usage and the threshold value.
7. The method for allocating pod resources of a K8S cluster of claim 1, wherein the predetermined period is 30 days.
8. A pod resource allocation apparatus of a K8S cluster, comprising:
the acquisition module is used for acquiring the resource utilization data of the pod/container in a preset period;
the calculation module is used for calculating based on the resource utilization data acquired by the acquisition module to obtain the maximum value and the average value of the usage of the CPU and the memory,
the judging module is used for comparing the maximum value and the average value of the use amount of the CPU and the memory with a preset threshold value to obtain a comparison result and giving a resource allocation suggestion;
and the allocation module is used for carrying out deployment based on the resource allocation proposal.
9. An electronic device, comprising:
at least one processor, and
a memory communicatively coupled to the at least one processor for storing executable instructions;
a processor capable of executing executable instructions stored in the memory to implement the pod resource allocation method of the K8S cluster of any of claims 1-7.
10. A computer readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the pod resource allocation method of a K8S cluster according to any of claims 1-7.
CN202310662313.2A 2023-06-06 2023-06-06 Pod resource allocation method of K8S cluster Pending CN116700966A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310662313.2A CN116700966A (en) 2023-06-06 2023-06-06 Pod resource allocation method of K8S cluster

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310662313.2A CN116700966A (en) 2023-06-06 2023-06-06 Pod resource allocation method of K8S cluster

Publications (1)

Publication Number Publication Date
CN116700966A true CN116700966A (en) 2023-09-05

Family

ID=87823311

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310662313.2A Pending CN116700966A (en) 2023-06-06 2023-06-06 Pod resource allocation method of K8S cluster

Country Status (1)

Country Link
CN (1) CN116700966A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117370032A (en) * 2023-12-07 2024-01-09 聊城金恒智慧城市运营有限公司 Cloud computing server resource optimization allocation method

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117370032A (en) * 2023-12-07 2024-01-09 聊城金恒智慧城市运营有限公司 Cloud computing server resource optimization allocation method
CN117370032B (en) * 2023-12-07 2024-03-01 聊城金恒智慧城市运营有限公司 Cloud computing server resource optimization allocation method

Similar Documents

Publication Publication Date Title
CN107291545A (en) The method for scheduling task and equipment of multi-user in computing cluster
EP4068090A1 (en) Container scheduling method and apparatus, and non-volatile computer-readable storage medium
CN105700948A (en) Method and device for scheduling calculation task in cluster
CN104407926B (en) A kind of dispatching method of cloud computing resources
CN110362392A (en) A kind of ETL method for scheduling task, system, equipment and storage medium
CN110347504B (en) Many-core computing resource scheduling method and device
US20120144008A1 (en) System and Method for Analyzing Computing System Resources
CN116700966A (en) Pod resource allocation method of K8S cluster
US7904689B1 (en) Just in time storage allocation analysis systems and methods
CN106919449A (en) The dispatch control method and electronic equipment of a kind of calculating task
WO2019196427A1 (en) Method and device for optimizing backup on the basis of supporting fault event constrained unit commitment
Hoseinyfarahabady et al. Handling uncertainty: Pareto-efficient bot scheduling on hybrid clouds
CN113010315A (en) Resource allocation method, resource allocation device and computer-readable storage medium
CN115220916B (en) Automatic calculation scheduling method, device and system of video intelligent analysis platform
US20230325235A1 (en) Training task queuing cause analysis method and system, device and medium
CN113568759B (en) Cloud computing-based big data processing method and system
CN109634714B (en) Intelligent scheduling method and device
CN103338246B (en) Virtual machine system of selection in a kind of infrastructure cloud resource allocation process and system
CN106708624B (en) Self-adaptive adjustment method for multi-working-domain computing resources
CN104794000B (en) A kind of job scheduling method and system
US20210004675A1 (en) Predictive apparatus and method for predicting workload group metrics of a workload management system of a database system
CN117075811A (en) Dynamic data storage method suitable for multiple tenants
CN114860449B (en) Data processing method, device, equipment and storage medium
CN116501468A (en) Batch job processing method and device and electronic equipment
CN114154962A (en) Batch processing monitoring method, device and equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination