CN116991589A - Kubernetes resource scheduling method and device and electronic equipment - Google Patents

Kubernetes resource scheduling method and device and electronic equipment Download PDF

Info

Publication number
CN116991589A
CN116991589A CN202311077744.9A CN202311077744A CN116991589A CN 116991589 A CN116991589 A CN 116991589A CN 202311077744 A CN202311077744 A CN 202311077744A CN 116991589 A CN116991589 A CN 116991589A
Authority
CN
China
Prior art keywords
target
resource scheduling
determining
pod
kubernetes
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311077744.9A
Other languages
Chinese (zh)
Inventor
任高锋
胡超超
欧阳熹
俞齐华
张敏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Shengbang Saiyun Technology Co ltd
Original Assignee
Beijing Shengbang Saiyun Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Shengbang Saiyun Technology Co ltd filed Critical Beijing Shengbang Saiyun Technology Co ltd
Priority to CN202311077744.9A priority Critical patent/CN116991589A/en
Publication of CN116991589A publication Critical patent/CN116991589A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083—Techniques for rebalancing the load in a distributed system

Abstract

The invention provides a Kubernetes resource scheduling method, a Kubernetes resource scheduling device and electronic equipment, and relates to the technical field of computers, wherein the method comprises the following steps: acquiring at least one to-be-scheduled pod in a Kubernetes cluster, at least one target node meeting a preset condition and target resource data corresponding to each target node; determining an objective function based on the objective resource data; determining a target resource scheduling mode based on the target function and the number comparison result of the at least one pod to be scheduled and the target node; and determining a target resource scheduling strategy of the Kubernetes cluster based on the target resource scheduling mode, wherein the target resource scheduling strategy is used for deploying the at least one pod to be scheduled to a corresponding target node. The invention can effectively reduce the resource loss of the control node and improve the scheduling efficiency and the application universality.

Description

Kubernetes resource scheduling method and device and electronic equipment
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a Kubernetes resource scheduling method, a Kubernetes resource scheduling device, and an electronic device.
Background
Kubernetes (k 8 s) is a container arrangement tool widely used at present, has the advantages of efficient deployment, rapid version iteration and the like, and is suitable for application scenes such as cloud primordia, micro services and the like. However, the research on cluster resource load balancing has the following defects: on one hand, most of the existing scheduling algorithms are static scheduling algorithms, and the dynamic property of cluster resources in actual use is not considered; on the other hand, the research on cluster load balancing only considers CPU and memory resources to optimize, but cannot give a complete resource portrait of the cluster, and the resource scheduling is relatively one-sided.
In the prior art, in order to realize resource load balancing during resource scheduling, network IO resources and disk IO resources are newly added on the basis of considering CPU and memory resources, and the intelligent optimization algorithm is combined to perform resource scheduling, so that the load balancing is achieved to a certain extent. However, for smaller-scale resource scheduling, such as single pod scheduling, continuous iteration of the intelligent optimization algorithm increases resource consumption of the control node (master node) and reduces resource scheduling efficiency.
Disclosure of Invention
The invention provides a method, a device and electronic equipment for scheduling Kubernetes resources, which are used for solving the defects of larger resource loss and lower resource scheduling efficiency caused by adopting an intelligent optimization algorithm for resource iteration when scheduling resources with smaller scale in the prior art, effectively reducing the resource loss of control nodes and improving scheduling efficiency and application universality.
The invention provides a Kubernetes resource scheduling method, which comprises the following steps:
acquiring at least one to-be-scheduled pod in a Kubernetes cluster, at least one target node meeting a preset condition and target resource data corresponding to each target node;
determining an objective function based on the objective resource data;
Determining a target resource scheduling mode based on the target function and the number comparison result of the at least one pod to be scheduled and the target node;
and determining a target resource scheduling strategy of the Kubernetes cluster based on the target resource scheduling mode, wherein the target resource scheduling strategy is used for deploying the at least one pod to be scheduled to a corresponding target node.
According to the Kubernetes resource scheduling method provided by the invention, the determining a target resource scheduling mode based on the target function and the number comparison result of the at least one pod to be scheduled and the target node comprises the following steps:
determining the target resource scheduling mode based on the objective function in the case that the number of the at least one pod to be scheduled is smaller than the number of the target nodes;
and determining the target resource scheduling mode based on an optimization algorithm and the objective function under the condition that the number of the at least one pod to be scheduled is greater than or equal to the number of the target nodes.
According to the Kubernetes resource scheduling method provided by the invention, the determining the target resource scheduling policy of the Kubernetes cluster based on the target resource scheduling mode comprises the following steps:
Determining an objective function value corresponding to each objective node based on the objective function and the objective resource data corresponding to each objective node in the objective resource scheduling mode when the number of the at least one pod to be scheduled is smaller than the number of the objective nodes; determining a target resource scheduling strategy of the Kubernetes cluster based on the reverse order sequencing results corresponding to all the target function values;
and under the condition that the number of the at least one pod to be scheduled is greater than or equal to the number of the target nodes, taking a minimized objective function as an optimization target, performing iterative optimization on all the target nodes in the target resource scheduling mode until the preset iterative times are reached, stopping iteration, and determining a target resource scheduling strategy of the Kubernetes cluster based on a final iterative result.
According to the Kubernetes resource scheduling method provided by the invention, the target resource data comprises target resource use data and target configuration data;
the determining an objective function based on the objective resource data comprises:
determining load data corresponding to the target node based on the target resource usage data;
Determining performance data corresponding to the target node based on the target configuration data;
the objective function is determined based on a ratio of the load data and the performance data.
According to the Kubernetes resource scheduling method provided by the invention, the target resource use data comprises CPU use amount, CPU total amount, memory use amount, memory total amount, network utilization rate and disk utilization rate;
the determining load data corresponding to the target node based on the target resource usage data includes:
acquiring CPU request quantity and memory request quantity corresponding to each pod to be scheduled;
determining a CPU usage rate based on the CPU usage amount, the CPU total amount, and the CPU request amount;
determining a memory usage rate based on the memory usage amount, the total memory amount, and the memory request amount;
and determining load data corresponding to the target node based on the CPU utilization rate, the memory utilization rate, the network utilization rate, the disk utilization rate and the first weights corresponding to the first weights, wherein the sum of the first weights is equal to 1.
According to the Kubernetes resource scheduling method provided by the invention, the target configuration data comprises CPU core number, CPU frequency, disk speed and network speed;
The determining, based on the target configuration data, performance data corresponding to the target node includes:
and determining performance data corresponding to the target node based on the CPU core number, the CPU frequency, the disk speed, the network speed and respective corresponding second weights, wherein the sum of the second weights is equal to 1.
According to the Kubernetes resource scheduling method provided by the invention, at least one target node meeting preset conditions in the Kubernetes cluster is obtained, and the method comprises the following steps:
acquiring all nodes in a Kubernetes cluster and resource data corresponding to all nodes respectively;
and determining the node with the memory usage amount smaller than or equal to a preset threshold value in the resource data as a target node.
The invention also provides a Kubernetes resource scheduling device, which comprises:
the acquisition module is used for acquiring at least one to-be-scheduled pod in the Kubernetes cluster, at least one target node meeting preset conditions and target resource data corresponding to each target node;
the first determining module is used for determining an objective function based on the objective resource data;
the second determining module is used for determining a target resource scheduling mode based on the target function and the number comparison result of the at least one pod to be scheduled and the target node;
And a third determining module, configured to determine a target resource scheduling policy of the Kubernetes cluster based on the target resource scheduling mode, where the target resource scheduling policy is used to deploy the at least one pod to be scheduled to a corresponding target node.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the Kubernetes resource scheduling method is realized by the processor when the program is executed.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a Kubernetes resource scheduling method as described in any of the above.
According to the method, the device and the electronic equipment for scheduling the Kubernetes resources, the target function is determined according to the target resource data corresponding to each target node by acquiring at least one to-be-scheduled pod in the Kubernetes cluster, at least one target node meeting the preset condition and the target resource data corresponding to each target node, the number of the to-be-scheduled pods is compared with the number of the target nodes meeting the preset condition, the target resource scheduling modes corresponding to different pod scales are determined according to the number comparison result and the target function, and the target resource scheduling strategy of the Kubernetes cluster is determined according to the target resource scheduling modes, so that the resource loss of the control node is effectively reduced, and the scheduling efficiency and the application universality are improved.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
Fig. 1 is one of flow diagrams of a Kubernetes resource scheduling method provided by an embodiment of the present invention;
fig. 2 is a second flow chart of a Kubernetes resource scheduling method according to an embodiment of the present invention;
fig. 3 is an exemplary flow schematic diagram of a Kubernetes resource scheduling method provided by an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a Kubernetes resource scheduling device according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Aiming at the problems of larger resource loss and lower resource scheduling efficiency caused by adopting an intelligent optimization algorithm to iterate resources when scheduling the resources with smaller scale in the prior art, the embodiment of the invention provides a Kubernetes resource scheduling method, and fig. 1 is one of flow diagrams of the Kubernetes resource scheduling method provided by the embodiment of the invention, as shown in fig. 1, the method comprises the following steps:
step 110, obtaining at least one to-be-scheduled pod in the Kubernetes cluster, at least one target node meeting a preset condition and target resource data corresponding to each target node.
It should be noted that Kubernetes cluster (K8 s) is an open-source container orchestration and management platform for automated deployment, extension, and manipulation of application containers. The K8s can run on various cloud platforms and can be deployed in a local data center, so that consistency of cross-environment management is ensured. A pod is the smallest deployable unit in K8s that can contain one or more tightly-associated containers. These containers share the same network namespace, IP address, and storage volume. After the pod combines the containers, resource sharing and intercommunication among the containers are realized. That is, pod provides a way to create an association between containers. The nodes can be physical devices or virtual devices in the K8s cluster and are used for running application programs corresponding to the containers. Each node has computing power, memory, storage resources, and the environment needed to run the container. A node may carry at least one pod, each of which may include at least one container therein. The target node may be a node meeting a preset condition in the K8s cluster. In the embodiment of the invention, when the resource scheduling is performed, the pod is scheduled to the corresponding target node, so that the resource data can be effectively allocated in the K8s cluster.
Alternatively, the Kubernetes cluster only supports YAML and Json formats to create resource objects, the Json format being used for the transfer of messages between interfaces, suitable for development. The YAML format is used for configuration and management and is suitable for cloud platform management. YAML is a non-markup language with high readability and easy understanding, which is used for expressing data serialization, and can simply express data forms such as arrays, hash tables, scalars and the like. The YAML file is used for storing a pod list, the pod list can be obtained by inquiring the YAML file, and then the names and the numbers corresponding to the pods to be scheduled are obtained through the pod list. The number of the to-be-scheduled pod is used for determining a target resource scheduling mode subsequently, and the name of the to-be-scheduled pod is used for accurately deploying the to-be-scheduled pod with the corresponding name to the corresponding target node after determining the target resource scheduling strategy.
Optionally, the target resource data includes target resource usage data and target configuration data. The target resource usage data includes CPU usage, CPU total, memory usage, memory total, network utilization and disk utilization. The target configuration data includes a CPU core number, a CPU frequency, a disk rate, and a network rate.
Alternatively, the target resource usage data may be obtained through a computel.sh script. Specifically, the total amount of the CPU can be obtained through an lscpu command; CPU usage can be obtained through a top command; the total memory and the memory usage can be obtained by the free-m combined with grep-i command; the disk utilization rate can be obtained through df-h commands; the network utilization rate can be obtained by obtaining the input and output flow of the network interface through ifstat, calculating the sum of the input flow and the output flow and dividing the sum by the network bandwidth.
In addition, the target configuration data may be acquired through a computers. Specifically, the CPU core number can be obtained through an nproc command; the content of the proc/CPU info file can be checked through the cat command, so that the CPU frequency is obtained; network port flow test can be carried out in a mode of generating a temporary speed measurement file to obtain network rate; the disk rate can be queried and parsed by the iostat command.
Further, obtaining at least one target node meeting a preset condition in the Kubernetes cluster includes:
acquiring all nodes in a Kubernetes cluster and resource data corresponding to all nodes respectively;
and determining the node with the memory usage amount smaller than or equal to a preset threshold value in the resource data as a target node.
Specifically, when resource scheduling is performed, all nodes in the K8s cluster are required to be acquired first, resource data corresponding to the nodes are further acquired, in order to ensure effective execution of application programs corresponding to containers in subsequent pops to be scheduled, each node needs to be screened according to preset conditions, when the memory usage of the nodes is greater than a preset threshold, the memory surplus may not meet the application programs corresponding to the containers in pops to be scheduled, so that the nodes with the memory usage greater than the preset threshold are removed, and the nodes with the memory usage less than or equal to the preset threshold are determined as target nodes, namely, the pops to be scheduled are configured to the corresponding target nodes. The predetermined threshold may be a predetermined proportion of the memory usage, which may be understood as the ratio of the memory usage to the total memory. For example, the preset ratio may be 80%, 85%, 90%, or the like, which is not limited in the embodiment of the present invention.
Optionally, when determining the target node from all nodes, a preset threshold may be set, and all nodes may be screened by using a k8s-resource-constraints. Specifically, the information of all nodes in the K8s cluster can be obtained through a kubecl command, nodes with the memory usage exceeding a preset threshold are filtered through a jq command, and the remaining nodes which are smaller than or equal to the preset threshold are determined as target nodes, namely, the nodes for bearing the to-be-scheduled pod.
And 120, determining an objective function based on the objective resource data.
Specifically, after the target resource is determined, an objective function can be further determined, and the objective function is used as a basis for scheduling so as to realize balanced scheduling.
Further, the determining an objective function based on the objective resource data includes:
determining load data corresponding to the target node based on the target resource usage data;
determining performance data corresponding to the target node based on the target configuration data;
the objective function is determined based on a ratio of the load data and the performance data.
Specifically, in order to ensure balanced scheduling in resource scheduling, importance of various resource data and different resource data in the cluster must be comprehensively considered. In the embodiment of the invention, load data corresponding to each target node is determined according to target resource usage data in target resource data, performance data corresponding to each target node is determined according to target configuration data in target resource data, and a target function is determined according to the ratio of the load data to the performance data, wherein the target function is shown in a formula (1), and the formula (1) is as follows:
Where N represents an objective function, L represents load data, and S represents performance data. The smaller the load data and the larger the performance data, the smaller the objective function value, and the larger the load data and the smaller the performance data, the larger the objective function value. Therefore, when the target resource scheduling policy is determined later, the objective function can be minimized as an optimization target.
Further, the determining load data corresponding to the target node based on the target resource usage data includes:
acquiring CPU request quantity and memory request quantity corresponding to each pod to be scheduled;
determining a CPU usage rate based on the CPU usage amount, the CPU total amount, and the CPU request amount;
determining a memory usage rate based on the memory usage amount, the total memory amount, and the memory request amount;
and determining load data corresponding to the target node based on the CPU utilization rate, the memory utilization rate, the network utilization rate, the disk utilization rate and the first weights corresponding to the first weights, wherein the sum of the first weights is equal to 1.
Specifically, when determining load data, considering resources requested by the pod to be scheduled, firstly acquiring a CPU request amount and a memory request amount of each pod to be scheduled, and determining a CPU utilization rate corresponding to each target node according to the CPU utilization amount, the CPU total amount and the CPU request amount by using the formula (2). Then, using the formula (3), determining the memory usage rate corresponding to each target node according to the memory usage amount, the total memory amount and the memory request amount. And then, carrying out weighted summation on the CPU utilization rate, the memory utilization rate, the network utilization rate and the disk utilization rate by utilizing the formula (4) to obtain load data corresponding to each target node. The CPU utilization rate, the memory utilization rate, the network utilization rate and the disk utilization rate respectively correspond to first weights to respectively represent the importance of corresponding resource data so as to realize real resource load balancing during resource scheduling. The formula (2) is:
Wherein c represents CPU utilization, c u Representing the CPU usage, c t Representing the total amount of CPU, c r Indicating the amount of CPU requests.
The formula (3) is:
wherein m represents the memory utilization, m u Represents the memory usage, m t Represents the total memory, m r Representing the amount of memory requests.
The formula (4) is:
L＝v1×c+v2×m+v3×n+v4×d
wherein n represents network utilization, d represents disk utilization, v 1 Representing a first weight, v, corresponding to CPU usage 2 Representing a first weight, v, corresponding to memory usage 3 Representing a first weight, v, corresponding to the network utilization 4 A first weight corresponding to the disk utilization is represented, and v 1 +v 2 +v 3 +v 4 ＝1。
Further, the determining, based on the target configuration data, performance data corresponding to the target node includes:
and determining performance data corresponding to the target node based on the CPU core number, the CPU frequency, the disk speed, the network speed and respective corresponding second weights, wherein the sum of the second weights is equal to 1.
Specifically, by using the formula (5), the CPU core number, the CPU frequency, the disk rate and the network rate are weighted and summed to obtain performance data corresponding to each target node. The second weights corresponding to the CPU core number, the CPU frequency, the disk speed and the network speed respectively represent the importance of the corresponding resource data so as to realize real resource load balancing during resource scheduling. The formula (5) is:
S＝w1×k+w2×h+w3×ns+w4×ds
Wherein k represents the number of CPU cores, h represents the CPU frequency, n s Representing network rate, d s Representing disk speed, w 1 Representing a second weight, w, corresponding to the CPU core number 2 Representing a second weight, w, corresponding to the CPU frequency 3 Representing a second weight, w, corresponding to the network rate 4 Represents a second weight corresponding to the disk speed, and w 1 +w 2 +w 3 +w 4 ＝1。
And 130, determining a target resource scheduling mode based on the comparison result of the number of the at least one pod to be scheduled and the target node and the target function.
Specifically, since the resource scheduling strategy is determined by iteration of the intelligent optimization algorithm in the prior art to be suitable for the scene with larger scale of the pod to be scheduled, in the embodiment of the invention, the quantity comparison result is determined by comparing the quantity of the pod to be scheduled with the quantity of the target nodes, and the corresponding target resource scheduling mode under different scales is determined according to the quantity comparison result, so that the target resource scheduling strategy is determined by the target resource scheduling mode, the resource loss and the time resource waste of the control node master caused by continuous iteration of the intelligent optimization algorithm when the scale of the pod to be scheduled is smaller are avoided, and the resource scheduling efficiency and the application universality are improved.
Further, fig. 2 is a second flow chart of a Kubernetes resource scheduling method according to the embodiment of the present invention, as shown in fig. 2, the determining a target resource scheduling mode based on the comparison result of the number of the at least one pod to be scheduled and the target node and the target function includes:
determining the target resource scheduling mode based on the objective function in the case that the number of the at least one pod to be scheduled is smaller than the number of the target nodes;
and determining the target resource scheduling mode based on an optimization algorithm and the objective function under the condition that the number of the at least one pod to be scheduled is greater than or equal to the number of the target nodes.
Specifically, after the number of the pod to be scheduled and the number of the target nodes are obtained, if the number of the pod to be scheduled is smaller than the number of the target nodes, the method indicates that the scale of the pod to be scheduled is smaller, and if the iteration of the intelligent optimization algorithm is adopted, a large amount of resources are wasted, so that when the scale of the pod to be scheduled is smaller, a target resource scheduling mode is determined according to a target function, that is, when a target resource scheduling strategy is determined according to the target resource scheduling mode, the target resource scheduling strategy can be obtained by directly calculating through the target function. If the number of the to-be-scheduled pod is greater than or equal to the number of the target nodes, the to-be-scheduled pod is indicated to have a larger scale, the target resource scheduling mode can be determined through an optimization algorithm and a target function, that is, when the target resource scheduling strategy is determined according to the target resource scheduling mode, the optimization algorithm can be adopted to minimize the target function to optimize the target, and the optimal target resource scheduling strategy can be obtained through continuous iteration.
Optionally, the optimization algorithm includes, but is not limited to: ant lion algorithm (Ant Lion Optimization, ALO), genetic algorithm (Genetic Algorithm, GA), particle swarm algorithm (Particle Swarm Optimization, PSO), or simulated annealing algorithm (Simulated annealing, SA), to which embodiments of the invention are not limited.
And 140, determining a target resource scheduling policy of the Kubernetes cluster based on the target resource scheduling mode, wherein the target resource scheduling policy is used for deploying the at least one pod to be scheduled to a corresponding target node.
Specifically, after determining the target resource scheduling mode, a target resource scheduling policy may be determined according to the target resource scheduling mode, that is, a scheduling policy for deploying each pod to be scheduled to a corresponding target node is determined, so as to implement resource load balancing.
Further, the determining, based on the target resource scheduling mode, a target resource scheduling policy of the Kubernetes cluster includes:
determining an objective function value corresponding to each objective node based on the objective function and the objective resource data corresponding to each objective node in the objective resource scheduling mode when the number of the at least one pod to be scheduled is smaller than the number of the objective nodes; determining a target resource scheduling strategy of the Kubernetes cluster based on the reverse order sequencing results corresponding to all the target function values;
And under the condition that the number of the at least one pod to be scheduled is greater than or equal to the number of the target nodes, taking a minimized objective function as an optimization target, performing iterative optimization on all the target nodes in the target resource scheduling mode until the preset iterative times are reached, stopping iteration, and determining a target resource scheduling strategy of the Kubernetes cluster based on a final iterative result.
Specifically, when the number of the pod to be scheduled is smaller than the number of the target nodes, the objective function value corresponding to each target node can be calculated according to the objective function and the objective resource data corresponding to each target node, wherein the objective function value is the ratio of the load data corresponding to the target node to the performance data, and the smaller the ratio is, the larger the resource amount of the target node for the pod to be scheduled is indicated. Therefore, after the objective function values corresponding to the objective nodes are determined, the objective function values are sorted in a reverse order, that is, the objective function values are sorted from small to large, and the objective nodes are sequentially allocated to the to-be-scheduled pod according to the reverse order sorting result. For example, taking the number of the to-be-scheduled pod as 2 as an example, after determining the reverse order sorting result of the objective function values corresponding to the objective nodes, the objective node with the smallest objective function value may be selected for the first time, that is, the first to-be-scheduled pod is allocated to the objective node with the smallest objective function value, and the objective node with the second to-be-scheduled pod may be selected for the second time, that is, the second to-be-scheduled pod is allocated to the objective node with the second to-be-scheduled.
In addition, when the number of the pod to be scheduled is greater than or equal to the number of the target nodes, an optimization algorithm can be adopted to take the minimized objective function as an optimization target, and an optimal solution, namely an optimal target resource scheduling strategy, is obtained through continuous iteration.
By way of example, taking an optimization algorithm as an ant lion algorithm, each ant lion in the ant lion algorithm can identify a pod to be scheduled, and the position of each ant lion can represent a feasible solution of the objective function, namely, the scheduling policy of the pod to be scheduled. After selecting one ant lion, the ant walks randomly in the trap range of the ant lion, and the ant lion eats the ant in the trap range, thus obtaining the optimal solution of the objective function, and the specific steps are as follows:
1) Initializing ant lions, taking each to-be-scheduled pod as an ant lion, and enabling the position of each ant lion to represent the scheduling strategy of the to-be-scheduled pod.
2) Each ant selects a target ant lion and walks randomly according to the position of the target ant lion.
3) And calculating the scaling of the trap range corresponding to the target ant lion, wherein the size of the trap range is irrelevant to the ant lion and is only relevant to the current iteration number, namely, the size of the trap range is continuously reduced along with the increase of the iteration number.
4) Ants walk by determining random numbers, and can determine random numbers within the range of [0,1], wherein the walk coefficient is-1 if the random number is smaller than or equal to 0.5, and the walk coefficient is 1 if the random number is larger than 0.5.
5) And determining the trap range of the target ant lion through the current iteration times and the scaling of the trap.
6) Calculating the accurate positions of the ants and the trap ranges, if the ants are not in the trap ranges, repeating the steps 2) -5), and if the ants are in the trap ranges, executing the step 7).
7) And the target ant lion moves to the position of the appointed ant, eats the ant and obtains the optimal solution, namely, the pod to be scheduled corresponding to the ant lion is deployed to the corresponding target node.
Taking an optimization algorithm as a genetic algorithm as an example, the specific steps are as follows:
1) And randomly generating an initialization population, wherein the initialization population comprises a plurality of initial individuals, and each initial individual is used for representing a scheduling strategy corresponding to each pod to be scheduled or an allocation strategy of each target node. Each initial individual corresponds to a scheduling policy. The association of the pod to be scheduled in each initial individual with the target node may be randomly generated or determined based on a priori knowledge.
2) And for each initial individual, calculating an objective function corresponding to each objective node in each initial individual, and determining the sum of the objective functions as an fitness value corresponding to each initial individual to evaluate the performance of each initial individual. The smaller the fitness value, the better the initial individual, i.e., the better the resource scheduling policy represented by the initial individual.
3) And according to the fitness value of each initial individual, using the fixed number or fixed proportion of initial individuals selected by the selection operator as parent individuals of the next round. The selection operator may employ roulette selection or tournament selection, etc., to ensure that the initial individual with the lower fitness is more likely to be selected.
4) And carrying out genetic operation on the selected initial individuals to generate new parent individuals in the next round. Genetic manipulation mainly includes crossover and mutation, wherein:
(1) Crossing: and selecting any two initial individuals, and generating a new parent individual by exchanging or combining the association relation between the pod to be scheduled and the target node in the two initial individuals. The cross operation may be single-point cross, multi-point cross, uniform cross, etc., which is not limited by the embodiment of the present invention.
(2) Variation: the diversity of parent individuals can be increased by changing the association relation between one or more to-be-scheduled pod in the initial individuals and the target node, a certain randomness can be introduced, and the situation that a local optimal solution cannot be trapped in the searching process is ensured.
5) Adding newly generated parent individuals into an initialized population, deleting some initial individuals according to the fitness value to obtain an iterative population, wherein the number of the iterative population is the same as that of the individuals in the initialized population so as to keep the scale of the population unchanged, and the individuals can comprise the initial individuals and the parent individuals.
6) And (3) iteratively executing the steps 2) to 5), continuously updating the iteration population, and ensuring that the iteration population of the current round is identical to the parent individuals of the iteration population of the previous round until the preset condition is met.
7) Outputting an optimal solution: and after the iteration is finished, selecting the minimum fitness value as an optimal solution, wherein a target individual corresponding to the minimum fitness value is a target resource scheduling strategy.
According to the Kubernetes resource scheduling method provided by the embodiment of the invention, the target function is determined according to the target resource data corresponding to each target node by acquiring at least one to-be-scheduled pod in the Kubernetes cluster, at least one target node meeting the preset condition and the target resource data corresponding to each target node, the number of the to-be-scheduled pods is compared with the number of the target nodes meeting the preset condition, the target resource scheduling modes corresponding to different pod scales are determined according to the number comparison result and the target function, and the target resource scheduling strategy of the Kubernetes cluster is determined according to the target resource scheduling modes, so that the resource loss of the control node is effectively reduced, and the scheduling efficiency and the application universality are improved.
The Kubernetes resource scheduling device provided by the invention is described below, and the Kubernetes resource scheduling device described below and the Kubernetes resource scheduling method described above can be correspondingly referred to each other.
The embodiment of the present invention further provides a Kubernetes resource scheduling device, and fig. 4 is a schematic structural diagram of the Kubernetes resource scheduling device provided by the embodiment of the present invention, as shown in fig. 4, where the Kubernetes resource scheduling device 400 includes: an acquisition module 410, a first determination module 420, a second determination module 430, and a third determination module 440, wherein:
an obtaining module 410, configured to obtain at least one pod to be scheduled in the Kubernetes cluster, at least one target node meeting a preset condition, and target resource data corresponding to each of the target nodes;
a first determining module 420, configured to determine an objective function based on the objective resource data;
a second determining module 430, configured to determine a target resource scheduling mode based on the target function and the result of comparing the number of the at least one pod to be scheduled with the target node;
a third determining module 440, configured to determine a target resource scheduling policy of the Kubernetes cluster based on the target resource scheduling mode, where the target resource scheduling policy is used to deploy the at least one pod to be scheduled to a corresponding target node.
According to the Kubernetes resource scheduling device provided by the embodiment of the invention, the target function is determined according to the target resource data corresponding to each target node by acquiring at least one to-be-scheduled pod in the Kubernetes cluster, at least one target node meeting the preset condition and the target resource data corresponding to each target node, the number of the to-be-scheduled pods is compared with the number of the target nodes meeting the preset condition, the target resource scheduling modes corresponding to different pod scales are determined according to the number comparison result and the target function, and the target resource scheduling strategy of the Kubernetes cluster is determined according to the target resource scheduling modes, so that the resource loss of the control node is effectively reduced, and the scheduling efficiency and the application universality are improved.
Optionally, the acquiring module 410 is specifically configured to:
acquiring all nodes in a Kubernetes cluster and resource data corresponding to all nodes respectively;
and determining the node with the memory usage amount smaller than or equal to a preset threshold value in the resource data as a target node.
Optionally, the target resource data includes target resource usage data and target configuration data.
Optionally, the first determining module 420 is specifically configured to:
Determining load data corresponding to the target node based on the target resource usage data;
determining performance data corresponding to the target node based on the target configuration data;
the objective function is determined based on a ratio of the load data and the performance data.
Optionally, the target resource usage data includes a CPU usage amount, a CPU total amount, a memory usage amount, a memory total amount, a network usage rate, and a disk usage rate.
Optionally, the first determining module 420 is specifically configured to:
acquiring CPU request quantity and memory request quantity corresponding to each pod to be scheduled;
determining a CPU usage rate based on the CPU usage amount, the CPU total amount, and the CPU request amount;
determining a memory usage rate based on the memory usage amount, the total memory amount, and the memory request amount;
and determining load data corresponding to the target node based on the CPU utilization rate, the memory utilization rate, the network utilization rate, the disk utilization rate and the first weights corresponding to the first weights, wherein the sum of the first weights is equal to 1.
Optionally, the target configuration data includes a CPU core number, a CPU frequency, a disk rate, and a network rate.
Optionally, the first determining module 420 is specifically configured to:
and determining performance data corresponding to the target node based on the CPU core number, the CPU frequency, the disk speed, the network speed and respective corresponding second weights, wherein the sum of the second weights is equal to 1.
Optionally, the second determining module 430 is specifically configured to:
determining the target resource scheduling mode based on the objective function in the case that the number of the at least one pod to be scheduled is smaller than the number of the target nodes;
and determining the target resource scheduling mode based on an optimization algorithm and the objective function under the condition that the number of the at least one pod to be scheduled is greater than or equal to the number of the target nodes.
Optionally, the third determining module 440 is specifically configured to:
determining an objective function value corresponding to each objective node based on the objective function and the objective resource data corresponding to each objective node in the objective resource scheduling mode when the number of the at least one pod to be scheduled is smaller than the number of the objective nodes; determining a target resource scheduling strategy of the Kubernetes cluster based on the reverse order sequencing results corresponding to all the target function values;
And under the condition that the number of the at least one pod to be scheduled is greater than or equal to the number of the target nodes, taking a minimized objective function as an optimization target, performing iterative optimization on all the target nodes in the target resource scheduling mode until the preset iterative times are reached, stopping iteration, and determining a target resource scheduling strategy of the Kubernetes cluster based on a final iterative result.
Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, as shown in fig. 5, the electronic device may include: processor 510, communication interface (Communications Interface) 520, memory 530, and communication bus 540, wherein processor 510, communication interface 520, memory 530 complete communication with each other through communication bus 540. Processor 510 may invoke logic instructions in memory 530 to perform a Kubernetes resource scheduling method comprising:
acquiring at least one to-be-scheduled pod in a Kubernetes cluster, at least one target node meeting a preset condition and target resource data corresponding to each target node;
determining an objective function based on the objective resource data;
determining a target resource scheduling mode based on the target function and the number comparison result of the at least one pod to be scheduled and the target node;
And determining a target resource scheduling strategy of the Kubernetes cluster based on the target resource scheduling mode, wherein the target resource scheduling strategy is used for deploying the at least one pod to be scheduled to a corresponding target node.
Further, the logic instructions in the memory 530 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, where the computer program product includes a computer program, where the computer program can be stored on a non-transitory computer readable storage medium, where the computer program when executed by a processor can perform a Kubernetes resource scheduling method provided by the foregoing methods, and the method includes:
Acquiring at least one to-be-scheduled pod in a Kubernetes cluster, at least one target node meeting a preset condition and target resource data corresponding to each target node;
determining an objective function based on the objective resource data;
determining a target resource scheduling mode based on the target function and the number comparison result of the at least one pod to be scheduled and the target node;
and determining a target resource scheduling strategy of the Kubernetes cluster based on the target resource scheduling mode, wherein the target resource scheduling strategy is used for deploying the at least one pod to be scheduled to a corresponding target node.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the Kubernetes resource scheduling method provided by the methods above, the method comprising:
acquiring at least one to-be-scheduled pod in a Kubernetes cluster, at least one target node meeting a preset condition and target resource data corresponding to each target node;
determining an objective function based on the objective resource data;
determining a target resource scheduling mode based on the target function and the number comparison result of the at least one pod to be scheduled and the target node;
And determining a target resource scheduling strategy of the Kubernetes cluster based on the target resource scheduling mode, wherein the target resource scheduling strategy is used for deploying the at least one pod to be scheduled to a corresponding target node.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. The Kubernetes resource scheduling method is characterized by comprising the following steps:
acquiring at least one to-be-scheduled pod in a Kubernetes cluster, at least one target node meeting a preset condition and target resource data corresponding to each target node;
determining an objective function based on the objective resource data;
determining a target resource scheduling mode based on the target function and the number comparison result of the at least one pod to be scheduled and the target node;
and determining a target resource scheduling strategy of the Kubernetes cluster based on the target resource scheduling mode, wherein the target resource scheduling strategy is used for deploying the at least one pod to be scheduled to a corresponding target node.
2. The Kubernetes resource scheduling method of claim 1, wherein the determining the target resource scheduling mode based on the target function and the result of comparing the number of the at least one pod to be scheduled with the target node comprises:
determining the target resource scheduling mode based on the objective function in the case that the number of the at least one pod to be scheduled is smaller than the number of the target nodes;
and determining the target resource scheduling mode based on an optimization algorithm and the objective function under the condition that the number of the at least one pod to be scheduled is greater than or equal to the number of the target nodes.
3. The Kubernetes resource scheduling method of claim 1 or 2, wherein the determining, based on the target resource scheduling pattern, a target resource scheduling policy of the Kubernetes cluster includes:
determining an objective function value corresponding to each objective node based on the objective function and the objective resource data corresponding to each objective node in the objective resource scheduling mode when the number of the at least one pod to be scheduled is smaller than the number of the objective nodes; determining a target resource scheduling strategy of the Kubernetes cluster based on the reverse order sequencing results corresponding to all the target function values;
And under the condition that the number of the at least one pod to be scheduled is greater than or equal to the number of the target nodes, taking a minimized objective function as an optimization target, performing iterative optimization on all the target nodes in the target resource scheduling mode until the preset iterative times are reached, stopping iteration, and determining a target resource scheduling strategy of the Kubernetes cluster based on a final iterative result.
4. A Kubernetes resource scheduling method according to any of claims 1 to 3, wherein the target resource data comprises target resource usage data and target configuration data;
the determining an objective function based on the objective resource data comprises:
determining load data corresponding to the target node based on the target resource usage data;
determining performance data corresponding to the target node based on the target configuration data;
the objective function is determined based on a ratio of the load data and the performance data.
5. The Kubernetes resource scheduling method of claim 4, wherein the target resource usage data comprises CPU usage, CPU total, memory usage, memory total, network utilization, and disk utilization;
The determining load data corresponding to the target node based on the target resource usage data includes:
acquiring CPU request quantity and memory request quantity corresponding to each pod to be scheduled;
determining a CPU usage rate based on the CPU usage amount, the CPU total amount, and the CPU request amount;
determining a memory usage rate based on the memory usage amount, the total memory amount, and the memory request amount;
and determining load data corresponding to the target node based on the CPU utilization rate, the memory utilization rate, the network utilization rate, the disk utilization rate and the first weights corresponding to the first weights, wherein the sum of the first weights is equal to 1.
6. The Kubernetes resource scheduling method of claim 4, wherein the target configuration data comprises a CPU core number, a CPU frequency, a disk rate, and a network rate;
the determining, based on the target configuration data, performance data corresponding to the target node includes:
and determining performance data corresponding to the target node based on the CPU core number, the CPU frequency, the disk speed, the network speed and respective corresponding second weights, wherein the sum of the second weights is equal to 1.
7. A Kubernetes resource scheduling method according to any of claims 1-3, wherein obtaining at least one target node in the Kubernetes cluster that meets a preset condition comprises:
acquiring all nodes in a Kubernetes cluster and resource data corresponding to all nodes respectively;
and determining the node with the memory usage amount smaller than or equal to a preset threshold value in the resource data as a target node.
8. A Kubernetes resource scheduling device, comprising:
the acquisition module is used for acquiring at least one to-be-scheduled pod in the Kubernetes cluster, at least one target node meeting preset conditions and target resource data corresponding to each target node;
the first determining module is used for determining an objective function based on the objective resource data;
the second determining module is used for determining a target resource scheduling mode based on the target function and the number comparison result of the at least one pod to be scheduled and the target node;
and a third determining module, configured to determine a target resource scheduling policy of the Kubernetes cluster based on the target resource scheduling mode, where the target resource scheduling policy is used to deploy the at least one pod to be scheduled to a corresponding target node.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the Kubernetes resource scheduling method of any of claims 1-7 when the program is executed by the processor.
10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the Kubernetes resource scheduling method of any of claims 1-7.
CN202311077744.9A 2023-08-24 2023-08-24 Kubernetes resource scheduling method and device and electronic equipment Pending CN116991589A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311077744.9A CN116991589A (en) 2023-08-24 2023-08-24 Kubernetes resource scheduling method and device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311077744.9A CN116991589A (en) 2023-08-24 2023-08-24 Kubernetes resource scheduling method and device and electronic equipment

Publications (1)

Publication Number Publication Date
CN116991589A true CN116991589A (en) 2023-11-03

Family

ID=88523212

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311077744.9A Pending CN116991589A (en) 2023-08-24 2023-08-24 Kubernetes resource scheduling method and device and electronic equipment

Country Status (1)

Country Link
CN (1) CN116991589A (en)

Similar Documents

Publication Publication Date Title
CN105704255B (en) A kind of server load balancing method based on genetic algorithm
Mechalikh et al. PureEdgeSim: A simulation toolkit for performance evaluation of cloud, fog, and pure edge computing environments
US20170329643A1 (en) Distributed node intra-group task scheduling method and system
Catanzaro et al. Diffusion-annihilation processes in complex networks
CN111371616B (en) Virtual network function chain deployment method and system for NUMA (non Uniform memory Access) architecture server
CN109656713B (en) Container scheduling method based on edge computing framework
CN107729514A (en) A kind of Replica placement node based on hadoop determines method and device
CN104461739B (en) A kind of virtual machine batch dispositions method based on cloudsim platforms
CN115134371A (en) Scheduling method, system, equipment and medium containing edge network computing resources
CN117041330B (en) Edge micro-service fine granularity deployment method and system based on reinforcement learning
CN113821332B (en) Method, device, equipment and medium for optimizing efficiency of automatic machine learning system
CN116700920A (en) Cloud primary hybrid deployment cluster resource scheduling method and device
CN116627241A (en) Method, system, equipment and storage medium for optimizing energy consumption of server
CN115220916A (en) Automatic computing power scheduling method, device and system for video intelligent analysis platform
Shukla et al. FAT-ETO: Fuzzy-AHP-TOPSIS-Based efficient task offloading algorithm for scientific workflows in heterogeneous fog–cloud environment
CN117061365B (en) Node selection method, device, equipment and readable storage medium
CN113886092A (en) Computation graph execution method and device and related equipment
Jiang et al. Hierarchical deployment of deep neural networks based on fog computing inferred acceleration model
Chang et al. Balanced job assignment based on ant algorithm for computing grids
CN112882805A (en) Profit optimization scheduling method based on task resource constraint
CN109889573B (en) NGSA multi-target-based copy placement method in hybrid cloud
CN116991589A (en) Kubernetes resource scheduling method and device and electronic equipment
CN106874108A (en) Thin cloud is minimized in mobile cloud computing use number technology
Wu et al. A genetic-ant-colony hybrid algorithm for task scheduling in cloud system
CN113722112B (en) Service resource load balancing processing method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination