Disclosure of Invention
The invention aims to provide a policy generation method based on cloud computing and a cloud platform, which solve the defects in the prior art, realize Pod dynamic expansion and contraction capacity through resource indexes on the basis of not changing the original functions of Kubernetes, dynamically adjust the allocated resources of newly created Pod nodes based on the utilization rate of various resource indexes on the Pod nodes when expansion is needed, and preferentially release Pod with low comprehensive utilization rate of resources when contraction is needed, so that the comprehensive utilization rate of Pod clusters to various resources is continuously improved.
An embodiment of the present application provides a policy generation method based on cloud computing, which is applied to a Pod cluster, and the method includes:
a periodically running resource index calculation task is configured in the Pod cluster;
constructing a Pod cluster intelligent strategy generation model;
training the Pod cluster intelligent strategy generation model by using a preset training method according to the calculation result of the periodically operated resource index calculation task to obtain a trained Pod cluster intelligent strategy generation model;
inputting information of a project to be executed into the trained Pod cluster intelligent strategy generation model to obtain a strategy generation mode of the project to be executed, wherein the strategy generation mode comprises dynamically adjusting resource allocation of a newly-built Pod and/or releasing the Pod with low resource utilization rate in the Pod cluster.
Optionally, after the periodically running resource index calculation task is configured in the Pod cluster, the method further includes:
and acquiring the historical resource utilization rate of the currently running Pod and the resource utilization rate of the available nodes in the Pod cluster through the number of the currently running Pod nodes and the resource index calculation task of the periodically running Pod, wherein the resource utilization rate comprises one of the memory resource utilization rate and the CPU resource utilization rate or the comprehensive utilization rate based on the weight.
Optionally, the building the Pod cluster intelligent policy generation model includes:
obtaining a plurality of training models for generating the intelligent strategy of the Pod cluster by utilizing the historical resource utilization rate of the currently running Pod and the resource utilization rate of the available nodes in the Pod cluster;
calculating the maximum mean value difference between the training models to form a construction function;
and processing the training model according to the construction function to obtain a corresponding Pod cluster intelligent strategy generation model.
Optionally, training the Pod cluster intelligent policy generation model by using a preset training method according to the calculation result of the periodically running resource index calculation task to obtain a trained Pod cluster intelligent policy generation model, including:
Obtaining a residual function of a calculation result of the resource index calculation task based on the periodic operation, and inputting the residual function into the Pod cluster intelligent strategy generation model;
according to the output result of the training model, respectively calculating the historical resource utilization rate of the current running Pod and the loss of the resource utilization rate of the available nodes in the Pod cluster;
and calculating joint loss according to the historical resource utilization rate of the currently running Pod and the loss of the resource utilization rate of the available nodes in the Pod cluster, and optimizing parameters of a training model based on the joint loss until a trained Pod cluster intelligent strategy generation model is obtained.
Optionally, the inputting the information of the item to be executed into the trained Pod cluster intelligent policy generation model to obtain a policy generation mode of the item to be executed includes:
when the policy generation mode is to dynamically adjust the resource allocation of the newly-built Pod, the trained Pod cluster intelligent policy generation model operates in the following mode:
according to the HPA function, calculating the number of newly built Pods through the resource index s;
calculating the average utilization rate H of the resource index j of the currently running Pod in the Pod cluster
j Wherein the said
The n is the number of Pod nodes currently running, and the M
ij For the actual usage of the resource index j on the node i, R is
ij A first allocation amount of the resource index j on the node i;
judging the average utilization rate H of the resource index j j If the utilization rate of the resource index j in the Pod cluster is smaller than the lower limit of the utilization rate of the resource index j, when a Pod is newly built, the resource index k is correspondingly built according to the number of the newly built Pod, and the resource index j is distributed according to a preset principle.
Optionally, the resource index j is allocated according to a preset rule, including:
judgment C
j /(1-Redis
j ) If the resource index j is smaller than 1, if so, when the Pod node i is newly built, the resource index j is distributed according to a second distribution amount, wherein the second distribution amount meets the requirement of
Said->
For a second allocation of resource index j on node i, the Redis
j The redundancy is preset for the resource index j in the Pod node;
if not, the resource index j is calculated according to the first allocation amount R ij The allocation is performed.
Optionally, the inputting the information of the item to be executed into the trained Pod cluster intelligent policy generation model to obtain a policy generation mode of the item to be executed, and further includes:
when the policy generation mode is to release the Pod with low resource utilization rate in the Pod cluster, the trained Pod cluster intelligent policy generation model operates in the following mode:
Obtaining other than the resource index sAverage utilization H of the remaining resource indicators on node i
i Wherein the said
The m is the category number of Pod node resource indexes;
according to the average utilization rate H i And determining and releasing the Pod nodes with low resource utilization rate in the Pod cluster.
Optionally, the preset training method includes a neural network algorithm.
Yet another embodiment of the present application provides a policy generation cloud platform based on cloud computing, applied to a Pod cluster, where the cloud platform includes:
the configuration module is used for configuring periodically running resource index calculation tasks in the Pod cluster;
the building module is used for building a Pod cluster intelligent strategy generation model;
the training module is used for training the Pod cluster intelligent strategy generation model by using a preset training method according to the calculation result of the periodically operated resource index calculation task to obtain a trained Pod cluster intelligent strategy generation model;
the generation module is used for inputting the information of the item to be executed into the trained Pod cluster intelligent strategy generation model to obtain a strategy generation mode of the item to be executed, wherein the strategy generation mode comprises dynamically adjusting the resource allocation of the newly-built Pod and/or releasing the Pod with low resource utilization rate in the Pod cluster.
Optionally, after the configuration module, the cloud platform further includes:
the acquisition module is used for acquiring the historical resource utilization rate of the currently running Pod and the resource utilization rate of the available nodes in the Pod cluster through the number of the currently running Pod nodes and the resource index calculation task of the periodically running Pod, wherein the resource utilization rate comprises one of the memory resource utilization rate and the CPU resource utilization rate or the comprehensive utilization rate based on the weight.
Optionally, the building module includes:
the first obtaining unit is used for obtaining a plurality of training models for generating the intelligent strategy of the Pod cluster by utilizing the historical resource utilization rate of the currently running Pod and the resource utilization rate of the available nodes in the Pod cluster;
the first calculation unit is used for calculating the maximum mean difference between the training models to form a construction function;
and the second obtaining unit is used for processing the training model according to the construction function to obtain a corresponding Pod cluster intelligent strategy generation model.
Optionally, the training module includes:
the obtaining unit is used for obtaining a residual function of a calculation result of the resource index calculation task running based on the period and inputting the residual function into the Pod cluster intelligent strategy generation model;
The second calculation unit is used for calculating the historical resource utilization rate of the current running Pod and the loss of the resource utilization rate of the available nodes in the Pod cluster respectively according to the output result of the training model;
and the optimization unit is used for calculating joint loss according to the historical resource utilization rate of the currently running Pod and the loss of the resource utilization rate of the available nodes in the Pod cluster, and optimizing parameters of a training model based on the joint loss until a trained Pod cluster intelligent strategy generation model is obtained.
Optionally, the generating module includes:
when the policy generation mode is to dynamically adjust the resource allocation of the newly-built Pod, the trained Pod cluster intelligent policy generation model operates in the following mode:
the third calculation unit is used for calculating the number of newly-built Pods according to the HPA function and through the resource index k;
a fourth calculation unit, configured to calculate an average utilization rate C of resource indexes j of currently running Pod in the Pod cluster
j Wherein the said
The n is the current operationNumber of Pod nodes, U
ij For the actual usage of the resource index j on the node i, R is
ij A first allocation amount of the resource index j on the node i;
A judging unit for judging the average utilization rate C of the resource index j j If the utilization rate of the resource index j in the Pod cluster is smaller than the lower limit of the utilization rate of the resource index j, when a Pod is newly built, the resource index k is correspondingly built according to the number of the newly built Pod, and the resource index j is distributed according to a preset principle.
Optionally, the judging unit includes:
a first judging subunit for judging C
j /(1-Red
j ) If the resource index j is smaller than 1, if so, when the Pod node i is newly built, the resource index j is distributed according to a second distribution amount, wherein the second distribution amount meets the requirement of
Said->
For the second allocation amount of the resource index j on the node i, the Red
j The redundancy is preset for the resource index j in the Pod node;
a second judging subunit, configured to, if not, determine the resource index j according to the first allocation amount R ij The allocation is performed.
Optionally, the generating module further includes:
when the policy generation mode is to release the Pod with low resource utilization rate in the Pod cluster, the trained Pod cluster intelligent policy generation model operates in the following mode:
an acquisition unit for acquiring average utilization C of the remaining resource indexes except the resource index k on the node i
i Wherein the said
The m is the category number of Pod node resource indexes;
a release unit for according to the average utilization C i And determining and releasing the Pod nodes with low resource utilization rate in the Pod cluster.
Yet another embodiment of the present application provides a policy generation cloud platform based on cloud computing, including a processor and a memory; the processor is communicatively coupled to the memory, and the processor is configured to read the computer program from the memory and execute the computer program to implement the method of any of the above.
A further embodiment of the present application provides a storage medium having a computer program stored therein, wherein the computer program is arranged to perform the method of any of the above when run.
Yet another embodiment of the present application provides an electronic device comprising a memory having a computer program stored therein and a processor configured to run the computer program to perform the method described in any of the above.
Compared with the prior art, the method comprises the steps of firstly configuring a periodically operated resource index calculation task in a Pod cluster, constructing a Pod cluster intelligent strategy generation model, training the Pod cluster intelligent strategy generation model by utilizing a preset training method according to the calculation result of the periodically operated resource index calculation task to obtain a trained Pod cluster intelligent strategy generation model, and finally inputting information of a project to be executed into the trained Pod cluster intelligent strategy generation model to obtain a strategy generation mode of the project to be executed, wherein the strategy generation mode comprises dynamically adjusting the resource allocation of a newly built Pod and/or releasing a Pod with low resource utilization rate in the Pod cluster, realizing Pod dynamic capacity expansion through the resource index on the basis of not changing the original function of Kubernetes, dynamically adjusting the allocated resource of a newly created Pod node on the basis of the utilization rate of various resource indexes on the Pod node when the capacity expansion is required, and releasing the Pod with low comprehensive utilization rate of the resource preferentially when the capacity expansion is required, so that the comprehensive utilization rate of the Pod cluster to various resources is continuously improved.
Detailed Description
The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the invention.
The embodiment of the invention firstly provides a policy generation method based on cloud computing, which can be applied to electronic equipment such as computer terminals, in particular to common computers, quantum computers and the like.
The following describes the operation of the computer terminal in detail by taking it as an example. Fig. 1 is a hardware structure block diagram of a computer terminal of a policy generation method based on cloud computing according to an embodiment of the present invention. As shown in fig. 1, the computer terminal may include one or more (only one is shown in fig. 1) processors 102 (the processor 102 may include, but is not limited to, a microprocessor MCU or a processing device such as a programmable logic device FPGA) and a memory 104 for storing data, and optionally, a transmission device 106 for communication functions and an input-output device 108. It will be appreciated by those skilled in the art that the configuration shown in fig. 1 is merely illustrative and is not intended to limit the configuration of the computer terminal described above. For example, the computer terminal may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.
The memory 104 may be used to store software programs and modules of application software, such as program instructions/modules corresponding to the cloud computing-based policy generation method in the embodiments of the present application, and the processor 102 executes the software programs and modules stored in the memory 104, thereby executing various functional applications and data processing, that is, implementing the method described above. Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory remotely located relative to the processor 102, which may be connected to the computer terminal via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission means 106 is arranged to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC) that can connect to other network devices through a base station to communicate with the internet. In one example, the transmission device 106 may be a radio frequency (RadioFrequency, RF) module for communicating wirelessly with the internet.
Referring to fig. 2, fig. 2 is a schematic flow chart of a policy generation method based on cloud computing according to an embodiment of the present invention, which may include the following steps:
s201: and periodically running resource index calculation tasks are configured in the Pod cluster.
Specifically, after the periodically running resource index calculation task is configured in the Pod cluster, the method further includes:
and acquiring the historical resource utilization rate of the currently running Pod and the resource utilization rate of the available nodes in the Pod cluster through the number of the currently running Pod nodes and the resource index calculation task of the periodically running Pod, wherein the resource utilization rate comprises one of the memory resource utilization rate and the CPU resource utilization rate or the comprehensive utilization rate based on the weight.
Kubernetes is a common container arrangement tool, which is focused by industry in advanced design concepts and is widely applied in practical production environments, one important task of Kubernetes is to select a suitable node (node) to run Pod (the smallest unit created and deployed in Kubernetes is an operation instance), the load of the whole cluster is determined by the resource utilization of each node in the cluster, and the utilization of each node is closely related to Pod running on the node. Thus, the policy generation manner for a cluster may be determined by the load state and resource utilization of the cluster.
By way of example, the historical resource utilization of the currently scheduled Pod and the resource utilization of the available nodes in the cluster may be obtained by deploying a preset cluster resource monitoring policy and periodically running resource index calculation tasks in the Pod cluster. The resources may specifically include three indexes: CPU utilization, memory utilization, and weight-based comprehensive utilization.
S202: and constructing a Pod cluster intelligent strategy generation model.
Specifically, constructing the Pod cluster intelligent policy generation model may include:
1. obtaining a plurality of training models for generating the intelligent strategy of the Pod cluster by utilizing the historical resource utilization rate of the currently running Pod and the resource utilization rate of the available nodes in the Pod cluster;
2. calculating the maximum mean value difference between the training models to form a construction function;
3. and processing the training model according to the construction function to obtain a corresponding Pod cluster intelligent strategy generation model.
In an alternative embodiment, the historical resource utilization rate of the currently running Pod and the resource utilization rate of the available nodes in the Pod cluster are obtained, and the first generation model and the second generation model for the intelligent strategy generation model of the Pod cluster can be obtained by training the historical resource utilization rate of the currently running Pod and the resource utilization rate of the available nodes in the Pod cluster. And carrying out normalization index processing on the characteristic information output by the characteristic layer of the first generation model to obtain first probability distribution, and carrying out normalization index processing on the characteristic information output by the characteristic layer of the second training model to obtain second probability distribution Y. The normalized index processing herein refers to converting the multi-class output into probability by an index function and a normalization method, and can be specifically expressed as: mapping the multi-classification result to zero to positive infinity by using an exponential function, then carrying out normalization processing to obtain approximate probability, and calculating the maximum mean value difference according to the first probability distribution X and the second probability distribution Y to form a construction function. And finally, processing the first training model according to the construction function, comparing the first generating model with the second generating model through processing, training the second generating model by utilizing probability distribution information obtained after comparison, the historical resource utilization rate of the current running Pod and the resource utilization rate information of the available nodes in the Pod cluster, and determining the learning degree of the second generating model to the first generating model after the dimension reduction processing by utilizing the construction function. And judging whether the constructed function is converged or not to obtain a corresponding Pod cluster intelligent strategy generation model.
S203: and training the Pod cluster intelligent strategy generation model by using a preset training method according to the calculation result of the periodically operated resource index calculation task to obtain a trained Pod cluster intelligent strategy generation model.
Specifically, training the Pod cluster intelligent strategy generation model by using a preset training method according to the calculation result of the periodically running resource index calculation task to obtain a trained Pod cluster intelligent strategy generation model, which may include:
step 1: and obtaining a residual function of a calculation result of the resource index calculation task based on the periodic operation, and inputting the residual function into the Pod cluster intelligent strategy generation model.
Step 2: and respectively calculating the historical resource utilization rate of the current running Pod and the loss of the resource utilization rate of the available nodes in the Pod cluster according to the output result of the training model.
Step 3: and calculating joint loss according to the historical resource utilization rate of the currently running Pod and the loss of the resource utilization rate of the available nodes in the Pod cluster, and optimizing parameters of a training model based on the joint loss until a trained Pod cluster intelligent strategy generation model is obtained.
Specifically, the Pod cluster intelligent strategy generation model can use a preset connection mode to make a reference function on the input of each layer, so that a residual function is formed by learning, the residual function is easier to optimize, and the layer number of the network can be greatly deepened. I.e. when the accuracy of the network model reaches saturation, an identity mapping layer can be added on the basis of the accuracy, i.e. the output is equal to the input, and the output of the previous layer is directly transmitted to the following network. Thus, the network depth is increased, and the error is not increased.
Illustratively, the calculation result of the resource index calculation task based on the periodic operation is written in one row as a single two-dimensional sample to generate two-dimensional data, and the two-dimensional data is normalized based on the maximum and minimum normalization method. For example, training or reasoning data of 180 x 2:60 x 2 is generated with a step length of 3, and the standardized data can improve the convergence speed of model training. The training data can be automatically retrained regularly by only setting fixed time to empty sample data, and for reasoning samples, only the latest 180 samples are kept, if Pod stops running because of certain anomalies, the data can also be collected, then the resource utilization rate data of Pod running, which is obtained in a certain time by finally generating the use condition of a CPU and a memory in the Pod running, is reserved in a file, and an initial Pod cluster intelligent strategy generation model is input for training, so that a final Pod cluster intelligent strategy generation model for predicting the Pod resource utilization rate is obtained.
The preset training method comprises a neural network algorithm.
S204: inputting information of a project to be executed into the trained Pod cluster intelligent strategy generation model to obtain a strategy generation mode of the project to be executed, wherein the strategy generation mode comprises dynamically adjusting resource allocation of a newly-built Pod and/or releasing the Pod with low resource utilization rate in the Pod cluster.
Specifically, the original HPA function of Kubernetes realizes Pod dynamic expansion and contraction through a single resource index, which may cause low resource utilization of other servers except for a single resource. The method and the device calculate the Pod expansion and contraction strategy based on the use condition of multiple types of resources of the Pod running in the Pod cluster, and provide a generation scheme of the Pod level automatic expansion and contraction strategy, and dynamically adjust the allocated resources of the newly created Pod node based on the utilization rate of the multiple types of resource indexes on the Pod node when the expansion is required on the basis of realizing the Pod dynamic expansion and contraction of the Pod through a single resource index without changing the original HPA function of the Kubernetes; when the capacity is required to be contracted, the pod with low comprehensive utilization rate of resources is released preferentially, so that the comprehensive utilization rate of the pod cluster to multiple types of resources is continuously improved.
Inputting the information of the item to be executed into the trained Pod cluster intelligent policy generation model to obtain a policy generation mode of the item to be executed, which may include:
When the policy generation mode is to dynamically adjust the resource allocation of the newly-built Pod, the trained Pod cluster intelligent policy generation model operates in the following mode:
according to the HPA function, calculating the number of newly built Pods through a resource index k;
calculating the average utilization rate H of the resource index j of the currently running Pod in the Pod cluster
j Wherein the said
The n is the number of Pod nodes currently running, and the M
ij For the actual usage of the resource index j on the node i, R is
ij A first allocation amount of the resource index j on the node i;
judging the average utilization rate H of the resource index j j If the utilization rate of the resource index j in the Pod cluster is smaller than the lower limit of the utilization rate of the resource index j, when a Pod is newly built, the resource index k is correspondingly built according to the number of the newly built Pod, and the resource index j is distributed according to a preset principle.
In an alternative embodiment, the resource index j is allocated according to a preset rule, which may include:
judgment C
j /(1-Redis
j ) If the resource index j is smaller than 1, if so, when the Pod node i is newly built, the resource index j is distributed according to a second distribution amount, wherein the second distribution amount meets the requirement of
Said->
For a second allocation of resource index j on node i, the Redis
j The redundancy is preset for the resource index j in the Pod node;
if not, the resource index j is calculated according to the first allocation amount R ij The allocation is performed.
Specifically, inputting the information of the item to be executed into the trained Pod cluster intelligent policy generation model to obtain a policy generation mode of the item to be executed, which may further include:
when the policy generation mode is to release the Pod with low resource utilization rate in the Pod cluster, the trained Pod cluster intelligent policy generation model operates in the following mode:
obtaining the average utilization rate H of the rest resource indexes except the resource index s on the node i
i Wherein the said
The m is the category number of Pod node resource indexes;
according to the average utilization rate H i And determining and releasing the Pod nodes with low resource utilization rate in the Pod cluster.
In another optional implementation mode, the method is optimized based on the original HPA function of the Kubernetes, and comprehensive utilization rate improvement of Pod clusters to multiple types of server resources can be achieved through cooperation of modules such as an automatic expander, a data collector, an executor and Pod. The automatic expander, the data collector and the executor module can be combined with or separated from modules such as KubernetesesHPA, metrics API, deployment and the like.
The application provides a cloud resource dynamic capacity expansion scheme based on a service delay priority strategy based on monitoring and analyzing a service delay of a customer experience index which is most important in mobile edge calculation, and the method comprises the following steps of: according to different characteristics of business on various resource demands, a Pod expansion and contraction strategy is calculated based on quantitative data of various resource use conditions of the Pod operated in the Pod cluster, so that comprehensive utilization rate of various resources of a server corresponding to the Pod cluster can be improved; on the basis of realizing Pod dynamic expansion and contraction through a single resource index without changing the original HPA function of the Kubernetes, when expansion is needed, the allocation resources of the newly created Pod node are dynamically adjusted based on the utilization rate of multiple resource indexes on the Pod node; when the capacity reduction is needed, the Pod with low comprehensive utilization rate of the resources is released preferentially, so that the comprehensive utilization rate of the Pod cluster on multiple types of resources is continuously improved.
The method comprises the steps of firstly configuring a periodically operated resource index calculation task in a Pod cluster, constructing a Pod cluster intelligent strategy generation model, training the Pod cluster intelligent strategy generation model by using a preset training method according to the calculation result of the periodically operated resource index calculation task to obtain a trained Pod cluster intelligent strategy generation model, finally inputting information of a project to be executed into the trained Pod cluster intelligent strategy generation model to obtain a strategy generation mode of the project to be executed, wherein the strategy generation mode comprises the steps of dynamically adjusting the resource allocation of a newly built Pod and/or releasing the Pod with low resource utilization rate in the Pod cluster, realizing Pod dynamic expansion and contraction through the resource index on the basis of not changing the original function of Kubernetes, dynamically adjusting the allocated resource of the newly built Pod node on the basis of the utilization rate of various resource indexes on the Pod node when the expansion is required, and releasing the Pod with low comprehensive utilization rate of the resource preferentially when the contraction is required, thereby continuously improving the comprehensive utilization rate of the Pod cluster on various resources.
Yet another embodiment of the present application provides a cloud computing-based policy generation cloud platform, as shown in fig. 3, which is a schematic structural diagram of the cloud computing-based policy generation cloud platform, where the cloud platform includes:
a configuration module 301, configured to configure a periodically running resource index calculation task in the Pod cluster;
a construction module 302, configured to construct a Pod cluster intelligent policy generation model;
the training module 303 is configured to train the Pod cluster intelligent policy generation model by using a preset training method according to a calculation result of the periodically running resource index calculation task, so as to obtain a trained Pod cluster intelligent policy generation model;
the generating module 304 is configured to input information of a to-be-executed item into the trained Pod cluster intelligent policy generating model to obtain a policy generating mode of the to-be-executed item, where the policy generating mode includes dynamically adjusting resource allocation of a newly-built Pod and/or releasing a Pod with low resource utilization rate in the Pod cluster.
Specifically, after the configuration module, the cloud platform further includes:
the acquisition module is used for acquiring the historical resource utilization rate of the currently running Pod and the resource utilization rate of the available nodes in the Pod cluster through the number of the currently running Pod nodes and the resource index calculation task of the periodically running Pod, wherein the resource utilization rate comprises one of the memory resource utilization rate and the CPU resource utilization rate or the comprehensive utilization rate based on the weight.
Specifically, the construction module includes:
the first obtaining unit is used for obtaining a plurality of training models for generating the intelligent strategy of the Pod cluster by utilizing the historical resource utilization rate of the currently running Pod and the resource utilization rate of the available nodes in the Pod cluster;
the first calculation unit is used for calculating the maximum mean difference between the training models to form a construction function;
and the second obtaining unit is used for processing the training model according to the construction function to obtain a corresponding Pod cluster intelligent strategy generation model.
Specifically, the training module includes:
the obtaining unit is used for obtaining a residual function of a calculation result of the resource index calculation task running based on the period and inputting the residual function into the Pod cluster intelligent strategy generation model;
the second calculation unit is used for calculating the historical resource utilization rate of the current running Pod and the loss of the resource utilization rate of the available nodes in the Pod cluster respectively according to the output result of the training model;
and the optimization unit is used for calculating joint loss according to the historical resource utilization rate of the currently running Pod and the loss of the resource utilization rate of the available nodes in the Pod cluster, and optimizing parameters of a training model based on the joint loss until a trained Pod cluster intelligent strategy generation model is obtained.
Specifically, the generating module includes:
when the policy generation mode is to dynamically adjust the resource allocation of the newly-built Pod, the trained Pod cluster intelligent policy generation model operates in the following mode:
the third calculation unit is used for calculating the number of newly-built Pods according to the HPA function and through the resource index k;
a fourth calculation unit, configured to calculate an average utilization rate H of resource indexes j of currently running Pod in the Pod cluster
j Wherein the said
The n is the number of Pod nodes currently running, and the M
ij For the actual usage of the resource index j on the node i, R is
ij A first allocation amount of the resource index j on the node i;
a judging unit for judging the average utilization rate H of the resource index j j If the utilization rate of the resource index j in the Pod cluster is smaller than the lower limit of the utilization rate of the resource index j, when a Pod is newly built, the resource index k is correspondingly built according to the number of the newly built Pod, and the resource index j is distributed according to a preset principle.
Specifically, the judging unit includes:
a first judging subunit for judging C
j /(1-Redis
j ) If the resource index j is smaller than 1, if so, when the Pod node i is newly built, the resource index j is distributed according to a second distribution amount, wherein the second distribution amount meets the requirement of
Said->
For a second allocation of resource index j on node i, the Redis
j The redundancy is preset for the resource index j in the Pod node;
a second judging subunit, configured to, if not, determine the resource index j according to the first allocation amount R ij The allocation is performed.
Specifically, the generating module further includes:
when the policy generation mode is to release the Pod with low resource utilization rate in the Pod cluster, the trained Pod cluster intelligent policy generation model operates in the following mode:
an acquisition unit for acquiring average utilization rate H of the remaining resource indexes on the node i except the resource index s
i Wherein the said
The m is the category number of Pod node resource indexes;
a release unit for according to the average utilization rate H i And determining and releasing the Pod nodes with low resource utilization rate in the Pod cluster.
Compared with the prior art, the method comprises the steps of firstly configuring a periodically operated resource index calculation task in a Pod cluster, constructing a Pod cluster intelligent strategy generation model, training the Pod cluster intelligent strategy generation model by utilizing a preset training method according to the calculation result of the periodically operated resource index calculation task to obtain a trained Pod cluster intelligent strategy generation model, and finally inputting information of a project to be executed into the trained Pod cluster intelligent strategy generation model to obtain a strategy generation mode of the project to be executed, wherein the strategy generation mode comprises dynamically adjusting the resource allocation of a newly built Pod and/or releasing a Pod with low resource utilization rate in the Pod cluster, realizing Pod dynamic capacity expansion through the resource index on the basis of not changing the original function of Kubernetes, dynamically adjusting the allocated resource of a newly created Pod node on the basis of the utilization rate of various resource indexes on the Pod node when the capacity expansion is required, and releasing the Pod with low comprehensive utilization rate of the resource preferentially when the capacity expansion is required, so that the comprehensive utilization rate of the Pod cluster to various resources is continuously improved.
Referring to fig. 4, fig. 4 is a schematic diagram of a cloud platform 401 for generating a policy based on cloud computing according to another embodiment of the present application, including a processor 402 and a memory 403; the processor 402 is communicatively connected to the memory 403, the processor 402 being configured to read a computer program from the memory 403 and execute the computer program to implement the method as described in any of the above.
Compared with the prior art, the method comprises the steps of firstly configuring a periodically operated resource index calculation task in a Pod cluster, constructing a Pod cluster intelligent strategy generation model, training the Pod cluster intelligent strategy generation model by utilizing a preset training method according to the calculation result of the periodically operated resource index calculation task to obtain a trained Pod cluster intelligent strategy generation model, and finally inputting information of a project to be executed into the trained Pod cluster intelligent strategy generation model to obtain a strategy generation mode of the project to be executed, wherein the strategy generation mode comprises dynamically adjusting the resource allocation of a newly built Pod and/or releasing a Pod with low resource utilization rate in the Pod cluster, realizing Pod dynamic capacity expansion through the resource index on the basis of not changing the original function of Kubernetes, dynamically adjusting the allocated resource of a newly created Pod node on the basis of the utilization rate of various resource indexes on the Pod node when the capacity expansion is required, and releasing the Pod with low comprehensive utilization rate of the resource preferentially when the capacity expansion is required, so that the comprehensive utilization rate of the Pod cluster to various resources is continuously improved.
The embodiment of the invention also provides a storage medium, in which a computer program is stored, wherein the computer program is configured to perform the steps of any of the method embodiments described above when run.
Specifically, in the present embodiment, the above-described storage medium may be configured to store a computer program for executing the steps of:
s201: a periodically running resource index calculation task is configured in the Pod cluster;
s202: constructing a Pod cluster intelligent strategy generation model;
s203: training the Pod cluster intelligent strategy generation model by using a preset training method according to the calculation result of the periodically operated resource index calculation task to obtain a trained Pod cluster intelligent strategy generation model;
s204: inputting information of a project to be executed into the trained Pod cluster intelligent strategy generation model to obtain a strategy generation mode of the project to be executed, wherein the strategy generation mode comprises dynamically adjusting resource allocation of a newly-built Pod and/or releasing the Pod with low resource utilization rate in the Pod cluster.
Specifically, in the present embodiment, the storage medium may include, but is not limited to: a usb disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing a computer program.
Compared with the prior art, the method comprises the steps of firstly configuring a periodically operated resource index calculation task in a Pod cluster, constructing a Pod cluster intelligent strategy generation model, training the Pod cluster intelligent strategy generation model by utilizing a preset training method according to the calculation result of the periodically operated resource index calculation task to obtain a trained Pod cluster intelligent strategy generation model, and finally inputting information of a project to be executed into the trained Pod cluster intelligent strategy generation model to obtain a strategy generation mode of the project to be executed, wherein the strategy generation mode comprises dynamically adjusting the resource allocation of a newly built Pod and/or releasing a Pod with low resource utilization rate in the Pod cluster, realizing Pod dynamic capacity expansion through the resource index on the basis of not changing the original function of Kubernetes, dynamically adjusting the allocated resource of a newly created Pod node on the basis of the utilization rate of various resource indexes on the Pod node when the capacity expansion is required, and releasing the Pod with low comprehensive utilization rate of the resource preferentially when the capacity expansion is required, so that the comprehensive utilization rate of the Pod cluster to various resources is continuously improved.
The present invention also provides an electronic device comprising a memory having a computer program stored therein and a processor arranged to run the computer program to perform the steps of any of the method embodiments described above.
Specifically, the electronic apparatus may further include a transmission device and an input/output device, where the transmission device is connected to the processor, and the input/output device is connected to the processor.
Specifically, in the present embodiment, the above-described processor may be configured to execute the following steps by a computer program:
s201: a periodically running resource index calculation task is configured in the Pod cluster;
s202: constructing a Pod cluster intelligent strategy generation model;
s203: training the Pod cluster intelligent strategy generation model by using a preset training method according to the calculation result of the periodically operated resource index calculation task to obtain a trained Pod cluster intelligent strategy generation model;
s204: inputting information of a project to be executed into the trained Pod cluster intelligent strategy generation model to obtain a strategy generation mode of the project to be executed, wherein the strategy generation mode comprises dynamically adjusting resource allocation of a newly-built Pod and/or releasing the Pod with low resource utilization rate in the Pod cluster.
Compared with the prior art, the method comprises the steps of firstly configuring a periodically operated resource index calculation task in a Pod cluster, constructing a Pod cluster intelligent strategy generation model, training the Pod cluster intelligent strategy generation model by utilizing a preset training method according to the calculation result of the periodically operated resource index calculation task to obtain a trained Pod cluster intelligent strategy generation model, and finally inputting information of a project to be executed into the trained Pod cluster intelligent strategy generation model to obtain a strategy generation mode of the project to be executed, wherein the strategy generation mode comprises dynamically adjusting the resource allocation of a newly built Pod and/or releasing a Pod with low resource utilization rate in the Pod cluster, realizing Pod dynamic capacity expansion through the resource index on the basis of not changing the original function of Kubernetes, dynamically adjusting the allocated resource of a newly created Pod node on the basis of the utilization rate of various resource indexes on the Pod node when the capacity expansion is required, and releasing the Pod with low comprehensive utilization rate of the resource preferentially when the capacity expansion is required, so that the comprehensive utilization rate of the Pod cluster to various resources is continuously improved.
It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present invention is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present invention. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required for the present invention.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.
In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, such as the above-described division of units, merely a logical function division, and there may be other manners of division in actual implementation, such as multiple units or components may be combined or integrated into another cloud platform, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, or may be in electrical or other forms.
The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units described above, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable memory. Based on such understanding, the technical solution of the present invention may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a memory, comprising several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the above-mentioned method of the various embodiments of the present invention. And the aforementioned memory includes: a U-disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The foregoing has outlined rather broadly the more detailed description of embodiments of the invention, wherein the principles and embodiments of the invention are explained in detail using specific examples, the above examples being provided solely to facilitate the understanding of the method and core concepts of the invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.