CN115269182A - Resource adjusting method, device, server and storage medium - Google Patents

Resource adjusting method, device, server and storage medium Download PDF

Info

Publication number
CN115269182A
CN115269182A CN202210855905.1A CN202210855905A CN115269182A CN 115269182 A CN115269182 A CN 115269182A CN 202210855905 A CN202210855905 A CN 202210855905A CN 115269182 A CN115269182 A CN 115269182A
Authority
CN
China
Prior art keywords
time period
load
historical time
predicted
historical
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210855905.1A
Other languages
Chinese (zh)
Inventor
石泽涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianyi Cloud Technology Co Ltd
Original Assignee
Tianyi Cloud Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianyi Cloud Technology Co Ltd filed Critical Tianyi Cloud Technology Co Ltd
Priority to CN202210855905.1A priority Critical patent/CN115269182A/en
Publication of CN115269182A publication Critical patent/CN115269182A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/5019—Workload prediction

Abstract

The invention provides a resource adjusting method, a resource adjusting device, a server and a storage medium, and relates to the technical field of cloud computing, wherein the method comprises the following steps: predicting a first load for a predicted time period based on the target user workloads for the plurality of first historical time periods, and predicting a second load for the predicted time period based on the target user workloads for the plurality of second historical time periods; predicting the load of the prediction time period according to the first load and the second load; and classifying the load of the prediction time period to obtain the total resource amount of the prediction time period, and determining a target user resource adjustment value according to the total resource amount of the prediction time period. The embodiment of the invention can predict the load through the historical workloads of different dates and the historical workloads of the same date and different time periods, and then determine the target user resource adjustment value through the load, thereby not only improving the prediction accuracy rate, but also improving the working efficiency.

Description

Resource adjusting method, device, server and storage medium
Technical Field
The invention relates to the technical field of cloud computing, in particular to a resource adjusting method, a resource adjusting device, a server and a storage medium.
Background
With the advent of large data centers, hosting IT services, cloud computing is becoming one of the key technologies in the IT industry. The elastic cloud service means that the virtual cloud resources in the cloud platform can dynamically adjust the allocated cloud resources according to the change of the workload, so that the actually allocated resources are close to the actually required resources. It allows users to increase or decrease the capacity of virtualized resources as needed, i.e., scale resource adjustments so that only the resources they use need to be paid for. Traditional businesses deploy applications or services on a cloud platform in the form of unique Virtual Machines (VMs) and are provided to users through a virtualized network of the cloud platform provider. There are many providers offering such services and claim to offer an "unlimited" view of virtual resources, which refers to scalability.
At present, cluster resources are in a response mode when being expanded and contracted, for example, load conditions can be collected in real time, a user modifies resources according to the load conditions, for example, when the load is large, the number of virtual machines is increased, and when the load is small, the number of virtual machines is reduced. Therefore, manual intervention is more, and the working efficiency is lower.
Disclosure of Invention
The invention provides a resource adjustment method, a resource adjustment device, a server and a storage medium, wherein the load is predicted through historical workloads on different dates and historical workloads in different time periods on the same date, and then the target user resource adjustment value is determined through the load, so that the prediction accuracy rate is improved, and the working efficiency is improved.
In a first aspect, an embodiment of the present invention provides a resource adjustment method, including:
predicting a first load for the predicted time period based on a plurality of target user workloads for a first historical time period, and predicting a second load for the predicted time period based on a plurality of target user workloads for a second historical time period; the first historical time period is a time period which is the date before the date to which the predicted time period belongs and is the same as the predicted time period; the second historical time period is a time period before the prediction time period, and the date of the second historical time period is the same as the date of the prediction time period;
predicting a load for the predicted time period based on the first load and the second load;
and classifying the load of the prediction time period to obtain the total resource amount of the prediction time period, and determining the target user resource adjustment value according to the total resource amount of the prediction time period.
According to the method, the first load can be predicted through the target user workload of a plurality of dates before the prediction time period, the second load can be predicted through the target user workload of the time period before the prediction time period of the current date of the prediction time period, the load of the prediction time period is determined according to the first load and the second load, and meanwhile, the target user resource adjustment value is determined according to the predicted load, so that the accuracy of prediction is improved through the load after prediction of the historical workload of the transverse time and the longitudinal time, meanwhile, the resource adjustment value can be determined through classification of the load, no staff is needed to intervene, and the working efficiency is improved.
In one possible implementation, predicting the load for the predicted time period based on the first load and the second load comprises:
and taking the sum of the product of the first load and the weight corresponding to the first load and the product of the second load and the weight corresponding to the second load as the load of the prediction time period.
According to the method, the load of the prediction time period can be determined through the load and the corresponding weight, and the prediction accuracy is improved.
In one possible implementation, before predicting the load for the predicted time period based on the first load and the second load, the method further comprises:
predicting the load of each second historical time period according to the load of a plurality of historical time periods corresponding to each second historical time period; the historical time period corresponding to the second historical time period is a time period which is the date before the date to which the second historical time period belongs and is the same as the second historical time period;
determining an adjustment error and an adjustment direction according to the predicted load of each second historical time period and the actual load of each second historical time period;
if the adjusting direction is a positive direction, taking the sum of the adjusting error and the first load as a new first load;
and if the adjusting direction is a negative direction, taking the difference between the adjusting error and the first load as a new first load.
The method can predict the date before the date of the predicted time period and the workload of the time period before the predicted time period, and determine the adjustment error and the adjustment direction from the workload, so that the currently predicted first load is adjusted, and the accuracy of predicting the first load is improved.
In a possible implementation manner, determining the adjustment error according to the predicted load of each second historical time period and the actual load of each second historical time period includes:
taking the average value of the corresponding errors of each second historical time period between the predicted load of each second historical time period and the actual load of each second historical time period as an adjustment error; or
And predicting an adjustment error according to the error corresponding to each second historical time period between the predicted load of each second historical time period and the actual load of each second historical time period.
The method can reflect the trend of the error by taking the average value of the error as the adjustment error, or predict the adjustment error by a plurality of errors, thereby improving the accuracy of prediction.
In a possible implementation manner, determining the adjustment direction according to the predicted load of each second historical time period and the actual load of each second historical time period includes:
if the sum of the errors in the first error group is larger than the sum of the errors in the second error group, the adjusting direction is a positive direction; the error in the first error group is an error corresponding to a second historical time period, wherein the predicted load of the second historical time period is greater than the actual load of the second historical time period; the error in the second error group is an error corresponding to a second historical time period when the predicted load of the second historical time period is smaller than the actual load of the second historical time period;
and if the sum of the errors in the first error group is smaller than the sum of the errors in the second error group, the adjusting direction is a negative direction.
According to the method, the adjustment direction is determined according to the magnitude relation between the plurality of predicted loads and the actual load, and the prediction accuracy is improved.
In one possible implementation, predicting a first load for the predicted time period based on a target user workload for a plurality of first historical time periods comprises: predicting a first load of the prediction time period according to target user workloads of a plurality of first historical time periods through a GM (1,1) gray model;
predicting a second load for the predicted time period based on a target user workload for a plurality of second historical time periods, comprising: predicting a second load of the predicted time period according to target user workloads of a plurality of second historical time periods through a CNN convolutional neural network;
classifying the load of the prediction time period to obtain the total resource amount of the prediction time period, wherein the classifying comprises the following steps: and classifying the load of the prediction time period through a DT classifier to obtain the total resource amount of the prediction time period.
According to the method, the first load, the second load and the load classification can be predicted through the three models respectively, and the accuracy of calculation is improved.
In a second aspect, an embodiment of the present invention provides a resource adjusting apparatus, including:
a first prediction module for predicting a first load for a prediction time period based on target user workloads for a plurality of first historical time periods, and predicting a second load for the prediction time period based on target user workloads for a plurality of second historical time periods; the first historical time period is a time period which is the date before the date to which the predicted time period belongs and is the same as the predicted time period; the second historical time period is the time period before the predicted time period, and the date of the second historical time period is the same as the date of the predicted time period;
a second prediction module for predicting a load of the prediction time period according to the first load and the second load;
and the classification module is used for classifying the load of the prediction time period to obtain the class of the load of the prediction time period, and determining the target user resource adjustment value according to the class of the load of the prediction time period.
In a possible implementation manner, the second prediction module is configured to:
and taking the sum of the product of the first load and the weight corresponding to the first load and the product of the second load and the weight corresponding to the second load as the load of the prediction time period.
In a third aspect, an embodiment of the present invention provides a server, including: a processor and a memory;
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the resource adjustment method of any one of the first aspect.
In a fourth aspect, an embodiment of the present invention provides a storage medium, where instructions executed by a processor of a server enable the server to perform the resource adjustment method according to any one of the first aspect.
In addition, for technical effects brought by any one implementation manner of the second aspect to the fourth aspect, reference may be made to technical effects brought by different implementation manners of the first aspect, and details are not described here.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
Drawings
Fig. 1 is a flowchart of a resource adjustment method according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating a first load prediction process according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating a second load prediction process according to an embodiment of the present invention;
fig. 4 is a flowchart of another resource adjustment method according to an embodiment of the present invention;
fig. 5 is a flowchart of a resource adjustment method based on a neural network according to an embodiment of the present invention;
fig. 6 is a schematic diagram of a CNN convolutional neural network according to an embodiment of the present invention;
FIG. 7 is a diagram illustrating resource adjustment according to an embodiment of the present invention;
FIG. 8 is a diagram illustrating another resource adjustment according to an embodiment of the present invention;
FIG. 9 is a flowchart of one manner of modifying the first load according to an embodiment of the present invention;
FIG. 10 is a schematic diagram illustrating a comparison of responsive telescoping and active telescoping provided by an embodiment of the present invention;
fig. 11 is a structural diagram of a resource adjustment apparatus according to an embodiment of the present invention.
Detailed Description
The technical solution in the embodiments of the present application will be described in detail and removed with reference to the accompanying drawings. In the description of the embodiments of the present application, a "/" indicates an alternative meaning, for example, a/B may indicate a or B, and "plurality" means two or more than two in the description of the embodiments of the present application, unless otherwise specified.
An embodiment of the present invention provides a resource adjustment method, which is shown in fig. 1 and includes:
s100: predicting a first load of a predicted time period according to target user workloads of a plurality of first historical time periods;
the first historical time period is a time period which is the date before the date to which the predicted time period belongs and is the same as the predicted time period; for example, the prediction time period is 7 month 1, 11 am to 7 month 1, 12 am, the 6 month 30, 11 am to 6 month 30, 12 am, the 6 month 29, 11 am to 6 month 29, 12 am, the 6 month 28, 11 am to 6 month 28, 12 am, and the 6 month 27, 11 am to 6 month 27, 12 am are first history time periods.
As shown in FIG. 2, the time period of the historical data is X(0)、X(1)、X(2)、X(3)、…、X(m-2)、X(m-1)、X(m)I.e. history period 0-m. The historical days are 1-n, and the date of the prediction time period is n +1.
Prediction time period X(0)(n + 1) for a time period of X(0)Date n +1, a plurality of first history time periods X corresponding to the prediction time period(0)(1)、X(0)(2)、…、X(0)(n) workload, X can be predicted(0)A first load of (n + 1);
prediction time period X(1)(n + 1) for a time period of X(1)Date n +1, a plurality of first history time periods X corresponding to the prediction time period(1)(1)、X(1)(2)、…、X(1)(n) workload, X can be predicted(1)A first load of (n + 1);
prediction time period X(2)(n + 1) for a time period of X(2)Date n +1, a plurality of first history time periods X corresponding to the prediction time period(2)(1)、X(2)(2)、…、X(2)(n) workload, X can be predicted(2)A first load of (n + 1);
prediction time period X(3)(n + 1) for a time period of X(3)Date n +1, a plurality of first history time periods X corresponding to the prediction time period(3)(1)、X(3)(2)、…、X(3)(n) workload, X can be predicted(3)A first load of (n + 1);
…
prediction time period X(m-2)(n + 1) for a time period of X(m-2)Date n +1, a plurality of first history time periods X corresponding to the prediction time period(m-2)(1)、X(m-2)(2)、…、X(m-2)(n) workload, X can be predicted(m-2)A first load of (n + 1);
predicting time period X(m-1)(n + 1) for a time period of X(m-1)Date n +1, a plurality of first history time periods X corresponding to the prediction time period(m-1)(1)、X(m-1)(2)、…、X(m-1)(n) workload, X can be predicted(m-1)A first load of (n + 1);
prediction time period X(m)(n + 1) for a time period of X(m)Date n +1, a plurality of first history time periods X corresponding to the prediction time period(m)(1)、X(m)(2)、…、X(m)(n) workload, X can be predicted(m)A first load of (n + 1).
S110: predicting a second load of the predicted time period according to the target user workload of the plurality of second historical time periods;
and the second historical time period is the time period before the predicted time period, and the date of the second historical time period is the same as the date of the predicted time period. For example, the predicted time period is 11 am to 12 am on 1 st of 7 th, 10 am to 11 am on 1 st of 7 th, 9 am to 10 am on 1 st of 7 th, 9 am to 1 am on 1 st of 7 th, 9 am to 7 am on 1 st of 7 th, 8 am to 9 am on 1 st of 7 th, and 8 am from 7 am to 1 am on 1 st of 7 th are the second historical time period.
As shown in FIG. 3, if the predicted time period is X of Day (n + 1)(1)(n + 1), then the second history period may be X(0)(n + 1) according to X(0)(n + 1) workload, forecast X(1)A second load of (n + 1);
if the predicted time period is X of Day (n + 1)(2)(n + 1), then X may be the second historical period(0)(n + 1) and X(1)(n + 1) according to X(0)(n + 1) and X(1)(n + 1) workload, forecast X(2)A second load of (n + 1);
if the predicted time period is X of Day (n + 1)(3)(n + 1), then X may be the second historical period(0)(n+1)、X(1)(n+1)、X(2)(n + 1) according to X(0)(n+1)、X(1)(n+1)、X(2)(n + 1) three workloads, forecast X(3)A second load of (n + 1);
if the predicted time period is X of Day (n + 1)(m-1)(n + 1), then the second history period may be X(0)(n+1)、X(1)(n+1)、…、X(m-2)(n + 1) according to X(0)(n+1)、X(1)(n+1)、…、X(m-2)(n + 1) the m-1 workloads, preMeasure X(m-1)A second load of (n + 1);
if the predicted time period is X of Day (n + 1)(m)(n + 1), then X may be the second historical period(0)(n+1)、X(1)(n+1)、…、X(m-1)(n + 1) according to X(0)(n+1)、X(1)(n+1)、…、X(m-1)(n + 1) the m workloads, predict X(m)(n + 1) second load.
S120: predicting the load of the prediction time period according to the first load and the second load;
specifically, the sum of the first load and the second load is used as the load of the prediction time period.
Over time, the workload may also change, for which the first load and the second load may contribute differently to the last determined load. Based on this, the embodiment of the present invention further provides:
and taking the sum of the product of the first load and the weight corresponding to the first load and the product of the second load and the weight corresponding to the second load as the load of the prediction time period.
The weight corresponding to the first load and the weight corresponding to the second load may be preset and determined according to historical experience.
S130: and classifying the load of the prediction time period to obtain the total resource amount of the prediction time period, and determining a target user resource adjustment value according to the total resource amount of the prediction time period.
For example, the total amount of resources in the prediction time period is the number of virtual machines required for the prediction time period. The resource adjustment value is the increased number of the virtual machines, +2, namely two virtual machines are added on the original basis; +3, namely adding three virtual machines on the basis of the original; 2, i.e. reduce two virtual machines on a remaking basis, etc.
The load comprises concurrency, CPU utilization rate and memory usage; illustratively, the size of the concurrency amount, the size of the CPU utilization rate, and the size of the memory usage amount are predicted in the above manner, then the number of virtual machines required for the prediction time period is determined according to the size of the concurrency amount, the size of the CPU utilization rate, and the size of the memory usage amount, and the adjustment number of the virtual machines of the target user is determined according to the number of virtual machines required for the prediction time period.
Determining a target user resource adjustment value according to the total resource amount of the prediction time period comprises the following steps:
and comparing the total amount of the resources in the predicted time period with the total amount of the resources corresponding to the target second historical time period, and determining the resource adjustment value of the target user according to the number of the differences between the predicted time period and the target second historical time period. Wherein the target second historical time period is the first historical time period most recent to the predicted time period. For example: the predicted time period is 11 am at 7 month 1 to 12 am at 7 month 1, and then 11 am at 7 month 1 to 10 am at 7 month 1 is the target second history time period.
Illustratively, referring to fig. 4, the present invention provides another resource adjustment method, including:
s400: predicting a first concurrency amount of a prediction time period according to the concurrency amounts of the target users of the plurality of first historical time periods, and predicting a second concurrency amount of the prediction time period according to the concurrency amounts of the target users of the plurality of second historical time periods; predicting the concurrency amount of the prediction time period according to the first concurrency amount and the second concurrency amount;
s410: predicting a first CPU utilization rate of a prediction time period according to the CPU utilization rates of the target users of the plurality of first historical time periods, and predicting a second CPU utilization rate of the prediction time period according to the CPU utilization rates of the target users of the plurality of second historical time periods; predicting the CPU utilization rate of the prediction time period according to the first CPU utilization rate and the second CPU utilization rate;
s420: predicting a first memory usage amount of a prediction time period according to memory usage amounts of target users of a plurality of first historical time periods, and predicting a second memory usage amount of the prediction time period according to memory usage amounts of the target users of a plurality of second historical time periods; predicting the memory usage of the prediction time period according to the first memory usage and the second memory usage;
s430: and classifying the concurrency amount, the CPU utilization rate and the memory usage amount of the prediction time period to obtain the number of the virtual machines required by the prediction time period, and determining the adjustment value of the target user virtual machine according to the number of the virtual machines required by the prediction time period.
Wherein, the above steps can adopt a model to calculate, specifically:
predicting a first load for the predicted time period based on a target user workload for a plurality of first historical time periods, comprising: predicting a first load of the prediction time period according to target user workloads of a plurality of first historical time periods through a GM (1,1) gray model;
predicting a second load for the predicted time period based on a target user workload for a plurality of second historical time periods, comprising: predicting a second load of the predicted time period according to target user workloads of a plurality of second historical time periods through a CNN convolutional neural network;
classifying the load of the prediction time period to obtain the total resource amount of the prediction time period, wherein the classifying comprises the following steps: and classifying the load of the prediction time period through a DT classifier to obtain the total resource amount of the prediction time period.
Illustratively, with reference to fig. 5, an embodiment of the present invention provides a resource adjustment method based on a neural network, including:
s500: predicting a first load of a prediction time period according to target user workloads of a plurality of first historical time periods through a GM (1,1) gray model;
illustratively, predicting a first concurrency quantity of a prediction time period according to concurrency quantities of a plurality of first historical time periods through a GM (1,1) gray model; predicting a first CPU utilization rate of a prediction time period according to the CPU utilization rates of a plurality of first historical time periods through a GM (1,1) gray model; and predicting the first memory usage amount of the prediction time period according to the memory usage amounts of a plurality of first historical time periods through a GM (1,1) gray model.
S510: predicting a second load of the predicted time period according to the target user workload of the plurality of second historical time periods through the CNN convolutional neural network;
illustratively, predicting a second concurrency amount of the prediction time period according to the concurrency amounts of the plurality of second historical time periods through the CNN convolutional neural network; predicting a second CPU utilization rate of the predicted time period according to the CPU utilization rates of the plurality of second historical time periods through the CNN convolutional neural network; and predicting the second memory usage amount of the predicted time period according to the memory usage amounts of the plurality of second historical time periods through the CNN convolutional neural network.
S520: predicting the load of the prediction time period according to the first load and the second load;
s530: and classifying the load of the prediction time period through a DT classifier to obtain the total resource amount of the prediction time period, and determining a target user resource adjustment value according to the total resource amount of the prediction time period.
Illustratively, the coexistence amount, the CPU utilization rate, and the memory usage amount of the prediction time period are input into the DT classifier to obtain the class of the load of the prediction time period, and the target user resource adjustment value is determined according to the class of the load of the prediction time period.
The method comprises the steps of taking the coexistence amount of historical time periods, the CPU utilization rate and the memory usage amount as a training set, training a GM (1,1) gray model, taking the coexistence amount of historical time periods, the CPU utilization rate and the memory usage amount as the training set, training a CNN convolutional neural network, and obtaining the trained GM (1,1) gray model and the CNN convolutional neural network respectively, so that the GM (5363 zxft 3263) gray model and the CNN convolutional neural network can be used.
As shown in fig. 6, the CNN convolutional neural network includes 1 hidden layer, that is, a mechanism of 1+2, and the input layer inputs the concurrency, or CPU utilization, or memory usage of the previous m time periods, and outputs the prediction data, that is, the concurrency, or CPU utilization, or memory usage of the current m +1 time.
The CNN convolutional neural network has the formula:
f(x)＝G(b(2)+W(2)(s(b(1)+W(1)x)))
all the parameters of the CNN convolutional neural network are the connection weights and biases among the layers, including W1, b1, W2, b2. And training the formula by taking the coexistence amount of the historical time periods, the CPU utilization rate and the memory usage amount as a training set, thereby obtaining all parameters of the CNN convolutional neural network.
After training is completed, the GM-CNN model is combined through weighting. Specifically, the method comprises the following steps:
loadGM-CNN＝ω1loadGM+ω2loadCNN
the weight set is omega1、ω2，ω1Weight, ω, corresponding to the first load2And is the weight corresponding to the second load. Load to be output by GM (1,1)GMLoad output by CNNCNNAs a training set, then train the loadGM-CNNFormula to obtain omega1、ω2。
The DT classifier can obtain an optimal cluster resource elastic expansion allocation rule set according to the training of data in the historical log, elastic expansion adjustment is carried out through the selection of the classifier, and therefore when the concurrency, the CPU utilization rate and the memory usage are predicted and then input into the trained DT classifier, the total resource amount of the prediction time period is obtained, and therefore the adjustment value is obtained according to the difference between the total resource amount of the prediction time period and the total resource amount of the previous adjacent time period. Namely, the number of virtual machines is increased or decreased.
The DT classifier mainly completes the process of resource allocation in the cluster, the resource allocation means that various cloud resources existing in a distributed manner in the cloud are allocated in a specific cluster according to a fixed threshold setting principle so as to meet the requirements of different tenants on different resources, wherein the selection of a proper elastic expansion rule is a key step of dynamic allocation of the resources in the cloud, the problem of cluster system jitter is mainly solved, and the adjustment and manual intervention times are reduced. The elastic expansion method selection comprises two processes, firstly, historical data are subjected to standardized analysis processing, processed data form an elastic expansion rule set, and a DT classifier is trained by the processed data. Then, when the new CPU utilization rate, the memory usage amount and the current environment concurrency amount are transmitted to the trained DT classifier, the current required resource total amount is obtained, corresponding adjustment actions are obtained through comparison operation, the DT classifier model can be used for improving the elastic expansion efficiency, reducing the original adjustment times of elastic expansion and reducing the times of manual intervention, and classification is efficiently made.
For example, as shown in fig. 7, after it is predicted that 1 virtual machine needs to be added from 8 am to 9 am on 1 am of 7 month, after it is predicted that 1 virtual machine needs to be added from 9 am to 10 am on 1 am of 7 month, it is considered that the current number of virtual machines is insufficient, a request may be made to cloud resources, and 1 virtual machine is added, so that 2 virtual machines may be obtained from 9 am to 10 am on 1 month of 7 month, and when it is predicted that 1 am to 11 am on 1 month of 7 month needs to be added by 1 virtual machine as compared with 9 am to 10 am on 1 month of 7 month, a request may be made to cloud resources, and 1 virtual machine is added, and 3 virtual machines are used from 10 am to 11 am on 1 month of 7 month.
It should be noted that: when cluster load changes, resources in the cluster must be changed to either enhance or reduce the processing power of the cluster. In the direction, there are two increasing ways: vertical and horizontal telescoping. Vertical scaling refers to increasing or decreasing the resource amount of a virtual machine in a cluster, for example, a virtual machine of 2 cores and 4G is increased to become 4 cores and 8G or decreased to become 1 core and 2G. And horizontal scaling refers to increasing or decreasing the amount of resources by changing the number of virtual machines in a cluster so that the cluster can cope with the change of load. In this document, a horizontal scaling manner is selected for elastic scaling, and this scaling manner is not to add resources to the virtual machines of the application, but to increase the number of the virtual machines to improve the service capability of the cluster, and this manner is more flexible and conforms to the idea of distribution. When the cluster is horizontally stretched and contracted, if the cluster load is too large and exceeds the threshold setting, a resource request is carried out, and the number of virtual machines is increased to meet the threshold setting of the cluster.
Illustratively, the invention relates to cluster resource allocation, and an OpenStack-based flexible scaling service system for cluster resources. The system development tool uses Pycharm, eclipse-jee-mars-2, the Server uses a physical Server and Tomcat 8.0, and the database uses MySQL Server 5.0. The foreground page of the system is made using WebStorm.
As shown in fig. 8, since OpenStack has a monitoring service, data in the text is collected from OpenStack, data in the gnochi service is collected periodically through a script file, and stored in a database, and when the data format is not required in the text, the data is processed into the data format required in the text. Respectively training a GM-CNN model and a DT classifier by processing and analyzing log information, predicting the load at the next moment when new data is monitored, inputting the predicted load into the DT classifier for classification, and selecting a corresponding adjustment rule, namely selecting a telescopic rule from a telescopic rule set, wherein the telescopic rule set comprises a telescopic rule 1, a telescopic rule 2, a telescopic rule 3, a telescopic rule 4, a telescopic rule 5 and … …. Therefore, the cluster resources are adjusted through the load balancer, namely the number of the virtual machines is requested from the cloud resources, and 4 virtual machines are requested.
The system can monitor cloud resource operation and environment prediction, and the flexible rule set is constructed and maintained according to performance change or cloud environment change.
In order to improve the accuracy of the first load, an embodiment of the present invention further provides a manner of modifying the first load, which is shown in fig. 9:
s900: predicting the load of each second historical time period according to the load of a plurality of historical time periods corresponding to each second historical time period;
the historical time period corresponding to the second historical time period is a time period which is the date before the date to which the second historical time period belongs and is the same as the second historical time period;
illustratively, the predicted time period is 11 am to 12 am on 1 am of 7 months, then 10 am to 11 am on 1 am of 7 months, 9 am to 10 am on 1 am of 7 months, 9 am to 1 am of 7 months, 9 am on 1 am of 7 months, 8 am to 9 am on 1 am of 7 months, and 8 am to 7 am on 1 am of 7 months, which are the second historical time period.
Then, for the second historical time period from 10 am on 1 st of 7 months to 11 am on 1 st of 7 months, the historical time periods corresponding to the second historical time period are 10 am on 30 am of 6 months to 11 am on 30 am of 6 months, 10 am on 29 am of 6 months to 11 am on 29 th of 6 months, and 10 am on 28 am of 6 months to 11 am on 28 am of 6 months.
Then, for the second historical time period of 9 am of 1 am of 7 months to 10 am of 1 am of 7 months, the historical time periods corresponding to the second historical time period are respectively 9 am of 6 months 30 am to 10 am of 6 months 30 am, 9 am of 6 months 29 am to 10 am of 6 months 29 am, and 9 am of 6 months 28 am to 10 am of 6 months 28 am.
Then, for the second historical time period from 8 am on 1 st of 7 months to 9 am on 1 st of 7 months, 8 am on 30 am on 6 months to 9 am on 30 th of 6 months, 8 am on 29 am on 6 months to 9 am on 29 th of 6 months, and 9 am on 28 am on 6 months to 28 am on 6 months, the historical time periods corresponding to the second historical time period are all set.
Then, for the second historical time period of 7 am to 7 am 1 am 8 am on 7 month 1, 7 am to 6 am 30 am 8 am on 6 month 30, 7 am to 6 am 29 am 8 am on 6 month 29, and 8 am on 6 month 28 am 7 am to 6 month 28 am 8 am, which are all the historical time periods corresponding to the second historical time period.
S910: determining an adjustment error and an adjustment direction according to the predicted load of each second historical time period and the actual load of each second historical time period;
in detail: taking the average value of the corresponding errors of each second historical time period between the predicted load of each second historical time period and the actual load of each second historical time period as an adjustment error; or alternatively
And predicting an adjustment error according to the corresponding error of each second historical time period between the predicted load of each second historical time period and the actual load of each second historical time period.
And inputting the error corresponding to each second historical time period between the predicted load of each second historical time period and the actual load of each second historical time period into the neural network as input to obtain an adjustment error.
For example, the load at 10 am to 11 am of 6 months 30, the load at 10 am to 6 months 29, the load at 11 am of 6 months 29, the load at 10 am to 6 months 28, the load at 11 am of 6 months 28, the load at 10 am to 6 months 30, the load at 11 am of 7 months 1, the load at 10 am to 6 months 30, the load at 11 am; the error corresponding to the second historical time period of 10 am-11 am on 1 am of 7 months is the difference between the predicted load at 11 am on 10 am-30 am on 1 am of 7 months and the actually detected load at 11 am on 10 am-30 am on 1 am of 7 months;
predicting the load of 9 am to 10 am of 30 am of 7 month (1 am) to 6 month (30 am) by the load of 9 am to 10 am of 30 am of 6 month (29 am), 9 am to 6 am 10 am of 29 am of 6 month (28 am), and 10 am of 9 am to 6 am 28 am of 6 month (6 am); the error corresponding to the second historical time period of 9 am to 10 am of 1 st of 7 th month is the difference between the predicted load of 9 am to 30 am of 6 th month of 1 st of 7 th month and the actually detected load of 10 am of 9 am to 30 am of 6 th month of 1 st of 7 th month;
through the loads of 8 am 8 to 6 am 30 am 9 am 6 to 29 am 6 to 9 am 6 to 28 am 6 to 28 am 9 am 6 to 7 am 1 am 8 to 6 to 30 am 9 am 7, the load of 7 am 1 am 8 am to 6 am 30 am is predicted; the error corresponding to the second historical time period from 8 am to 9 am of 7 month 1, which is the difference between the predicted load from 8 am to 6 am 30 am of 7 month 1 and the actually detected load from 8 am to 6 am 30 am of 7 month 1.
Taking the average value of the error corresponding to the second historical time period of 10 am to 11 am on 1 st of 7 th month, the error corresponding to the second historical time period of 9 am to 1 am of 7 th month and the error corresponding to the second historical time period of 9 am to 1 am of 7 th month as the adjustment error;
and predicting and adjusting errors according to the error corresponding to the second historical time period from 10 am of 1 am of 7 months to 11 am of 1 st of 7 months, the error corresponding to the second historical time period from 9 am of 1 am of 7 months to 10 am of 1 st of 7 months, and the error corresponding to the second historical time period from 8 am of 1 am of 7 months to 9 am of 1 am of 7 months.
Wherein, according to the predicted load of each second historical time period and the actual load of each second historical time period, determining the adjusting direction comprises:
if the sum of the errors in the first error group is larger than the sum of the errors in the second error group, the direction is adjusted to be a positive direction; the error in the first error group is an error corresponding to a second historical time period, wherein the predicted load of the second historical time period is greater than the actual load of the second historical time period; the error in the second error group is the error corresponding to the second historical time period when the predicted load of the second historical time period is smaller than the actual load of the second historical time period;
and if the sum of the errors in the first error group is smaller than the sum of the errors in the second error group, the direction is adjusted to be a negative direction.
The error in the first error group is the predicted load minus the actual load to be a positive value, which is the error in the first error group. The error in the second error set is the predicted load minus the actual load to a negative value, which is the error in the second error set.
When the sum of all errors in the first error group is larger than that in the second error group, namely the positive values are more, the direction is adjusted to be a positive direction, namely the first load is increased; when the sum of all errors in the first error group is smaller than the sum of the errors in the second error group, i.e. more negative, then the adjustment direction is negative, i.e. the first load is reduced.
S920: if the adjusting direction is a positive direction, taking the sum of the adjusting error and the first load as a new first load;
s930: if the adjustment direction is a negative direction, the difference between the adjustment error and the first load is used as a new first load.
In summary, as shown in fig. 10, the current algorithm knows the actual load of t5 when t5 arrives today, and then adjusts the resource according to the actual load, which is also called responsive scaling. The method is predictive expansion and contraction, when the predicted time period is t5 of today, the second load of t5 is predicted according to the workload of the target users of t 1-t 4 of today, the first load of t5 of today is predicted according to the workload of Day 1-Day 5 of t5, the load of t5 is obtained according to the predicted first load and second load of t5, and then the adjustment amount of resources is obtained according to the load of t 5. The working efficiency is improved from the practical prediction point of view.
As shown in fig. 11, the present invention further provides a resource adjusting apparatus, including:
a first prediction module 1100 configured to predict a first load for the predicted time period based on target user workloads for a plurality of first historical time periods, and predict a second load for the predicted time period based on target user workloads for a plurality of second historical time periods; the first historical time period is a time period which is the date before the date to which the predicted time period belongs and is the same as the predicted time period; the second historical time period is the time period before the predicted time period, and the date of the second historical time period is the same as the date of the predicted time period;
a second prediction module 1110, configured to predict a load of the prediction time period according to the first load and the second load;
the classification module 1120 is configured to classify the load of the prediction time period to obtain a total amount of resources of the prediction time period, and determine the target user resource adjustment value according to the total amount of resources of the prediction time period.
Optionally, the second prediction module 1110 is configured to:
and taking the sum of the product of the first load and the weight corresponding to the first load and the product of the second load and the weight corresponding to the second load as the load of the prediction time period.
Optionally, the apparatus further comprises: a modification module;
the modification module is used for predicting the load of each second historical time period according to the load of a plurality of historical time periods corresponding to each second historical time period; the historical time period corresponding to the second historical time period is a time period which is the date before the date to which the second historical time period belongs and is the same as the second historical time period;
determining an adjustment error and an adjustment direction according to the predicted load of each second historical time period and the actual load of each second historical time period;
if the adjusting direction is a positive direction, taking the sum of the adjusting error and the first load as a new first load;
and if the adjusting direction is a negative direction, taking the difference between the adjusting error and the first load as a new first load.
Optionally, the modification module is specifically configured to:
taking the average value of the corresponding errors of each second historical time period between the predicted load of each second historical time period and the actual load of each second historical time period as an adjustment error; or
And predicting an adjustment error according to the corresponding error of each second historical time period between the predicted load of each second historical time period and the actual load of each second historical time period.
Optionally, the modification module is specifically configured to:
if the sum of the errors in the first error group is larger than the sum of the errors in the second error group, the adjusting direction is a positive direction; the error in the first error group is an error corresponding to a second historical time period, wherein the predicted load of the second historical time period is greater than the actual load of the second historical time period; the error in the second error group is an error corresponding to a second historical time period when the predicted load of the second historical time period is smaller than the actual load of the second historical time period;
and if the sum of the errors in the first error group is smaller than the sum of the errors in the second error group, the adjusting direction is a negative direction.
Optionally:
a first prediction module 1100, specifically configured to predict, according to target user workloads of a plurality of first historical time periods, first loads of the prediction time periods through a GM (1,1) gray model; predicting a second load of the predicted time period according to target user workloads of a plurality of second historical time periods through a CNN convolutional neural network;
the classifying module 1120 is specifically configured to classify the load of the prediction time period by a DT classifier to obtain a total resource amount of the prediction time period.
In addition, the resource adjustment method and apparatus of the embodiments of the present invention described in conjunction with fig. 1 to 11 may be implemented by a server.
The server includes: a processor and a memory;
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the resource adjustment method as in any one of the above embodiments.
The memory may be used to store software programs and data. The processor performs various functions of the server and data processing by executing software programs or data stored in the memory. The memory may include high speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. The memory stores an operating system that enables the server to run. The memory of the present application may store an operating system and various application programs, and may also store codes for performing the methods described in the embodiments of the present application.
The processor is a control center of the server, connects various parts of the entire terminal using various interfaces and lines, and performs various functions of the server and processes data by operating or executing software programs stored in the memory and calling data stored in the memory. In some embodiments, a processor may include one or more processing units; the processor may also integrate an application processor, which primarily handles operating systems, user interfaces, applications, etc., and a baseband processor, which primarily handles wireless communications. It will be appreciated that the baseband processor described above may not be integrated into the processor. The processor can run an operating system, an application program, user interface display and touch response, and the processing method in the embodiment of the application.
In addition, in combination with the server in the above embodiments, an embodiment of the present invention may provide a storage medium, where instructions executed by a processor of the server enable the server to perform the resource adjustment method according to any one of the above embodiments.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.

Claims (10)

1. A method for resource adjustment, comprising:
predicting a first load for the predicted time period based on a plurality of target user workloads for a first historical time period, and predicting a second load for the predicted time period based on a plurality of target user workloads for a second historical time period; the first historical time period is a time period which is the date before the date to which the predicted time period belongs and is the same as the predicted time period; the second historical time period is the time period before the predicted time period, and the date of the second historical time period is the same as the date of the predicted time period;
predicting a load for the predicted time period based on the first load and the second load;
and classifying the load of the prediction time period to obtain the total resource amount of the prediction time period, and determining the target user resource adjustment value according to the total resource amount of the prediction time period.
2. The resource adjustment method of claim 1, wherein predicting the load for the predicted time period based on the first load and the second load comprises:
and taking the sum of the product of the first load and the weight corresponding to the first load and the product of the second load and the weight corresponding to the second load as the load of the prediction time period.
3. The resource adjustment method according to claim 1 or 2, wherein before predicting the load of the predicted time period from the first load and the second load, the method further comprises:
predicting the load of each second historical time period according to the load of a plurality of historical time periods corresponding to each second historical time period; the historical time period corresponding to the second historical time period is a time period which is the date before the date to which the second historical time period belongs and is the same as the second historical time period;
determining an adjustment error and an adjustment direction according to the predicted load of each second historical time period and the actual load of each second historical time period;
if the adjusting direction is a positive direction, taking the sum of the adjusting error and the first load as a new first load;
and if the adjusting direction is a negative direction, taking the difference between the adjusting error and the first load as a new first load.
4. The resource adjustment method according to claim 3, wherein determining the adjustment error according to the predicted load of each second historical time period and the actual load of each second historical time period comprises:
taking the average value of the corresponding errors of each second historical time period between the predicted load of each second historical time period and the actual load of each second historical time period as an adjustment error; or
And predicting an adjustment error according to the corresponding error of each second historical time period between the predicted load of each second historical time period and the actual load of each second historical time period.
5. The resource adjustment method according to claim 3, wherein determining the adjustment direction according to the predicted load of each second historical time period and the actual load of each second historical time period comprises:
if the sum of the errors in the first error group is larger than the sum of the errors in the second error group, the adjusting direction is a positive direction; the error in the first error group is an error corresponding to a second historical time period, wherein the predicted load of the second historical time period is greater than the actual load of the second historical time period; the error in the second error group is an error corresponding to a second historical time period when the predicted load of the second historical time period is smaller than the actual load of the second historical time period;
and if the sum of the errors in the first error group is smaller than the sum of the errors in the second error group, the adjusting direction is a negative direction.
6. The resource adjustment method according to claim 1, wherein:
predicting a first load for the predicted time period based on a target user workload for a plurality of first historical time periods, comprising: predicting a first load of the prediction time period according to target user workloads of a plurality of first historical time periods through a GM (1,1) gray model;
predicting a second load for the predicted time period based on a target user workload for a plurality of second historical time periods, comprising: predicting a second load of the predicted time period according to target user workloads of a plurality of second historical time periods through a CNN convolutional neural network;
classifying the load of the prediction time period to obtain the total resource amount of the prediction time period, wherein the steps of: and classifying the load of the prediction time period through a DT classifier to obtain the total resource amount of the prediction time period.
7. A resource adjustment apparatus, comprising:
a first prediction module for predicting a first load for a prediction time period based on target user workloads for a plurality of first historical time periods, and predicting a second load for the prediction time period based on target user workloads for a plurality of second historical time periods; the first historical time period is a time period which is the date before the date to which the predicted time period belongs and is the same as the predicted time period; the second historical time period is a time period before the prediction time period, and the date of the second historical time period is the same as the date of the prediction time period;
a second prediction module configured to predict a load of the prediction time period according to the first load and the second load;
and the classification module is used for classifying the load of the prediction time period to obtain the total resource amount of the prediction time period, and determining the target user resource adjustment value according to the total resource amount of the prediction time period.
8. The resource adjustment apparatus of claim 7, wherein the second prediction module is configured to:
and taking the sum of the product of the first load and the weight corresponding to the first load and the product of the second load and the weight corresponding to the second load as the load of the prediction time period.
9. A server, comprising: a processor and a memory;
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the resource adjustment method of any one of claim 1 to claim 6.
10. A storage medium, wherein instructions in the storage medium, when executed by a processor of a server, enable the server to perform the resource adjustment method of any one of claims 1 to 6.
CN202210855905.1A 2022-07-20 2022-07-20 Resource adjusting method, device, server and storage medium Pending CN115269182A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210855905.1A CN115269182A (en) 2022-07-20 2022-07-20 Resource adjusting method, device, server and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210855905.1A CN115269182A (en) 2022-07-20 2022-07-20 Resource adjusting method, device, server and storage medium

Publications (1)

Publication Number Publication Date
CN115269182A true CN115269182A (en) 2022-11-01

Family

ID=83767644

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210855905.1A Pending CN115269182A (en) 2022-07-20 2022-07-20 Resource adjusting method, device, server and storage medium

Country Status (1)

Country Link
CN (1) CN115269182A (en)

Similar Documents

Publication Publication Date Title
CN108009016B (en) Resource load balancing control method and cluster scheduler
CN107273185B (en) Load balancing control method based on virtual machine
CN108089921B (en) Server for cloud big data operation architecture and operation resource optimization method thereof
US9571347B2 (en) Reactive auto-scaling of capacity
CN109324875B (en) Data center server power consumption management and optimization method based on reinforcement learning
CN110389820B (en) Private cloud task scheduling method for resource prediction based on v-TGRU model
WO2021159638A1 (en) Method, apparatus and device for scheduling cluster queue resources, and storage medium
Bi et al. SLA-based optimisation of virtualised resource for multi-tier web applications in cloud data centres
CN111026553B (en) Resource scheduling method and server system for offline mixed part operation
WO2013174451A1 (en) Method for executing processes on a worker machine of a distributed computing system and a distributed computing system
CN105373432A (en) Cloud computing resource scheduling method based on virtual resource state prediction
CN107317836A (en) One kind mixing cloud environment lower time appreciable request scheduling method
CN112835698A (en) Heterogeneous cluster-based dynamic load balancing method for request classification processing
CN116263701A (en) Computing power network task scheduling method and device, computer equipment and storage medium
CN112181664A (en) Load balancing method and device, computer readable storage medium and electronic equipment
CN107301088A (en) A kind of method and apparatus for managing virtual machine batch migration
JP6094594B2 (en) Information system construction support apparatus, information system construction support method, and information system construction support program
Zhang et al. A data stream prediction strategy for elastic stream computing systems
Zhang et al. PRMRAP: A proactive virtual resource management framework in cloud
CN115269182A (en) Resource adjusting method, device, server and storage medium
CN116360921A (en) Cloud platform resource optimal scheduling method and system for electric power Internet of things
JP5997659B2 (en) Distributed processing system and distributed processing method
JP4322094B2 (en) Information processing method, service time deriving method, and processing unit number adjusting method
CN112130979B (en) Method, device, terminal and medium for scheduling task and training neural network model
CN114741160A (en) Dynamic virtual machine integration method and system based on balanced energy consumption and service quality

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination