CN113127803A - Method and device for establishing service cluster capacity estimation model and electronic equipment - Google Patents
Method and device for establishing service cluster capacity estimation model and electronic equipment Download PDFInfo
- Publication number
- CN113127803A CN113127803A CN201911398084.8A CN201911398084A CN113127803A CN 113127803 A CN113127803 A CN 113127803A CN 201911398084 A CN201911398084 A CN 201911398084A CN 113127803 A CN113127803 A CN 113127803A
- Authority
- CN
- China
- Prior art keywords
- service
- capacity
- model
- historical
- service cluster
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 72
- 238000013016 damping Methods 0.000 claims abstract description 11
- 238000004364 calculation method Methods 0.000 claims description 10
- 238000004422 calculation algorithm Methods 0.000 claims description 6
- 238000012897 Levenberg–Marquardt algorithm Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000012937 correction Methods 0.000 claims description 4
- 238000000926 separation method Methods 0.000 claims description 4
- 238000012163 sequencing technique Methods 0.000 claims 1
- 230000004043 responsiveness Effects 0.000 abstract description 5
- 230000009466 transformation Effects 0.000 description 10
- 230000008569 process Effects 0.000 description 7
- 238000012545 processing Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 239000000126 substance Substances 0.000 description 3
- 238000004590 computer program Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000007476 Maximum Likelihood Methods 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013398 bayesian method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/11—Complex mathematical operations for solving equations, e.g. nonlinear equations, general mathematical optimization problems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
Abstract
The application discloses a method and a device for establishing a service cluster capacity estimation model and electronic equipment, and relates to the field of cloud computing. The method for establishing the service cluster capacity estimation model comprises the following steps: establishing an autoregressive moving average model for estimating the capacity of the service cluster; selecting a damping least square method to calculate based on a historical service capacity use sequence of the service cluster, and obtaining a model parameter of an autoregressive moving average model when the sum of squares of residual errors is a minimum value; correcting model parameters of an autoregressive moving average model based on the latest service capacity usage of a service cluster to obtain a service cluster capacity estimation model; wherein the historical traffic capacity usage sequence includes traffic capacity usage by the traffic cluster over a plurality of consecutive historical time periods. The establishment method and device of the service cluster capacity pre-estimation model and the electronic equipment can improve the responsiveness of the model to external factors and improve the accuracy of model pre-estimation.
Description
Technical Field
The application relates to the field of cloud computing, in particular to a method and a device for establishing a service cluster capacity estimation model and electronic equipment.
Background
In a cloud environment, the service cluster capacity represents the processing capacity, particularly the maximum processing capacity, of the service cluster. The method and the device can accurately predict the use amount of the capacity of the service cluster, and have very important significance for normal operation of a system, planning of service capacity, service guarantee and the like.
At present, the prediction of the usage of the service cluster capacity generally includes monitoring the service cluster capacity, periodically collecting the usage of the service cluster capacity, analyzing the collected historical data, and estimating the future usage of the service cluster capacity according to the trend of the historical data.
However, in practical situations, due to the influence of complex and variable external environmental factors, it is often difficult to predict future use conditions of the cluster capacity according to the trend of historical data to realize accurate prediction of the use conditions of the cluster capacity.
Disclosure of Invention
The embodiment of the application provides a method and a device for establishing a service cluster capacity estimation model and electronic equipment, so as to at least solve the problem that accurate prediction of the service condition of the cluster capacity is difficult to realize in the related technology.
The embodiment of the application adopts the following technical scheme:
a method for establishing a service cluster capacity estimation model comprises the following steps:
establishing an autoregressive moving average model for estimating the capacity of the service cluster;
selecting a damping least square method to calculate based on a historical service capacity use sequence of a service cluster, and obtaining a model parameter of the autoregressive moving average model when the sum of squares of residual errors is a minimum value;
correcting the model parameters of the autoregressive moving average model based on the latest service capacity usage of the service cluster to obtain a service cluster capacity estimation model;
wherein the historical traffic capacity usage sequence includes traffic capacity usage by the traffic cluster over a plurality of consecutive historical time periods.
A device for establishing a service cluster capacity estimation model comprises:
the establishing module is configured to establish an autoregressive moving average model for service cluster capacity estimation;
the calculation module is configured to select a damping least square method to calculate based on a historical service capacity use sequence of the service cluster, and obtain a model parameter of the autoregressive moving average model when the sum of squares of residual errors is a minimum value;
the correction module is configured to correct the model parameters of the autoregressive moving average model based on the latest service capacity usage of the service cluster to obtain a service cluster capacity estimation model;
wherein the historical traffic capacity usage sequence includes traffic capacity usage by the traffic cluster over a plurality of consecutive historical time periods.
An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the method for establishing the service cluster capacity prediction model according to any one of claims 1 to 8.
The embodiment of the application adopts at least one technical scheme which can achieve the following beneficial effects:
the damping least square method is selected for calculation when the service cluster capacity estimation model is established, the model parameters of the autoregressive moving average model when the sum of squares of residual errors is a minimum value are obtained, and the model parameters of the autoregressive moving average model are corrected based on the latest service capacity usage of the service cluster, so that the responsiveness of the model to external factors can be improved, and the accuracy of model estimation is improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
fig. 1 is a flowchart of a method for establishing a service cluster capacity estimation model according to an embodiment of the present application.
Fig. 2 is a flowchart of a method for building another service cluster capacity prediction model according to an embodiment of the present application.
Fig. 3 is a flowchart of denoising a historical traffic capacity usage sequence according to an embodiment of the present disclosure.
Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Fig. 5 is a schematic structural diagram of an apparatus for building a service cluster capacity prediction model according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In order to realize trend prediction of future cluster capacity use conditions, the embodiment of the application provides a method, a device and electronic equipment for establishing a service cluster capacity prediction model, the method, the device and the electronic equipment for establishing the service cluster capacity prediction model can establish an autoregressive sliding average model for service cluster capacity prediction, confirm model parameters of the model according to historical service capacity and correct the model parameters of the model according to the latest service capacity use amount, so that the responsiveness of the model to external factors is improved, and the accuracy of model prediction is improved.
The service cluster refers to a plurality of servers which provide the same service together, and the server which provides the same service is the service cluster.
The method for establishing the service cluster capacity estimation model provided by the embodiment of the present application is explained in detail below.
The method provided by the embodiment of the present application may be applied to a server, where the server may be one of servers in a service cluster or other servers besides the service cluster, and except for special description, the embodiment of the present application all uses the server as an execution subject for description.
It is to be understood that the described execution body does not constitute a limitation of the embodiments of the present application.
Specifically, the flow of the method for establishing the service cluster capacity estimation model is shown in fig. 1, and may include the following steps:
and step S11, establishing an autoregressive moving average model for estimating the capacity of the service cluster.
In the embodiment of the present application, the Autoregressive moving average (ARMA) model is formed by combining an Autoregressive (AR) model moving average (MA and a) model. The mathematical expression of the ARMA model may be:
yt＝Φ1yt-1+Φ2yt-2+…+Φpyt-p+Φt-θ1Φt-1-θ2Φt-2-…-θqΦt-q (1)
wherein p is the order of AR model, q is the order of MA model, ytRepresenting the actual traffic capacity usage, y, in the historical traffic capacity usage sequence over a time period tt-pRepresents the traffic capacity usage, Φ, of the p-th time period before the time period t1Represents the positive correlation coefficient, phi, corresponding to the first 1 time segment of the time segment t2Represents the positive correlation coefficient, phi, corresponding to the 2 nd time period before the time period tpRepresents the positive correlation coefficient, theta, corresponding to the p-th time period before the time period t1Represents the partial correlation coefficient, theta, corresponding to the 1 st time segment before the time segment t2Representing the partial correlation coefficient, theta, corresponding to the 2 nd time period before the time period tqRepresents the partial correlation coefficient, phi, corresponding to the qth time period before the time period ttRepresents the residual error corresponding to the time period t, i.e. the difference between the actual value (actual traffic volume usage) and the predicted value (estimated traffic volume usage), Φt-1Residual error corresponding to the first 1 time segment of the time segment t，Φt-2Representing the residual of the 2 nd slot pair before slot t, Φt-qRepresenting the residual corresponding to the qth time segment before time segment t.
The service capacity may be, but is not limited to, at least one of a CPU, a memory, a disk, and a network bandwidth of the service cluster.
The historical service capacity usage sequence includes service capacity usage of the service cluster in a plurality of continuous historical time periods, and the service capacity usage in the sequence may be a sampling value of the service capacity usage in the corresponding time period, and the like.
In step S12, based on the historical service capacity usage sequence of the service cluster, a damped least squares method is selected for calculation, and a model parameter of the autoregressive moving average model is obtained when the sum of squares of residuals is a minimum value.
In the embodiment of the present application, the model parameters refer to a positive correlation coefficient and a partial correlation coefficient of an autoregressive moving average model.
Specifically, the initial values of the model parameters of the autoregressive moving average model are determined by a reverse process, and the known historical traffic capacity use sequence omega is assumed1，ω2，…，ωnNeed to estimate ωn+1(wherein, ω isnRefers to the traffic capacity usage at time n, i.e. yn) From the above equation (1), we can obtain:
φ(F)ωt＝θ(F)Φt (2)
in the formula, phi (F) omegat＝yt-Φ1yt-1-Φ2yt-2-...-Φpyt-p，θ(F)Φt＝Φt-θ1Φt-1-θ2Φt-2-...-θqΦt-q. Due to the stationarity of the operator, the value (y) is estimated outside a certain time range t ═ epsilont) Substantially equal to zero, and therefore, to a sufficient approximation, the following equation can be used to solve:
the blending process is replaced by an epsilon order moving average process, then the model is linearized and the sum of the squares of the residuals is minimized to obtain:
then [ phi ] is mixedt]Using taylor series expansion, using β as the sign of p + q parameters (phi, theta), and having a set of estimated corresponding parameter values β around this value0＝(β1,0,β2,0,…,βk,0) Then, there are:
wherein the content of the first and second substances,
for model, value
Using recursion method for t-1-epsilon, …, n, then repeating recursion calculation and calculating [ phi [ ]t]And finally, calculating model parameters of an expected ARMA model, namely a positive correlation coefficient and a partial correlation coefficient.
In step S13, the model parameters of the autoregressive moving average model are corrected based on the latest service capacity usage of the service cluster, and a service cluster capacity estimation model is obtained.
In the embodiment of the application, the model parameters of the autoregressive moving average model can be corrected through a Kalman filtering algorithm based on the latest service capacity usage amount of the service cluster, so that a service cluster capacity prediction model is obtained.
Specifically, according to modern control theory, the ARMA model can be transformed into the following spatial form:
the measurement equation: y (k) ═ hx (k) + v (k) (6)
The state equation is as follows: x (k +1) ═ Φ X (k) + Γ e (k) (7)
Wherein the content of the first and second substances,
b is formed byq+1＝bq+2＝…＝bp＝0,
Then, based on the state space forms (6) and (7), the kalman filtering algorithm is obtained as follows:
wherein the content of the first and second substances,
for the estimation of X (k), P (k +1| k) is the prediction bias covariance matrix, and P (k +1| k +1) is the measurement correction bias covariance matrix.
From this, the calculated optimized [ phi ] can be obtainedt]And calculating the corrected ARMA model parameters to finally obtain a service cluster capacity estimation model.
When the future cluster capacity use condition is estimated, the previous service capacity use sequence can be input into the service cluster capacity estimation model for calculation, and the estimated service capacity use amount in the next time period is obtained.
In the embodiment of the present application, the model parameters of the autoregressive moving average model are corrected by using a kalman filter algorithm, and it can be understood that in some other embodiments, other algorithms may be used to correct the model parameters of the autoregressive moving average model, for example, an NLBFGS algorithm may also be used.
Furthermore, after the estimated service capacity usage of the next time period is obtained, resource allocation can be performed on the service cluster according to the estimated service capacity usage. For example, resources may be reclaimed when the projected traffic volume usage is too low, and resources may be expanded when projected traffic volume usage is too high.
Please refer to fig. 2, which is a flowchart of a method for establishing a prediction model of service cluster capacity according to another embodiment of the present application, and as shown in fig. 2, the method may include the following steps:
in step S21, the historical traffic volume usage sequence is denoised.
When a server in a service cluster provides service, the service capacity usage (such as CPU usage) may be instantaneously too high, but the server may quickly recover to normal, and if the acquired value is the maximum value and is used for training a service cluster capacity prediction model, the service cluster capacity prediction model predicts the cluster capacity usage in the next time period or time periods, and a large deviation may exist, resulting in inaccurate prediction. Therefore, before training the service cluster capacity prediction model, the service capacity usage in the collected historical service capacity usage sequence needs to be denoised.
The historical service capacity use sequence is denoised by adopting, but not limited to, a binning method, wavelet analysis denoising and the like. In the embodiment of the application, a box separation method is adopted to denoise the historical service capacity use sequence.
Specifically, the process may be as shown in fig. 3, and includes the following steps:
in step S31, the traffic volume usages in the historical traffic volume usage sequence are sorted by numerical size.
The sorting mode can be sorting from small to large or sorting from large to small.
In step S32, the upper quartile and the lower quartile corresponding to the historical traffic capacity usage sequence are determined.
After the traffic capacity usage in the historical traffic capacity usage sequence is sorted, the upper quartile Q and the lower quartile P in the sorted sequence may be taken, and the difference between the two may be represented as N ═ Q-P.
It is understood that, in some other embodiments, the upper and lower eighth bits in the sequence, or the upper and lower sixteenth bits in the sequence, etc. may also be taken, and the embodiments of the present application are not particularly limited.
In step S33, the minimum value and the maximum value are determined based on the upper quartile and the lower quartile.
After the upper quartile Q and the lower quartile P are obtained, the maximum value and the minimum value of the required service capacity usage can be determined according to the upper quartile Q, the lower quartile P and a preset coefficient.
The maximum value may be denoted as max ═ Q + k × N, and the minimum value may be denoted as min ═ P-k × N. Where k is an artificially set coefficient indicating the tolerance to an abnormal value, and is generally about 1.5.
In step S34, the traffic volume usage amount corresponding to the interval between the minimum value and the maximum value is excluded from the historical traffic volume usage sequence.
It can be understood that after the service capacity usage amount of the corresponding numerical value outside the interval of the minimum value and the maximum value is eliminated, the eliminated data can be complemented by an interpolation method.
In the embodiment of the application, the service capacity usage in the historical service capacity usage sequence after the removing step can be subjected to box-cox conversion.
The formula of the box-cox transform can be expressed as
Where λ is a transformation parameter of the box-cox transformation, the transformation is a logarithmic transformation when λ is 0, a reciprocal transformation when λ is-1, and a square root transformation when λ is 0.5. The estimation of the parameter lambda in the box-cox transformation can be carried out by maximum likelihood estimation or a Bayesian method, and the transformation form can be determined by solving the lambda.
After the box-co conversion is performed, the service capacity usage amount after the box-co conversion needs to be reduced.
The reduction formula can be expressed as
In step S22, an autoregressive moving average model for service cluster capacity estimation is established.
In step S23, based on the history service capacity usage sequence after denoising, a damped least squares method is selected for calculation, and a model parameter of the autoregressive moving average model is obtained when the sum of squares of residuals is a minimum value.
In step S24, the model parameters of the autoregressive moving average model are corrected based on the latest service capacity usage of the service cluster, and a service cluster capacity estimation model is obtained.
The method for establishing the service cluster capacity estimation model provided by the embodiment of the application can be used for denoising a historical service capacity use sequence by a box separation method before establishing the model, so that the influence of noise data on model training is avoided, meanwhile, data variance can be stabilized through box-cox transformation, the established model is closer to the actual condition, and the estimation accuracy of the service cluster capacity estimation model is ensured. Meanwhile, parameters meeting the precision of the ARMA model are solved by combining a damping least square method, and the precision of the model is ensured. In addition, parameters of the model are corrected by combining Kalman filtering with the minimum service capacity usage amount, so that the responsiveness of the model to external factors can be improved, and the accuracy of model estimation is further improved.
Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application. Referring to fig. 4, at a hardware level, the electronic device includes a processor, and optionally further includes an internal bus, a network interface, and a memory. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory, such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.
The processor, the network interface, and the memory may be connected to each other via an internal bus, which may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 4, but that does not indicate only one bus or one type of bus.
And the memory is used for storing programs. In particular, the program may include program code comprising computer operating instructions. The memory may include both memory and non-volatile storage and provides instructions and data to the processor.
The processor reads a corresponding computer program from the nonvolatile memory to the memory and then runs the computer program to form a device for establishing a service cluster capacity estimation model on a logic level. The processor is used for executing the program stored in the memory and is specifically used for executing the following operations:
establishing an autoregressive moving average model for estimating the capacity of the service cluster;
selecting a damping least square method to calculate based on a historical service capacity use sequence of a service cluster, and obtaining a model parameter of the autoregressive moving average model when the sum of squares of residual errors is a minimum value;
correcting the model parameters of the autoregressive moving average model based on the latest service capacity usage of the service cluster to obtain a service cluster capacity estimation model;
wherein the historical traffic capacity usage sequence includes traffic capacity usage by the traffic cluster over a plurality of consecutive historical time periods.
The method executed by the apparatus for building a service cluster capacity prediction model according to the embodiments shown in fig. 1 to 3 of the present application may be applied to a processor, or implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.
The electronic device may also execute the method shown in fig. 1-3, and implement the functions of the apparatus for establishing a service cluster capacity estimation model in the embodiment shown in fig. 1-3, which are not described herein again in this embodiment of the present application.
Of course, besides the software implementation, the electronic device of the present application does not exclude other implementations, such as a logic device or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, and may also be hardware or a logic device.
Embodiments of the present application also provide a computer-readable storage medium storing one or more programs, where the one or more programs include instructions, which when executed by a portable electronic device including a plurality of application programs, enable the portable electronic device to perform the method of the embodiment shown in fig. 1, and are specifically configured to:
establishing an autoregressive moving average model for estimating the capacity of the service cluster;
selecting a damping least square method to calculate based on a historical service capacity use sequence of a service cluster, and obtaining a model parameter of the autoregressive moving average model when the sum of squares of residual errors is a minimum value;
correcting the model parameters of the autoregressive moving average model based on the latest service capacity usage of the service cluster to obtain a service cluster capacity estimation model;
wherein the historical traffic capacity usage sequence includes traffic capacity usage by the traffic cluster over a plurality of consecutive historical time periods.
Fig. 5 is a schematic structural diagram of an apparatus for building a service cluster capacity prediction model according to an embodiment of the present application. Referring to fig. 5, in a software implementation, the apparatus for building a service cluster capacity prediction model may include:
an establishing module 51 configured to establish an autoregressive moving average model for service cluster capacity estimation;
a calculation module 52 configured to select a damped least square method for calculation based on a historical service capacity usage sequence of the service cluster, and obtain a model parameter of the autoregressive moving average model when a sum of squares of residuals is a minimum value;
a correction module 53 configured to correct the model parameters of the autoregressive moving average model based on the latest service capacity usage of the service cluster, so as to obtain the service cluster capacity estimation model;
wherein the historical traffic capacity usage sequence includes traffic capacity usage by the traffic cluster over a plurality of consecutive historical time periods.
The device for establishing the service cluster capacity estimation model provided by the embodiment of the application can be used for denoising a historical service capacity use sequence by a box separation method before establishing the model, so that the influence of noise data on model training is avoided, and meanwhile, data variance can be stabilized through box-cox transformation, so that the established model is closer to the actual condition, and the estimation accuracy of the service cluster capacity estimation model is ensured. Meanwhile, parameters meeting the precision of the ARMA model are solved by combining a damping least square method, and the precision of the model is ensured. In addition, parameters of the model are corrected by combining Kalman filtering with the minimum service capacity usage amount, so that the responsiveness of the model to external factors can be improved, and the accuracy of model estimation is further improved.
In short, the above description is only a preferred embodiment of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
Claims (10)
1. A method for establishing a service cluster capacity estimation model is characterized by comprising the following steps:
establishing an autoregressive moving average model for estimating the capacity of the service cluster;
selecting a damping least square method to calculate based on a historical service capacity use sequence of a service cluster, and obtaining a model parameter of the autoregressive moving average model when the sum of squares of residual errors is a minimum value;
correcting the model parameters of the autoregressive moving average model based on the latest service capacity usage of the service cluster to obtain a service cluster capacity estimation model;
wherein the historical traffic capacity usage sequence includes traffic capacity usage by the traffic cluster over a plurality of consecutive historical time periods.
2. The method of claim 1, wherein the calculating based on the historical service capacities of the service clusters by using a sequence and selecting a damped least squares method is performed, and before obtaining model parameters of the autoregressive moving average model when a sum of squares of residuals is a minimum value, the method further comprises:
denoising the historical service capacity usage sequence;
the historical service capacity use sequence based on the service cluster is calculated by selecting a damped least square method, and model parameters of the autoregressive moving average model when the sum of squares of residual errors is a minimum value are obtained, and the method comprises the following steps:
and selecting a damping least square method for calculation based on the denoised historical service capacity use sequence to obtain the model parameter when the sum of squares of the residual errors is a minimum value.
3. The method of claim 2, wherein denoising the sequence of historical traffic capacity usages comprises:
and denoising the historical service capacity use sequence by a box separation method.
4. The method of claim 3, wherein the de-noising the sequence of historical traffic volume usages by binning comprises:
sequencing the service capacity usage in the historical service capacity usage sequence according to the numerical value;
determining an upper quartile and a lower quartile corresponding to the historical service capacity use sequence;
determining a minimum value and a maximum value based on the upper quartile and the lower quartile;
and eliminating the service capacity usage amount of which the corresponding numerical value is outside the interval of the minimum value and the maximum value from the historical service capacity usage sequence.
5. The method of claim 4, wherein after removing traffic volume usage amounts corresponding to values outside the interval between the minimum value and the maximum value from the historical sequence of traffic volume usage, the method further comprises:
carrying out box-cox conversion on the service capacity usage in the historical service capacity usage sequence after the removing step;
and restoring the service capacity utilization amount converted by the box-cox.
6. The method of claim 1, wherein the modifying the model parameters of the autoregressive moving average model based on the latest service capacity usage of the service cluster to obtain the service cluster capacity prediction model comprises:
and correcting the model parameters of the autoregressive moving average model through a Kalman filtering algorithm based on the latest service capacity usage of the service cluster to obtain the service cluster capacity estimation model.
7. The method of claim 1, further comprising:
and inputting the previous service capacity use sequence into the service cluster capacity prediction model to obtain the predicted service capacity use amount of the next time period.
8. The method of claim 7, wherein after obtaining the predicted traffic capacity usage for the next time period, the method further comprises:
and performing resource configuration on the service cluster according to the estimated service capacity usage amount.
9. A device for establishing a service cluster capacity estimation model is characterized by comprising:
the establishing module is configured to establish an autoregressive moving average model for service cluster capacity estimation;
the calculation module is configured to select a damping least square method to calculate based on a historical service capacity use sequence of the service cluster, and obtain a model parameter of the autoregressive moving average model when the sum of squares of residual errors is a minimum value;
the correction module is configured to correct the model parameters of the autoregressive moving average model based on the latest service capacity usage of the service cluster to obtain a service cluster capacity estimation model;
wherein the historical traffic capacity usage sequence includes traffic capacity usage by the traffic cluster over a plurality of consecutive historical time periods.
10. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the method for establishing the service cluster capacity prediction model according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911398084.8A CN113127803A (en) | 2019-12-30 | 2019-12-30 | Method and device for establishing service cluster capacity estimation model and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911398084.8A CN113127803A (en) | 2019-12-30 | 2019-12-30 | Method and device for establishing service cluster capacity estimation model and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113127803A true CN113127803A (en) | 2021-07-16 |
Family
ID=76768962
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911398084.8A Pending CN113127803A (en) | 2019-12-30 | 2019-12-30 | Method and device for establishing service cluster capacity estimation model and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113127803A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115834388A (en) * | 2022-10-21 | 2023-03-21 | 支付宝(杭州)信息技术有限公司 | System control method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107748931A (en) * | 2017-10-20 | 2018-03-02 | 江苏电力信息技术有限公司 | A kind of income of electricity charge Forecasting Methodology based on least square method |
| CN109766234A (en) * | 2018-12-11 | 2019-05-17 | 国网甘肃省电力公司信息通信公司 | Disk storage capacity prediction technique based on time series models |
| CN111813535A (en) * | 2019-04-11 | 2020-10-23 | 中国移动通信集团四川有限公司 | Resource configuration determining method and device and electronic equipment |
-
2019
- 2019-12-30 CN CN201911398084.8A patent/CN113127803A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107748931A (en) * | 2017-10-20 | 2018-03-02 | 江苏电力信息技术有限公司 | A kind of income of electricity charge Forecasting Methodology based on least square method |
| CN109766234A (en) * | 2018-12-11 | 2019-05-17 | 国网甘肃省电力公司信息通信公司 | Disk storage capacity prediction technique based on time series models |
| CN111813535A (en) * | 2019-04-11 | 2020-10-23 | 中国移动通信集团四川有限公司 | Resource configuration determining method and device and electronic equipment |
Non-Patent Citations (3)
| Title |
|---|
| 杨旭 等: "《多元统计分析 从数据到结论》", 北京理工大学出版社, pages: 127 - 150 * |
| 鸣人吃土豆: ""如何处理噪声数据"", 《HTTPS://WWW.JIANSHU.COM/P/389682AA5429》, 26 December 2017 (2017-12-26), pages 1 - 3 * |
| 鸣人吃土豆: ""如何处理噪声数据"", 《HTTPS://WWW.JIANSHU.COM/P/389682AA5429》, pages 1 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115834388A (en) * | 2022-10-21 | 2023-03-21 | 支付宝(杭州)信息技术有限公司 | System control method and device |
| CN115834388B (en) * | 2022-10-21 | 2023-11-14 | 支付宝(杭州)信息技术有限公司 | System control method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11836576B2 (en) | Distributed machine learning at edge nodes | |
| CN109344789B (en) | Face tracking method and device | |
| CN110008080B (en) | Business index anomaly detection method and device based on time sequence and electronic equipment | |
| AU2021232839A1 (en) | Updating Attribute Data Structures to Indicate Trends in Attribute Data Provided to Automated Modelling Systems | |
| CN111896914A (en) | Cooperative positioning method, device, equipment and storage medium | |
| CN109426912B (en) | Wind control system optimization method, system and device and electronic equipment | |
| CN110334013B (en) | Decision engine testing method and device and electronic equipment | |
| CN108111353B (en) | Prepaid card remaining flow prediction method, network terminal and storage medium | |
| CN114500339B (en) | Node bandwidth monitoring method and device, electronic equipment and storage medium | |
| CN112737894A (en) | Network quality monitoring method and device, storage medium and electronic equipment | |
| CN112737798B (en) | Host resource allocation method and device, scheduling server and storage medium | |
| CN113127803A (en) | Method and device for establishing service cluster capacity estimation model and electronic equipment | |
| CN112398670A (en) | Flow prediction method, flow prediction device, computing equipment and computer storage medium | |
| CN111104438A (en) | Method and device for determining periodicity of time sequence and electronic equipment | |
| CN111798263A (en) | Transaction trend prediction method and device | |
| CN113190429B (en) | Server performance prediction method and device and terminal equipment | |
| CN103530190A (en) | Load forecasting method and device | |
| US9537740B2 (en) | Monitoring device usage | |
| CN115225543B (en) | Flow prediction method and device, electronic equipment and storage medium | |
| US9998347B2 (en) | Monitoring device usage | |
| CN115525394A (en) | Method and device for adjusting number of containers | |
| CN110896524B (en) | Network structure monitoring method and device | |
| CN111585739B (en) | Phase adjustment method and device | |
| CN115374989A (en) | Load prediction method, device, equipment and computer readable storage medium | |
| CN114637809A (en) | Method, device, electronic equipment and medium for dynamic configuration of synchronous delay time |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |