CN117544635A - Resource management method, device, equipment and storage medium - Google Patents
Resource management method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN117544635A CN117544635A CN202210914934.0A CN202210914934A CN117544635A CN 117544635 A CN117544635 A CN 117544635A CN 202210914934 A CN202210914934 A CN 202210914934A CN 117544635 A CN117544635 A CN 117544635A
- Authority
- CN
- China
- Prior art keywords
- data
- resource
- target resource
- target
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007726 management method Methods 0.000 title claims abstract description 130
- 230000008859 change Effects 0.000 claims abstract description 84
- 238000000034 method Methods 0.000 claims abstract description 59
- 230000000737 periodic effect Effects 0.000 claims abstract description 38
- 238000012545 processing Methods 0.000 claims abstract description 30
- 238000001514 detection method Methods 0.000 claims description 39
- 230000008569 process Effects 0.000 claims description 24
- 238000012549 training Methods 0.000 claims description 19
- 230000015654 memory Effects 0.000 claims description 18
- 238000000605 extraction Methods 0.000 claims description 14
- 238000013528 artificial neural network Methods 0.000 claims description 8
- 125000004122 cyclic group Chemical group 0.000 claims description 4
- 230000010354 integration Effects 0.000 claims description 3
- 238000003062 neural network model Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 5
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- YHXISWVBGDMDLQ-UHFFFAOYSA-N moclobemide Chemical compound C1=CC(Cl)=CC=C1C(=O)NCCN1CCOCC1 YHXISWVBGDMDLQ-UHFFFAOYSA-N 0.000 description 11
- 238000004590 computer program Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 230000007423 decrease Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000012384 transportation and delivery Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007787 long-term memory Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000013439 planning Methods 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
- H04L67/1074—Peer-to-peer [P2P] networks for supporting data block transmission mechanisms
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a resource management method, a device, equipment and a storage medium. The embodiment of the application can be applied to various scenes such as cloud technology, cloud security, artificial intelligence, intelligent traffic, auxiliary driving and the like. The method comprises the following steps: acquiring time sequence data of a target resource, and extracting dynamic change characteristics from the time sequence data; the dynamically changing features include one or more of periodic features, monotonic features, and stationary features; and calling a target prediction model matched with the dynamic change characteristics to perform prediction processing on the time sequence data, so as to obtain the predicted usage amount of the target resource in the next service period. The more accurate predicted usage amount can be obtained, so that excessive resources or insufficient resources can be avoided, the utilization rate of the resources can be improved, the cost can be reduced, and the operation quality can be improved when the resource management operation is executed based on the predicted usage amount.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a storage medium for resource management.
Background
With the rapid development of Cloud technology (Cloud technology), more and more businesses, institutions (e.g., government institutions, schools, etc.), deploy services to Cloud platforms. Under the background of high requirements of cloud resource virtualization and cloud base safety performance, a cloud management platform aiming at a cloud platform is generated.
The cloud management platform can allocate resources for each service system. However, when the allocated resources are too much, the resources are easy to be idle, the utilization rate of the resources is low, and the operation cost of enterprises or institutions is increased; when the allocated resources are insufficient, when the service demand is suddenly increased or the access amount is increased, the situation that the service is not accessible is easy to occur, and huge loss is caused. Therefore, how to accurately perform resource management is a problem to be solved.
Disclosure of Invention
The embodiment of the application provides a resource management method, a device, equipment and a storage medium, which can accurately predict the predicted usage of the next service period and avoid excessive or insufficient resources.
In one aspect, an embodiment of the present application provides a resource management method, where the resource management method includes:
acquiring time sequence data of a target resource, wherein the time sequence data comprises historical use data of the target resource at each time point of a preset time period;
Extracting features of the time sequence data to obtain dynamic change features; the dynamically changing features include one or more of periodic features, monotonic features, and stationary features;
invoking a target prediction model matched with the dynamic change characteristics in at least one prediction model to predict the time sequence data to obtain the predicted usage amount of the target resource in the next service period;
a resource management operation is performed for the target resource based on the predicted usage.
In another aspect, an embodiment of the present application provides a resource management device, including:
an acquisition unit configured to acquire time-series data of a target resource, the time-series data including historical usage data of the target resource at each point in time of a preset time period;
the feature extraction unit is used for extracting features of the time sequence data to obtain dynamic change features; the dynamically changing features include one or more of periodic features, monotonic features, and stationary features;
the prediction processing unit is used for calling a target prediction model matched with the dynamic change characteristics in at least one prediction model to perform prediction processing on the time sequence data so as to obtain the predicted usage amount of the target resource in the next service period;
And a processing unit for performing a resource management operation for the target resource based on the predicted usage.
In still another aspect, an embodiment of the present application provides a resource management device, where the resource management device includes an input interface and an output interface, and the resource management device further includes:
a processor adapted to implement one or more instructions; the method comprises the steps of,
a computer storage medium storing one or more instructions adapted to be loaded by a processor and to perform the steps of:
acquiring time sequence data of a target resource, wherein the time sequence data comprises historical use data of the target resource at each time point of a preset time period;
extracting features of the time sequence data to obtain dynamic change features; the dynamically changing features include one or more of periodic features, monotonic features, and stationary features;
invoking a target prediction model matched with the dynamic change characteristics in at least one prediction model to predict the time sequence data to obtain the predicted usage amount of the target resource in the next service period;
a resource management operation is performed for the target resource based on the predicted usage.
In yet another aspect, the present application provides a computer program product comprising computer instructions stored in a computer-readable storage medium. The processor of the resource management device reads the computer instructions from the computer readable storage medium, the processor executing the computer instructions to cause the resource management device to implement the steps of:
Acquiring time sequence data of a target resource, wherein the time sequence data comprises historical use data of the target resource at each time point of a preset time period;
extracting features of the time sequence data to obtain dynamic change features; the dynamically changing features include one or more of periodic features, monotonic features, and stationary features;
invoking a target prediction model matched with the dynamic change characteristics in at least one prediction model to predict the time sequence data to obtain the predicted usage amount of the target resource in the next service period;
a resource management operation is performed for the target resource based on the predicted usage.
In yet another aspect, embodiments of the present application provide a computer storage medium having one or more instructions stored thereon, the one or more instructions being adapted to be loaded by a processor and to perform the steps of:
acquiring time sequence data of a target resource, wherein the time sequence data comprises historical use data of the target resource at each time point of a preset time period;
extracting features of the time sequence data to obtain dynamic change features; the dynamically changing features include one or more of periodic features, monotonic features, and stationary features;
Invoking a target prediction model matched with the dynamic change characteristics in at least one prediction model to predict the time sequence data to obtain the predicted usage amount of the target resource in the next service period;
a resource management operation is performed for the target resource based on the predicted usage.
In the embodiment of the application, the resource management device acquires time sequence data of the target resource, performs feature extraction on the time sequence data to obtain dynamic change features, and then calls a target prediction model matched with the dynamic change features in at least one prediction model to predict the time sequence data to obtain the predicted usage amount of the target resource in the next service period. The resource management method can predict the predicted use amount of the next service period based on the historical use data included in the time sequence data, can predict the use condition of the resources in advance and predict the resources in real time, so that excessive allocated resources or insufficient allocated resources can be avoided, the cost can be reduced, and the operation quality is improved. In addition, the method and the device call the target prediction model matched with the dynamic change characteristics of the time sequence characteristics in at least one prediction model to predict the time sequence data, namely the resource management equipment can adaptively select the matched target prediction model from the at least one prediction model based on the dynamic change characteristics, and the at least one prediction model can be suitable for the time sequence data with different dynamic change characteristics, so that the flexibility and the compatibility of the prediction model are effectively improved. In addition, the target prediction model is matched with the dynamic change characteristics, and the predicted usage amount of the next service period predicted by the target prediction model is more accurate, so that excessive allocated resources or insufficient allocated resources can be avoided, the cost can be reduced, and the operation quality can be improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic structural diagram of a resource management system according to an embodiment of the present application;
fig. 2 is a schematic flow chart of a resource management method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a monitoring interface provided in an embodiment of the present application;
FIG. 4 is a schematic diagram of time series data and associated data provided by an embodiment of the present application;
FIG. 5 is a flowchart of another resource management method according to an embodiment of the present disclosure;
FIG. 6 is a flowchart of another resource management method according to an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of a resource management device according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of a resource management device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
Referring to fig. 1, fig. 1 shows a schematic architecture of a resource management system. As shown in fig. 1, the resource management system may include at least: a resource device 101 and a resource management device 102. The resource device 101 may be any device that provides a resource. A plurality of resource devices 101 may constitute a resource pool. Alternatively, the resource device 101 may be a terminal device, which may include, but is not limited to, a smart phone, a tablet, a notebook, a desktop computer, a smart home appliance, a smart voice interaction device, a wearable device, a vehicle-mounted terminal, an aircraft, and the like. Alternatively, as shown in fig. 1, the resource device 101 may be a server, where the server may be an independent physical server, may be a server cluster or a distributed system formed by a plurality of physical servers, and may be an artificial intelligence platform that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data.
The resource management device 102 may be any device having resource management capabilities. Alternatively, the resource management device 102 may be a device deployed on a cloud management platform. The resource management device 102 may be used to manage a cloud platform. The cloud platform can be an informationized platform built based on a cloud computing technology. Cloud computing is a computing model that distributes computing tasks across a large number of computer-made resource pools (e.g., resource pools provided by resource devices 101) to enable various business systems to obtain computing power, storage space, and information services as needed.
In one embodiment, the resource management device 102 may allocate resources for each business system. However, when the allocated resources are too much, the resources are easy to be idle, the utilization rate of the resources is low, and the operation cost of enterprises or institutions is increased; when the allocated resources are insufficient, when the service demand is suddenly increased or the access amount is increased, the situation that the service is not accessible is easy to occur, and huge loss is caused. Therefore, how to accurately perform resource management is a problem to be solved. Based on the above, the embodiment of the application provides a resource management method. In the method, the resource management device can acquire time series data of the target resource and extract dynamic change characteristics from the time series data; the dynamically changing features include one or more of periodic features, monotonic features, and stationary features; and calling a target prediction model matched with the dynamic change characteristics to perform prediction processing on the time sequence data, so as to obtain the predicted usage amount of the target resource in the next service period. The more accurate predicted usage amount can be obtained, so that excessive resources or insufficient resources can be avoided, the utilization rate of the resources can be improved, the cost can be reduced, and the operation quality can be improved when the resource management operation is executed based on the predicted usage amount.
In a possible implementation manner, the resource management method of the embodiment of the application can be applied to the related field of cloud resource management. For example, the method can be applied to a container cloud management platform, and a resource pool of the container cloud management platform comprises resources which are not limited to a central processing unit (Central Processing Unit, CPU), integrated memory, network bandwidth and the like and used by tenants. The resource management device can manage the resources corresponding to the tenants based on the resource management method, can meet the resource requirements of the tenants, avoid excessive or insufficient resources of the tenants, can effectively improve the resource utilization rate of the container cloud management platform, and reduces the operation cost. It should be noted that, along with the requirement of service development, the resource management method may also be applied to other resource management scenarios, which will not be described in detail in this application.
Referring to fig. 2, fig. 2 is a flow chart of a resource management method according to an embodiment of the present application. The resource management method may be performed by the above-mentioned resource management device. As shown in fig. 2, the resource management method may include, but is not limited to, S201-S204:
s201: time-series data of the target resource is acquired, wherein the time-series data comprises historical use data of the target resource at each time point of a preset time period.
Resources may include, but are not limited to, CPU, memory, network bandwidth, and the like.
Wherein the usage data is used to indicate the usage of the resource. Alternatively, the usage data may include, but is not limited to, one or both of usage amount, usage rate. The usage rate may refer to a ratio of a usage amount to an amount of allocated resources. For example, when the target resource is a CPU, the allocated resource amount of the CPU is 100 cores, and the usage amount is 30 cores, it can be determined that the usage is 30%. For another example, when the target resource is a memory, the allocated resource amount of the memory is 4000 Megabytes (MB), and the usage amount is 2000MB, the usage rate may be determined to be 50%. It should be noted that, for the network bandwidth, the usage data may include, but is not limited to, one or more of a network in bandwidth, a network out bandwidth, a data packet transmission amount, and a data packet reception amount.
The preset time period may be any history time period. For example, the preset time period may be five days nearest to the current day, or one month nearest to the current month, or 46 minutes 16 seconds at 15 on 29 th 11 th 2021 to 46 minutes 16 seconds at 15 on 06 th 12 th 2021, or the like.
In a possible implementation manner, the resource management device may monitor usage data of various resources, and obtain historical usage data of various resources at various time points in a preset time period. As shown in fig. 3, for example, for the CPU, the CPU usage rate at each point of time of the preset period of time may be acquired; for the memory, the memory utilization rate and the memory utilization amount of each time point of the preset time period can be obtained; for the network bandwidth, the network outgoing bandwidth, the network incoming bandwidth, the data packet transmission amount, and the data packet reception amount at each point in time of the preset period of time may be acquired. Based on this, the resource management device may acquire time-series data of the target resource from the historical usage data of each time point of the preset time period of each resource, for example, when the target resource is a memory, the time-series data of the target resource may be 301 shown in fig. 3.
In a possible embodiment, one or more of an extremum (including a minimum and a maximum), a maximum (including a minimum and a maximum) and an average in the time series data may also be obtained, so as to facilitate the execution of the subsequent steps. As shown in 301 of fig. 3, the time-series data can be obtained with a maximum value of 2934.11, a minimum value of 2104.23, and an average value of 2668.58.
S202: extracting features of the time sequence data to obtain dynamic change features; the dynamically changing features include one or more of periodic features, monotonic features, and stationary features.
The periodic characteristic may be periodicity. The stationary characteristic may also be stationarity, which may mean that the time series data has no significant fluctuations. Monotonic features may also be monotonic (including monotonic increases and monotonic decreases). It should be noted that the monotonic feature may mean that the time series data has a tendency to rise or fall, and that the jitter present does not affect the monotonic feature, i.e., the monotonic feature referred to in the present application may be a non-strict monotonic rise or a monotonic fall.
In one possible implementation, the resource management device may invoke different detection models to determine different dynamic change characteristics. Optionally, the resource management device may invoke the period detection model to process the time series data, and determine whether the dynamic change feature includes periodicity; and invoking a moving average algorithm model to process the time sequence data, and determining whether the dynamic change characteristics comprise stationarity or monotonicity.
The period detection model may include, but is not limited to, one or both of a similarity period detection model constructed based on similarity and an autocorrelation period detection model constructed based on autocorrelation coefficients. In one embodiment, in the case that the period detection model is a similarity period detection model, the resource management device may invoke the similarity period detection model to process the time-series data and determine the frequency-domain sequence data. When the frequency domain sequence data has periodic spike discrete data, determining the dynamic change characteristic includes periodicity. The period of the burr discrete data is the period of the time series data. When the frequency domain sequence data does not have periodic glitch discrete data, the dynamic change characteristic is determined to not include periodicity. Alternatively, the similarity period detection model may be a fourier transform model. The resource management device may invoke a fourier transform model to perform fourier transform processing on the time-series data, to obtain frequency-domain sequence data.
In another embodiment, in the case that the period detection model is an autocorrelation period detection model, the resource management device may call the autocorrelation period detection model to process the time-series data to obtain the correlation data; wherein the associated data includes a hysteresis value i and a correlation coefficient r i Corresponding relation of (3). The determining the dynamically changing feature includes periodicity when the associated data has periodic spike discrete data. The period of the burr discrete data is the period of the time series data. When the associated data does not have periodic glitch discrete data, it is determined that the dynamically changing feature does not include periodicity.
Optionally, the correlation coefficient r i Is the correlation coefficient between the time series data and the lag series data. Specifically, the correlation coefficient r can be determined by the following expression i :r i =corr(Y(t),Y(t-T i ))。
Wherein r is i The time sequence data is the correlation coefficient between the time sequence data and the lag sequence data, t is the time point, Y (t) is the time sequence data, and the time sequence data comprises the use data of the target resource at the time point t; T-T i For a time point T to lag i time points, i.e. a time point T lags time period T i ,Y(t-T i ) For hysteresis sequence data, usage data for i time points is included for the target resource. corr refers to a correlation operation.
Referring to fig. 4, fig. 4 shows a schematic diagram of time series data and associated data. As shown in FIG. 4, the time t is taken as the horizontal axis, and the data Y is usedt) is the vertical axis, and time-series data (solid line shown in fig. 4) as shown in fig. 4 can be constructed. Taking the hysteresis value i as the transverse axis, the correlation coefficient r i For the vertical axis, association data as shown in fig. 4 (as shown by the broken line in fig. 4) may be constructed. As can be seen from fig. 4, the associated data has periodic spike discrete data, such as the associated data having i=n×t i Where there is a glitch discrete data, n is a positive integer (i.e., the period of the glitch discrete data is T i )。
The moving average algorithm model is a model for sequentially calculating a sequential time average value containing a certain number of terms according to time sequence data in a gradual transition mode so as to reflect long-term trend. Assume that the time-series data is y= [ Y 1 ,...Y t ]If a moving average of any time stamp a is considered, that is, an average of usage data in a window from the time stamp a is considered, it is described by a mathematical formula:
wherein w is the window size, and w is more than or equal to 1.M is M w (a) The time window from the time stamp a has a size w, which is the average value of the usage data within the time window.
Let the window value l>s≥1,M s (n)-M l (n)>0, which indicates that a long line is penetrated on a short line, and time sequence data has a rising trend; m is M s (n)-M l (n)<0 represents a long line passing under the short line, and the time series data has a tendency of falling; namely when M s (n)-M l (n)>0 or M s (n)-M l (n)<At 0, the dynamic change characteristic of the time series data includes a monotone characteristic. M is M s (n)-M l (n) ≡0 indicates that the time-series data is in a stationary state, and the dynamically changing feature of the time-series data includes a stationary feature. Note that the short line refers to a moving average line corresponding to the window value s, and the long line refers to a moving average line corresponding to the window value l.
S203: and calling a target prediction model matched with the dynamic change characteristics in at least one prediction model to predict the time sequence data, so as to obtain the predicted usage amount of the target resource in the next service period.
In one embodiment, the resource management device may divide the traffic cycle for the traffic system so as to allocate different resources (i.e., perform resource management operations) in different traffic cycles.
Wherein the time of the next service period is later than the time of the preset time period. The method and the device can predict the predicted usage of the next service period through the historical usage data in the preset time period. It should be noted that, when the service period is sufficiently small, the predicted usage amount can be obtained in real time. Further, the preset time period may include P service periods, where P is a positive integer. For example, the preset time period may be the 1 st to the P-th historical service periods, and the predicted usage of the p+1st service period may be predicted by the historical usage data of the target resource in the 1 st to the P-th historical service periods. For another example, the preset time period may be from the 1 st historical service period to the P-th historical service period, and the predicted usage of the Q-th service period may be predicted by the historical usage data of the target resource from the 1 st historical service period to the P-th historical service period, where P < Q, P, Q are positive integers.
As can be seen from the foregoing S202, the dynamic change features include one or more of periodic features, monotonic features, and stationary features. Different target prediction models that match different dynamically changing features are described in detail below.
In one embodiment, if the dynamically changing features include periodic features and monotonic features (i.e., the dynamically changing features include periodic features and monotonic features, not including stationary features), a Long-short term memory recurrent neural network (Long-Short Term Memory, LSTM) model or a limit gradient lifting algorithm (eXtreme Gradient Boosting, xgboost) model in the at least one predictive model is determined as the target predictive model.
The LSTM model is a variant of a cyclic neural network (Recurrent Neural Network, RNN) model, and overcomes the defect of the traditional RNN model on a long-term memory information processing mode. In the embodiment of the present application, when determining that the LSTM model in the at least one prediction model is the target prediction model, step 203 specifically includes: and acquiring the characteristic data of the time sequence data, and calling the LSTM model to predict the characteristic data to obtain the predicted usage amount of the target resource in the next service period. Optionally, the feature data of the time series data may include historical usage data of P service periods, and then the LSTM model may be invoked to predict the historical usage data of P service periods to obtain a predicted usage of the target resource in the p+1th service period.
The xgboost model can convert the prediction problem of time series data into a supervised learning regression problem. An xgbregress model of xgboost models may be employed. In the embodiment of the present application, when determining that the xgboost model in the at least one prediction model is the target prediction model, step 203 specifically includes: and acquiring characteristic data of the time sequence data, and calling an XGBRegresor model in the xgboost model to predict the characteristic data to obtain the predicted usage amount of the target resource in the next service period. Alternatively, the feature data may comprise a sequence feature. The sequence features may include, but are not limited to, one or more of sequence statistics (e.g., maximum/minimum, average, median, variance, standard deviation, co-ratio, loop ratio, autocorrelation coefficients, etc.), sequence fit features (e.g., moving average algorithm features, autoregressive features, learning features, etc.), and sequence classification features (e.g., wavelet analysis features, numerical analysis features, entropy features, etc.).
In another embodiment, if the dynamically changing features include periodic features (i.e., the dynamically changing features include periodic features, excluding stationary features and monotonic features), then a same-loop-ratio regular model or a differentially integrated moving average autoregressive (Autoregressive Integrated Moving Average, ARIMA) model of the at least one predictive model is determined as the target predictive model.
The same-ring ratio rule model is a rule model for determining the predicted usage of the next service period based on the same ratio or the ring ratio value and the usage of the current service period. The predicted usage of the next traffic cycle can be determined by the following formula:Y K+1 =Y K * (1+z) wherein Y K The usage amount of the Kth service period can be referred to, Z can be the same-ring value or ring ratio, and can be positive number or negative number. Y is Y K+1 The predicted usage of the k+1th service period may be referred to, where K is a positive integer.
For example, using the above formula, when the equivalence ratio increases by 30%, it can be determined that the predicted usage of the next service period is 1.3 times the usage of the current service period; when the ring ratio is 30% down, it can be determined that the predicted usage of the next service period is 0.7 of the usage of the current service period.
In this embodiment, when determining that the same-loop ratio rule model in the at least one prediction model is the target prediction model, step 203 specifically includes: and acquiring the characteristic data of the time sequence data, and calling the same-ring-ratio rule model to predict the characteristic data to obtain the predicted usage amount of the target resource in the next service period. Optionally, the feature data of the time series data may include a usage amount of a kth service period, and then the same-ring-ratio rule model may be called to predict the usage amount of the kth service period, so as to obtain a predicted usage amount of the target resource in the kth+1th service period.
Wherein the ARIMA model is a model for predicting time-series data. The ARIMA model can be described by the following formula:wherein Y is t Delta as time series data d Y t Represents Y t Stationary time series data after d times of differentiation epsilon t-1 White noise random error sequence with zero mean value phi m (m=1, 2 … p) and θ n (m=1, 2 … p) is the parameter to be estimated of the ARIMA model, and p and q are the orders of the model. Note that ARIMA model may be expressed as ARIMA (p, d, q).
In the embodiment of the present application, when determining that the ARIMA model in the at least one prediction model is the target prediction model, step 203 specifically includes: and acquiring the characteristic data of the time sequence data, and calling an ARIMA model to predict the characteristic data to obtain the predicted usage amount of the target resource in the next service period. Optionally, the feature data of the time series data may include a usage amount of a preset time period, and then the ARIMA model may be called to predict the usage amount of the preset time period, so as to obtain a predicted usage amount of the target resource in a next service period.
In yet another embodiment, an exponentially weighted moving average (Exponentially Weighted Moving Averages, EWMA) model of the at least one predictive model is determined to be the target predictive model if the dynamically changing feature comprises a monotonic feature (i.e., the dynamically changing feature comprises a monotonic feature, excluding periodic and stationary features).
The EWMA is a method for respectively giving different weights to EWMA values, obtaining a moving average value according to the different weights, and determining the predicted usage amount based on the final moving average value. An exponential weighted moving average model is adopted, because the recent EWMA value has a larger influence on the predicted usage amount, and the trend of the recent resource usage amount can be reflected more; specifically, the predicted usage of the next traffic cycle may be determined by the following expression: v (V) k+1 =βV k +(1-β)Y k Wherein Y is k And K is a positive integer for predicting the usage amount of the Kth service period. The coefficient beta represents the rate of weight decrease, the smaller the value, the faster the rate of decrease, 0<β<1,V k EWMA value, V for historical service period k+1 The EWMA value for the next service period. For example, when β=0.9, V k+1 =0.9V k +0.1Y k+1 The method comprises the steps of carrying out a first treatment on the surface of the When β=0.98, V k+1 =0.98V k +0.02Y k+1 。
In the embodiment of the present application, when determining that the EWMA model in the at least one prediction model is the target prediction model, step 203 specifically includes: and acquiring the characteristic data of the time sequence data, and calling an EWMA model to predict the characteristic data to obtain the predicted usage amount of the target resource in the next service period. Optionally, the feature data of the time series data may include a usage amount of a kth service period, and then the EWMA model may be invoked to determine an EWMA value of a kth+1th service period based on the usage amount of the kth service period and the EWMA value of the kth service period, so that a predicted usage amount of the target resource in the kth+1th service period may be determined.
In yet another embodiment, if the dynamically changing features include stationary features (i.e., the dynamically changing features include stationary features, excluding periodic features and monotonic features), a three sigma criterion (3 sigma) model of the at least one predictive model is determined as the target predictive model. In the 3sigma model, the probability that the predicted usage value is distributed in (u-3 delta, u+3 delta) is 0.9974, so that any value in (u-3 delta, u+3 delta) can be used as the predicted usage value of the next service period. Where u is the average value of time series data, and δ is the standard deviation of time series data. In the embodiment of the present application, when determining that the 3sigma model in the at least one prediction model is the target prediction model, step 203 specifically includes: and acquiring the characteristic data of the time sequence data, and calling a 3sigma model to predict the characteristic data to obtain the predicted usage amount of the target resource in the next service period. Optionally, the characteristic data includes an average value and a standard deviation of the time series data.
It is clear that before the target prediction model matched with the dynamic change characteristics in at least one prediction model is called to perform prediction processing on the time series data to obtain the predicted usage amount of the target resource in the next service period, the initialized neural network is required to be trained to obtain the prediction model. The method specifically comprises the following steps:
S11: acquiring sample sequence data; the sample sequence data comprises the use data of the target resource in each service period of the H service periods; h is an integer greater than 1.
Optionally, the description of the sample sequence data is similar to the time sequence data in step S201, and will not be repeated here.
S12: the method comprises the steps of determining a training sample based on the use data of a target resource in at least one service period in H service periods, and determining the supervision information of the training sample based on the use data of other service periods except at least one service period in the H service periods.
That is, the training samples include usage data for a first number of service periods and the supervision information includes usage data for a second number of service periods in the sample sequence data that is time after the first number of service periods. For example, if the sample sequence data is collected with daily usage data of 30 days, that is, 1 day from 2 months to 1 day from 3 months, the sample sequence data may be divided into a single service period per day, and the usage data of the first 20 days (1 day from 2 months to 20 days from 2 months) may be used as training samples, and the usage data of the last 10 days (1 day from 2 months to 21 days from 3 months) may be used as supervision information corresponding to the training samples. In general, the first number is larger than the second number, and in the above example, the first number corresponds to the first 20 days, and the second number corresponds to the second 10 days.
S13: acquiring characteristic data of a training sample; the characteristic data comprises one or two of sequence characteristics and usage data of a target resource in at least one service period of the H service periods, and the initialized neural network is trained according to one or two of the characteristic data and the supervision information to obtain a prediction model.
In an alternative implementation manner, for an initialized LSTM model, usage data of a target resource in at least one service period of the H service periods may be obtained, the initialized LSTM model is called to predict the usage data of the target resource in any service period of the at least one service period, so as to obtain a predicted usage amount of the target resource in a next service period, a loss value is determined according to the predicted usage amount and the usage amount in the supervision information, and parameters of the initialized LSTM model are updated based on the loss value, so as to obtain a trained LSTM model, that is, a prediction model.
In another alternative embodiment, for the initialized xgboost model, the sequence feature of the training sample may be obtained, the initialized xgboost model is called to perform prediction processing on the sequence feature, so as to obtain the predicted usage of the target resource in the next service period, a loss value is determined according to the predicted usage and the usage in the supervision information, and the parameters of the initialized xgboost model are updated based on the loss value, so as to obtain the trained xgboost model, that is, the predicted model.
In still another alternative embodiment, for the initialized ARIMA model, the usage data of the target resource in at least one service period of the H service periods may be obtained, and the d-level difference processing is performed on the usage data of the target resource in at least one service period of the H service periods to obtain the stable data. The stationary data is then processed to obtain an Autocorrelation Coefficient (ACF) and a Partial Autocorrelation Coefficient (PACF). And finally, analyzing the ACF constructed autocorrelation diagram and the PACF constructed partial autocorrelation diagram to obtain the optimal orders p and q, thereby obtaining the ARIMA model.
S204: a resource management operation is performed for the target resource based on the predicted usage.
In a possible implementation, the resource management device may perform a capacity expansion operation or a capacity contraction operation for the target resource based on the predicted usage amount.
In the embodiment of the application, the resource management device acquires time sequence data of the target resource, performs feature extraction on the time sequence data to obtain dynamic change features, and then calls a target prediction model matched with the dynamic change features in at least one prediction model to predict the time sequence data to obtain the predicted usage amount of the target resource in the next service period. The resource management method can predict the predicted use amount of the next service period based on the historical use data included in the time sequence data, can predict the use condition of the resources in advance and predict the resources in real time, so that excessive allocated resources or insufficient allocated resources can be avoided, the cost can be reduced, and the operation quality is improved. In addition, the method and the device call the target prediction model matched with the dynamic change characteristics of the time sequence characteristics in at least one prediction model to predict the time sequence data, namely the resource management equipment can adaptively select the matched target prediction model from the at least one prediction model based on the dynamic change characteristics, and the at least one prediction model can be suitable for the time sequence data with different dynamic change characteristics, so that the flexibility and the compatibility of the prediction model are effectively improved. In addition, the target prediction model is matched with the dynamic change characteristics, and the predicted usage amount of the next service period predicted by the target prediction model is more accurate, so that excessive allocated resources or insufficient allocated resources can be avoided, the cost can be reduced, and the operation quality can be improved.
As can be seen from the above description of the method embodiment shown in fig. 2, in the resource management method shown in fig. 2, the resource management device may perform a resource management operation on the target resource based on the predicted usage amount. Referring to fig. 5, fig. 5 is a flow chart illustrating another resource management method, and the embodiment illustrated in fig. 5 differs from the embodiment illustrated in fig. 2 in that the embodiment illustrated in fig. 5 details how resource management operations are performed for a target resource based on predicted usage. As shown in fig. 5, the resource management method may include S501-S505:
s501: time-series data of the target resource is acquired, wherein the time-series data comprises historical use data of the target resource at each time point of a preset time period.
S502: extracting features of the time sequence data to obtain dynamic change features; the dynamically changing features include one or more of periodic features, monotonic features, and stationary features.
S503: and calling a target prediction model matched with the dynamic change characteristics in at least one prediction model to predict the time sequence data, so as to obtain the predicted usage amount of the target resource in the next service period.
It should be noted that, steps S501 to S503 may refer to related embodiments of S201 to S203, and will not be described again.
S504: acquiring current use data of a target resource; the current usage data includes a usage amount and a remaining amount in the allocated resource amount, and when one or more of the current usage data and the predicted usage amount satisfies a capacity expansion condition, it is determined to perform a capacity expansion operation for the target resource.
Optionally, the current usage data includes a usage amount and a remaining amount in the allocated resource amount, for example, the current usage data of the memory is: the amount of allocated resources was 4000MB, the amount of usage was 2000MB, and the remaining amount was 2000MB.
In a possible embodiment, it is determined to perform the expansion operation for the target resource when one or more of the current usage data and the predicted usage amount meets one or more of the following expansion conditions. Optionally, the capacity expansion condition includes: (1) the usage is greater than a first threshold. The usage rate refers to the ratio of the usage amount to the amount of allocated resources. (2) The usage is greater than a first threshold and the remaining amount in the allocated resource amount is less than a second threshold. (3) the allocated resource amount is smaller than the predicted usage amount. (4) the predicted usage is greater than a third threshold.
In an alternative embodiment, it may also be determined that the scaling operation is performed for the target resource when one or more of the current usage data and the predicted usage meet one or more of the following scaling conditions. Alternatively, the shrinkage condition may include: (1) the usage is less than a fourth threshold. (2) The usage is less than a first threshold and the remaining amount in the allocated resource amount is greater than a fifth threshold. (3) allocating a larger amount of resources than the predicted usage. (4) the predicted usage is less than a sixth threshold.
It should be noted that the first threshold, the second threshold, the third threshold, the fourth threshold, the fifth threshold, and the sixth threshold may be set according to business requirements or experience.
In an alternative embodiment, the resource management device may further output alert information for the target resource when one or more of the current usage data and the predicted usage amount satisfy one or more of the expansion conditions.
S505: and executing the capacity expansion operation for the target resource.
In a possible implementation manner, the resource management device may determine the predicted demand of the next service period based on the predicted usage of the next service period and the current usage data, take the predicted demand of the next service period as the resource expansion capacity, and perform the resource management operation on the target resource based on the resource expansion capacity. For example, when the resource expansion capacity is greater than a preset value, it is determined to perform an expansion operation for the target resource. And when the expansion capacity of the resource is smaller than a preset value, determining to execute the capacity shrinking operation aiming at the target resource.
Optionally, determining the predicted demand for the next service period based on the predicted usage for the next service period and the current usage data includes: and taking the predicted usage of the next service period as the predicted demand of the next service period.
Optionally, determining the predicted demand for the next service period based on the predicted usage for the next service period and the current usage data includes: the difference between the predicted usage of the next service period and the usage in the allocated resource amount is taken as the predicted demand of the next service period.
Optionally, determining the predicted demand of the next service period based on the predicted usage of the next service period and the current usage data may further include: and taking the difference between the predicted usage of the next service period and the allocated resource amount as the predicted demand of the next service period.
In another possible embodiment, the resource expansion may be determined in combination with the traffic constraint pre-estimate for the next traffic cycle, the remaining superscalar of the superscalar, and the predicted demand for the next traffic cycle, so as to perform a resource management operation with respect to the target resource based on the resource expansion. As shown in fig. 6:
601. and determining the residual excess corresponding to the residual quantity based on the excess ratio example.
Optionally, for 601, the superdivision pool may support superdivision of physical resources, so that the utilization rate of the physical resources may be effectively improved. For example, the physical resources can be reasonably oversubscribed, the resources allocated to the service systems are larger than the physical resources actually owned by the service systems, some service systems utilize the physical resources in the daytime, and some service systems utilize the physical resources at night, so that the physical resources can be fully utilized. In the embodiment of the present application, in order to effectively improve the utilization rate of the remaining resources, the remaining amount in the allocated resource amount may be subjected to an superprocessing to obtain a remaining superparameter, that is, the remaining superparameter corresponding to the remaining amount is determined based on the superparameter. For example, when the remaining amount of the memory is B and the superdivision ratio is 1:2, the remaining superamount corresponding to the remaining amount may be determined to be 2B based on the superdivision ratio. It should be noted that the remaining excess in the superdivision pool is not used, and when the capacity expansion of the resources is calculated, the remaining excess can be subtracted, so that the resources in the superdivision pool can be fully utilized, and the excessive capacity expansion is avoided.
602. And acquiring the service constraint pre-estimation of the target resource in the next service period.
Optionally, for 602, when constraint conditions such as service growth planning or key performance indicators (Key Performance Indicator, KPI) exist in the service system, the resource requirement of the service system is changed greatly, so that when performing the capacity expansion operation, the service constraint pre-estimation corresponding to the constraint conditions needs to be considered. Specifically, the resource management device may acquire a service constraint condition of a next service period, and determine a service constraint pre-estimation amount of the target resource in the next service period based on the allocated resource amount of the target resource and the service constraint condition. For example, when the constraint condition of the service system is that the service is increased by 2 times of the current service period, if the allocated resource amount of the target resource of the current service period is S, the service constraint pre-estimation amount of the next service period is 2S.
603. The predicted demand for the next business cycle is determined based on the predicted usage for the next business cycle and the current usage data.
Optionally, for 603, when calculating the capacity expansion of the resource, the predicted demand needs to be considered, so that the resource demand of the service system can be fully satisfied, and the resource deficiency is avoided.
604. And determining the resource expansion capacity of the target resource according to the business constraint pre-estimation quantity, the residual excess quantity and the predicted demand quantity.
Optionally, for 604, specific including: and acquiring the total demand quantity of the target resource in the next service period based on the service constraint pre-estimation and the predicted demand quantity, and acquiring the resource expansion capacity of the target resource based on the difference value between the total demand quantity and the residual excess quantity and the excess ratio example.
In an alternative embodiment, the resource expansion capacity may be determined by the following formula: (traffic constraint pre-measure + predicted demand-excess) super-division ratio.
For example, when the expanded resources can also be subjected to the super-processing, the super-division ratio is 1:2, the service constraint pre-estimation amount is 2S, the predicted demand amount is A, and when the remaining super-division amount is 2B,
for another example, when the expanded resources are not subjected to the super-division process, then the super-division ratio is 1:1, and the expanded capacity of the resources=2s+a-2B.
In a possible implementation manner, the corresponding resource management operation may be performed according to the size relationship between the resource expansion capacity and the preset value. Specifically, when the capacity expansion of the resource is greater than a preset value (e.g., 2s+a-2b > 0), the resource management device determines that the total required amount of the next service period is greater than the remaining excess amount, and performs the capacity expansion operation on the target resource based on the capacity expansion of the resource. When the capacity of the resource expansion is equal to a preset value (e.g., 2s+a-2b=0), the resource management device determines that the total required amount of the next service period is equal to the remaining excess amount, and performs the excess processing on all the remaining amounts without performing the capacity expansion operation for the target resource. When the capacity of the resource expansion is smaller than a preset value (such as 2s+a-2b < 0), the resource management device determines that the total required amount of the next service period is smaller than the remaining excess amount, and the remaining amount is not processed in an excess manner or is processed in a partial excess manner, without executing the capacity expansion operation for the target resource.
Furthermore, when the capacity expansion operation for the target resource is not required to be executed, the capacity reduction operation for the target resource can be considered to be executed, so that the resource can be effectively saved, and the resource waste is avoided. For example, when the resource expansion capacity is equal to a preset value (e.g., 2s+a-2b=0), the excess fraction of the remaining amount may be raised, and the excess remaining amount may be released. For example, updating the superdivision ratio from 1:2 to 1:4, determining that the remaining superdivision amount corresponding to the B/2 remaining amount is 2B based on the superdivision ratio of 1:4, and performing the capacity reduction operation on the target resource of the other B/2 remaining amount (i.e. releasing the target resource of the portion).
605. And performing a resource management operation on the target resource based on the resource capacity expansion.
In the embodiment of the application, the super-division pool is considered, so that the resources can be super-divided, the utilization rate of the resources is effectively improved, and the capacity expansion cost is reduced. In addition, the service constraint pre-estimation of the next service period is considered, the service condition of the service growth dimension is considered, and the resource deficiency is avoided. In addition, the embodiment of the application combines the service constraint pre-estimation of the next service period, the residual excess of the superdivision pool and the predicted demand of the next service period to determine the resource expansion capacity, fully considers the relevant factors and scene demands of the resource management scene, and can meet the actual demands of a service system. Excessive or insufficient expanded resources can be avoided, the utilization rate of the resources can be effectively improved, the cost is reduced, and the operation quality is improved.
Based on the above description of the embodiments of the resource management method, the embodiments of the present application also disclose a resource management device, which may be a computer program (including program code) running in the above-mentioned resource management apparatus. The resource management device may perform the method shown in fig. 2 or fig. 5.
Referring to fig. 7, the resource management device may operate the following units:
an acquisition unit 701 for acquiring time-series data of a target resource, the time-series data including historical usage data of the target resource at each point in time of a preset period;
a feature extraction unit 702, configured to perform feature extraction on the time-series data to obtain a dynamic change feature; the dynamically changing features include one or more of periodic features, monotonic features, and stationary features;
a prediction processing unit 703, configured to invoke a target prediction model matched with the dynamic change feature in the at least one prediction model to perform prediction processing on the time sequence data, so as to obtain a predicted usage amount of the target resource in a next service period;
a processing unit 704 for performing a resource management operation for the target resource based on the predicted usage.
In one embodiment, the resource management operations include capacity expansion operations, and the processing unit 704 is configured to perform the resource management operations for the target resource based on the predicted usage, including:
Acquiring current use data of a target resource; the current usage data includes a usage amount and a remaining amount in the allocated resource amount;
when one or more of the current usage data and the predicted usage amount satisfies the capacity expansion condition, a capacity expansion operation is performed with respect to the target resource.
In yet another embodiment, the processing unit 704 is configured to perform a capacity expansion operation for a target resource, including:
determining a predicted demand in a next service period based on the current usage data and the predicted usage;
determining the residual excess corresponding to the residual quantity based on the excess proportion;
acquiring a business constraint pre-estimation value of a target resource in a next business period, and determining the resource expansion capacity of the target resource according to the business constraint pre-estimation value, the residual excess and the predicted demand;
and if the resource expansion capacity is larger than the preset value, executing expansion operation on the target resource based on the resource expansion capacity.
In yet another embodiment, the processing unit 704 is configured to determine a resource expansion capacity of the target resource according to the traffic constraint pre-estimation, the remaining excess and the predicted demand, including:
acquiring the total demand of the target resource in the next service period based on the service constraint pre-estimation and the predicted demand;
And obtaining the resource expansion capacity of the target resource based on the difference value between the total demand and the residual excess and the excess ratio example.
In yet another embodiment, the processing unit 704 is configured to obtain a traffic constraint forecast of the target resource in a next traffic cycle, including:
acquiring a service constraint condition of the next service period;
and determining the service constraint pre-estimation of the target resource in the next service period based on the allocated resource quantity of the target resource and the service constraint condition.
In yet another embodiment, the prediction processing unit 703 is further configured to:
and if the dynamic change features comprise periodic features and monotonic features, determining a long-short-period memory cyclic neural network model or a limit gradient lifting algorithm model in at least one prediction model as a target prediction model.
If the dynamic change features comprise periodic features, determining a same-loop ratio rule model or a differential integration moving average autoregressive model in at least one prediction model as a target prediction model;
if the dynamic change features comprise monotonic features, determining an exponentially weighted moving average model in at least one prediction model as a target prediction model;
and if the dynamic change feature comprises a stable feature, determining a three-sigma criterion model in at least one prediction model as a target prediction model.
In yet another embodiment, the prediction processing unit 703 is further configured to:
acquiring sample sequence data; the sample sequence data comprises the use data of the target resource in each service period of the H service periods; h is an integer greater than 1;
determining a training sample based on the use data of the target resource in at least one service period in the H service periods, and determining the supervision information of the training sample based on the use data of the target resource in other service periods except for the at least one service period in the H service periods;
acquiring characteristic data of a training sample; the characteristic data comprises one or two of sequence characteristics and usage data of a target resource in at least one service period in H service periods;
and training the initialized neural network according to one or two of the characteristic data and the supervision information to obtain a prediction model.
In yet another embodiment, the feature extraction unit 702 is configured to perform feature extraction on time-series data to obtain a dynamically changing feature, and includes:
invoking a period detection model to process the time sequence data and determining whether the dynamic change characteristics comprise periodicity;
and calling a moving average algorithm model to process the time sequence data, and determining whether the dynamic change characteristics comprise stationarity or monotonicity.
In still another embodiment, the period detection model includes a similarity period detection model, and the feature extraction unit 702 is configured to invoke the period detection model to process the time-series data to determine whether the dynamic change feature includes periodicity, and includes:
invoking a similarity period detection model to process the time sequence data and determining frequency domain sequence data;
when the frequency domain sequence data has periodic spike discrete data, determining the dynamic change characteristic includes periodicity.
In yet another embodiment, the period detection model includes an autocorrelation period detection model, and the feature extraction unit 702 is configured to invoke the period detection model to process the time-series data to determine whether the dynamic change feature includes periodicity, including:
calling an autocorrelation period detection model to process the time sequence data to obtain associated data; the associated data comprises a corresponding relation between a hysteresis value i and a correlation coefficient; the correlation coefficient is a correlation coefficient between time series data and lag series data; the hysteresis sequence data comprises usage data of a target resource lagged by i time points; i is a positive integer;
the determining the dynamically changing feature includes periodicity when the associated data has periodic spike discrete data.
According to one embodiment of the present application, each of the steps involved in the method shown in fig. 2 or fig. 5 may be performed by each unit in the resource management device shown in fig. 7. According to another embodiment of the present application, each unit in the resource management device shown in fig. 7 may be separately or completely combined into one or several other units, or some (some) units may be further split into a plurality of units with smaller functions to form a unit, which may achieve the same operation without affecting the implementation of the technical effects of the embodiments of the present application. The above units are divided based on logic functions, and in practical applications, the functions of one unit may be implemented by a plurality of units, or the functions of a plurality of units may be implemented by one unit. In other embodiments of the present application, the resource-based management device may also include other units, and in practical applications, these functions may also be implemented with assistance from other units, and may be implemented by cooperation of multiple units.
According to another embodiment of the present application, the processing elements and storage elements may be implemented by including a central processing unit (Central Processing Unit, CPU), a random access storage medium (RAM), a read only storage medium (ROM), and the like. A general-purpose computing device, such as a computer, runs a computer program (including program code) capable of executing steps involved in the respective methods as shown in fig. 2 or 5 to construct a resource management apparatus as shown in fig. 7, and to implement the resource management methods of the embodiments of the present application. The computer program of (2) may be recorded on, for example, a computer-readable recording medium, and loaded into and run in the above-described resource management device through the computer-readable recording medium.
In the embodiment of the application, the resource management device acquires time sequence data of the target resource, performs feature extraction on the time sequence data to obtain dynamic change features, and then calls a target prediction model matched with the dynamic change features in at least one prediction model to predict the time sequence data to obtain the predicted usage amount of the target resource in the next service period. The resource management method can predict the predicted use amount of the next service period based on the historical use data included in the time sequence data, can predict the use condition of the resources in advance and predict the resources in real time, so that excessive allocated resources or insufficient allocated resources can be avoided, the cost can be reduced, and the operation quality is improved. In addition, the method and the device call the target prediction model matched with the dynamic change characteristics of the time sequence characteristics in at least one prediction model to predict the time sequence data, namely the resource management device can adaptively select the matched target prediction model from the at least one prediction model based on the dynamic change characteristics, and the at least one prediction model can be suitable for the time sequence data with different dynamic change characteristics, so that the flexibility and the compatibility of the prediction model are effectively improved. In addition, the target prediction model is matched with the dynamic change characteristics, and the predicted usage amount of the next service period predicted by the target prediction model is more accurate, so that excessive allocated resources or insufficient allocated resources can be avoided, the cost can be reduced, and the operation quality can be improved.
Based on the description of the embodiments of the resource management method, the embodiments of the present application also disclose a resource management device. Referring to fig. 8, the resource management device includes at least a processor 801, an input interface 802, an output interface 803, and a computer storage medium 804, which may be connected by a bus or other means.
The computer storage media 804 is a memory device in the resource management device for storing programs and data. It will be appreciated that the computer storage media 804 herein may include both built-in storage media for the resource management device and extended storage media supported by the resource management device. The computer storage media 804 provides storage space that stores the operating system of the resource management device. Also stored in this memory space are one or more instructions, which may be one or more computer programs (including program code), adapted to be loaded and executed by the processor 801. Note that the computer storage medium herein may be a high-speed RAM memory; optionally, it may also be at least one computer storage medium remote from the foregoing processor, where the processor may be referred to as a central processing unit (Central Processing Unit, CPU), is a core of the resource management device and a control center, and is adapted to be implemented with one or more instructions, and specifically load and execute the one or more instructions to implement the corresponding method flow or function.
In one embodiment, one or more instructions stored in computer storage medium 804 may be loaded and executed by processor 801 to implement the steps involved in performing the corresponding method as shown in fig. 2 or 5, in a specific implementation, one or more instructions in computer storage medium 804 are loaded and executed by processor 801 to:
acquiring time sequence data of a target resource, wherein the time sequence data comprises historical use data of the target resource at each time point of a preset time period;
extracting features of the time sequence data to obtain dynamic change features; the dynamically changing features include one or more of periodic features, monotonic features, and stationary features;
invoking a target prediction model matched with the dynamic change characteristics in at least one prediction model to predict the time sequence data to obtain the predicted usage amount of the target resource in the next service period;
a resource management operation is performed for the target resource based on the predicted usage.
In one embodiment, the resource management operations include capacity expansion operations, and the processor 801 is configured to perform the resource management operations for the target resource based on the predicted usage, including:
Acquiring current use data of a target resource; the current usage data includes a usage amount and a remaining amount in the allocated resource amount;
when one or more of the current usage data and the predicted usage amount satisfies the capacity expansion condition, a capacity expansion operation is performed with respect to the target resource.
In yet another embodiment, the processor 801 is configured to perform a capacity expansion operation for a target resource, including:
determining a predicted demand in a next service period based on the current usage data and the predicted usage;
determining the residual excess corresponding to the residual quantity based on the excess proportion;
acquiring a business constraint pre-estimation value of a target resource in a next business period, and determining the resource expansion capacity of the target resource according to the business constraint pre-estimation value, the residual excess and the predicted demand;
and if the resource expansion capacity is larger than the preset value, executing expansion operation on the target resource based on the resource expansion capacity.
In yet another embodiment, the processor 801 is configured to determine a resource expansion capacity of the target resource according to the traffic constraint pre-estimate, the remaining excess, and the predicted demand, including:
acquiring the total demand of the target resource in the next service period based on the service constraint pre-estimation and the predicted demand;
And obtaining the resource expansion capacity of the target resource based on the difference value between the total demand and the residual excess and the excess ratio example.
In yet another embodiment, the processor 801 is configured to obtain a traffic constraint forecast of a target resource in a next traffic cycle, including:
acquiring a service constraint condition of the next service period;
and determining the service constraint pre-estimation of the target resource in the next service period based on the allocated resource quantity of the target resource and the service constraint condition.
In yet another embodiment, the processor 801 is further configured to:
and if the dynamic change features comprise periodic features and monotonic features, determining a long-short-period memory cyclic neural network model or a limit gradient lifting algorithm model in at least one prediction model as a target prediction model.
If the dynamic change features comprise periodic features, determining a same-loop ratio rule model or a differential integration moving average autoregressive model in at least one prediction model as a target prediction model;
if the dynamic change features comprise monotonic features, determining an exponentially weighted moving average model in at least one prediction model as a target prediction model;
and if the dynamic change feature comprises a stable feature, determining a three-sigma criterion model in at least one prediction model as a target prediction model.
In yet another embodiment, the processor 801 is further configured to:
acquiring sample sequence data; the sample sequence data comprises the use data of the target resource in each service period of the H service periods; h is an integer greater than 1;
determining a training sample based on the use data of the target resource in at least one service period in the H service periods, and determining the supervision information of the training sample based on the use data of the target resource in other service periods except for the at least one service period in the H service periods;
acquiring characteristic data of a training sample; the characteristic data comprises one or two of sequence characteristics and usage data of a target resource in at least one service period in H service periods;
and training the initialized neural network according to one or two of the characteristic data and the supervision information to obtain a prediction model.
In yet another embodiment, the processor 801 is configured to perform feature extraction on time series data to obtain a dynamic change feature, and the method includes:
invoking a period detection model to process the time sequence data and determining whether the dynamic change characteristics comprise periodicity;
and calling a moving average algorithm model to process the time sequence data, and determining whether the dynamic change characteristics comprise stationarity or monotonicity.
In yet another embodiment, the period detection model includes a similarity period detection model, and the processor 801 is configured to invoke the period detection model to process the time series data to determine whether the dynamic change feature includes periodicity, including:
invoking a similarity period detection model to process the time sequence data and determining frequency domain sequence data;
when the frequency domain sequence data has periodic spike discrete data, determining the dynamic change characteristic includes periodicity.
In yet another embodiment, the period detection model includes an autocorrelation period detection model, and the processor 801 is configured to invoke the period detection model to process the time series data to determine whether the dynamic change feature includes periodicity, including:
calling an autocorrelation period detection model to process the time sequence data to obtain associated data; the associated data comprises a corresponding relation between a hysteresis value i and a correlation coefficient; the correlation coefficient is a correlation coefficient between time series data and lag series data; the hysteresis sequence data comprises usage data of a target resource lagged by i time points; i is a positive integer;
the determining the dynamically changing feature includes periodicity when the associated data has periodic spike discrete data.
In the embodiment of the application, the resource management device acquires time sequence data of the target resource, performs feature extraction on the time sequence data to obtain dynamic change features, and then calls a target prediction model matched with the dynamic change features in at least one prediction model to predict the time sequence data to obtain the predicted usage amount of the target resource in the next service period. The resource management method can predict the predicted use amount of the next service period based on the historical use data included in the time sequence data, can predict the use condition of the resources in advance and predict the resources in real time, so that excessive allocated resources or insufficient allocated resources can be avoided, the cost can be reduced, and the operation quality is improved. In addition, the method and the device call the target prediction model matched with the dynamic change characteristics of the time sequence characteristics in at least one prediction model to predict the time sequence data, namely the resource management equipment can adaptively select the matched target prediction model from the at least one prediction model based on the dynamic change characteristics, and the at least one prediction model can be suitable for the time sequence data with different dynamic change characteristics, so that the flexibility and the compatibility of the prediction model are effectively improved. In addition, the target prediction model is matched with the dynamic change characteristics, and the predicted usage amount of the next service period predicted by the target prediction model is more accurate, so that excessive allocated resources or insufficient allocated resources can be avoided, the cost can be reduced, and the operation quality can be improved.
Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium and executes the computer instructions to cause the computer device to perform the steps performed in fig. 2 or fig. 5 of the above-described resource management method embodiment.
A computer-readable storage medium is also provided in an embodiment of the present application. The computer readable storage medium stores a computer program comprising program instructions that, when executed by a processor, perform the steps performed in fig. 2 or fig. 5 of the above-described resource management method embodiments.
Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored on a computer readable storage medium, which when executed may comprise the steps of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), or the like.
The foregoing disclosure is only a preferred embodiment of the present application, and it is not intended to limit the scope of the claims, and one of ordinary skill in the art will understand that all or part of the processes for implementing the embodiments described above may be performed with equivalent changes in the claims of the present application and still fall within the scope of the claims.
Claims (13)
1. A method of resource management, the method comprising:
acquiring time sequence data of a target resource, wherein the time sequence data comprises historical use data of the target resource at each time point of a preset time period;
extracting the characteristics of the time sequence data to obtain dynamic change characteristics; the dynamically changing features include one or more of periodic features, monotonic features, and stationary features;
invoking a target prediction model matched with the dynamic change characteristics in at least one prediction model to predict the time sequence data to obtain the predicted usage amount of the target resource in the next service period;
performing a resource management operation for the target resource based on the predicted usage.
2. The method of claim 1, wherein the resource management operation comprises a capacity expansion operation, the performing a resource management operation for the target resource based on the predicted usage, comprising:
Acquiring current use data of the target resource; the current usage data comprises a usage amount and a remaining amount in the allocated resource amount;
and when one or more of the current usage data and the predicted usage amount meet a capacity expansion condition, performing a capacity expansion operation for the target resource.
3. The method of claim 2, wherein the performing a capacity expansion operation for the target resource comprises:
determining a predicted demand at the next business cycle based on the current usage data and the predicted usage;
determining the residual excess amount corresponding to the residual amount based on the excess ratio example;
acquiring a service constraint pre-estimation amount of the target resource in the next service period, and determining the resource expansion capacity of the target resource according to the service constraint pre-estimation amount, the residual excess amount and the predicted demand;
and if the resource expansion capacity is larger than a preset value, executing expansion operation on the target resource based on the resource expansion capacity.
4. The method of claim 3, wherein said determining the resource expansion capacity of the target resource based on the traffic constraint pre-estimate, the remaining excess amount, and the predicted demand comprises:
Acquiring the total demand of the target resource in the next service period based on the service constraint pre-estimation and the predicted demand;
and obtaining the resource expansion capacity of the target resource based on the difference value between the total demand and the residual excess and the excess proportion.
5. The method of claim 3, wherein the obtaining a traffic constraint pre-estimate for the target resource at the next traffic cycle comprises:
acquiring a service constraint condition of the next service period;
and determining a service constraint pre-estimation of the target resource in the next service period based on the allocated resource quantity of the target resource and the service constraint condition.
6. The method of any one of claims 1-5, further comprising:
if the dynamic change features comprise the periodic features and the monotonic features, determining a long-short-term memory cyclic neural network model or a limit gradient lifting algorithm model in the at least one prediction model as the target prediction model;
if the dynamic change characteristics comprise the periodic characteristics, determining a same-loop-ratio rule model or a differential integration moving average autoregressive model in the at least one prediction model as the target prediction model;
If the dynamic change feature comprises the monotonic feature, determining an exponentially weighted moving average model in the at least one predictive model as the target predictive model;
and if the dynamic change feature comprises the stable feature, determining a three-sigma criterion model in the at least one prediction model as the target prediction model.
7. The method of any one of claims 1-5, further comprising:
acquiring sample sequence data; the sample sequence data comprises the use data of the target resource in each service period of H service periods; h is an integer greater than 1;
determining a training sample based on the use data of the target resource in at least one service period in the H service periods, and determining supervision information of the training sample based on the use data of the target resource in other service periods except the at least one service period in the H service periods;
acquiring characteristic data of the training sample; the characteristic data comprises one or two of sequence characteristics and the use data of the target resource in at least one service period in the H service periods;
And training the initialized neural network according to one or two of the characteristic data and the supervision information to obtain a prediction model.
8. The method according to any one of claims 1 to 5, wherein the feature extraction of the time-series data to obtain a dynamic change feature includes:
invoking a period detection model to process the time sequence data, and determining whether the dynamic change feature comprises the periodicity;
and calling a moving average algorithm model to process the time sequence data, and determining whether the dynamic change characteristics comprise the stationarity or the monotonicity.
9. The method of claim 8, wherein the period detection model comprises a similarity period detection model, and wherein the invoking the period detection model to process the time series data determines whether the dynamically changing feature comprises the periodicity comprises:
invoking the similarity period detection model to process the time sequence data and determining frequency domain sequence data;
when periodic glitch discrete data exists in the frequency domain sequence data, determining that the dynamically changing feature includes the periodicity.
10. The method of claim 8, wherein the period detection model comprises an autocorrelation period detection model, and wherein the invoking the period detection model to process the time series data determines whether the dynamically changing feature comprises the periodicity comprises:
invoking the autocorrelation period detection model to process the time sequence data to obtain associated data; the associated data comprises a corresponding relation between a hysteresis value i and a correlation coefficient; the correlation coefficient is a correlation coefficient between the time series data and the lag series data; the hysteresis sequence data comprises usage data of the target resource that lags by i time points; i is a positive integer;
when the associated data has periodic spike discrete data, determining that the dynamically changing characteristic includes the periodicity.
11. A resource management apparatus, the apparatus comprising:
an acquisition unit configured to acquire time-series data of a target resource, where the time-series data includes historical usage data of the target resource at each time point in a preset time period;
the feature extraction unit is used for extracting features of the time sequence data to obtain dynamic change features; the dynamically changing features include one or more of periodic features, monotonic features, and stationary features;
The prediction processing unit is used for calling a target prediction model matched with the dynamic change characteristics in at least one prediction model to perform prediction processing on the time sequence data so as to obtain the predicted usage amount of the target resource in the next service period;
and the processing unit is used for executing resource management operation on the target resource based on the predicted usage.
12. A resource management device comprising an input interface, an output interface, and further comprising:
a processor adapted to implement one or more instructions; the method comprises the steps of,
a computer storage medium storing one or more instructions adapted to be loaded by the processor and to perform the resource management method of any one of claims 1 to 10.
13. A computer storage medium storing one or more instructions adapted to be loaded by a processor and to perform the resource management method of any one of claims 1 to 10.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210914934.0A CN117544635A (en) | 2022-08-01 | 2022-08-01 | Resource management method, device, equipment and storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210914934.0A CN117544635A (en) | 2022-08-01 | 2022-08-01 | Resource management method, device, equipment and storage medium |
Publications (1)
Publication Number | Publication Date |
---|---|
CN117544635A true CN117544635A (en) | 2024-02-09 |
Family
ID=89788589
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210914934.0A Pending CN117544635A (en) | 2022-08-01 | 2022-08-01 | Resource management method, device, equipment and storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN117544635A (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117992597A (en) * | 2024-04-03 | 2024-05-07 | 江苏微皓智能科技有限公司 | Information feedback method, device, computer equipment and computer storage medium |
-
2022
- 2022-08-01 CN CN202210914934.0A patent/CN117544635A/en active Pending
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117992597A (en) * | 2024-04-03 | 2024-05-07 | 江苏微皓智能科技有限公司 | Information feedback method, device, computer equipment and computer storage medium |
CN117992597B (en) * | 2024-04-03 | 2024-06-07 | 江苏微皓智能科技有限公司 | Information feedback method, device, computer equipment and computer storage medium |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11240121B2 (en) | Methods and systems for controlling data backup | |
CN112363813A (en) | Resource scheduling method and device, electronic equipment and computer readable medium | |
Wang et al. | Predicting CPU usage for proactive autoscaling | |
CN107943582B (en) | Feature processing method, feature processing device, storage medium and electronic equipment | |
CN113485792B (en) | Pod scheduling method in kubernetes cluster, terminal equipment and storage medium | |
CN117544635A (en) | Resource management method, device, equipment and storage medium | |
US11636377B1 (en) | Artificial intelligence system incorporating automatic model updates based on change point detection using time series decomposing and clustering | |
CN113553138A (en) | Cloud resource scheduling method and device | |
US20220114457A1 (en) | Quantization of tree-based machine learning models | |
CN117193992B (en) | Model training method, task scheduling device and computer storage medium | |
CN117076906B (en) | Distributed intelligent fault diagnosis method and system, computer equipment and storage medium | |
CN116257360B (en) | Method and system for planning container group resources based on historical usage data | |
CN117687774A (en) | Task model training method for computing power scheduling and computing power scheduling method and system | |
CN116432069A (en) | Information processing method, service providing method, device, equipment and medium | |
CN115883392A (en) | Data perception method and device of computing power network, electronic equipment and storage medium | |
CN115169692A (en) | Time series prediction method and related device | |
CN114756352B (en) | Method, device and medium for scheduling server computing resources | |
CN118093097B (en) | Data storage cluster resource scheduling method, device, electronic equipment and medium | |
CN118260091B (en) | Mirror image file placement method based on edge calculation | |
CN115242648B (en) | Expansion and contraction capacity discrimination model training method and operator expansion and contraction capacity method | |
US11836531B2 (en) | Method, device, and program product for managing computing system | |
CN117009028A (en) | Container group scheduling method and device, electronic equipment and storage medium | |
CN111709787B (en) | Method, device, electronic equipment and medium for generating user retention time | |
CN118394534A (en) | Method, system, product, device and medium for expanding and shrinking volume of cluster application service | |
CN117234553A (en) | Model updating method and device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |