CN117076882A - Dynamic prediction management method for cloud service resources - Google Patents
Dynamic prediction management method for cloud service resources Download PDFInfo
- Publication number
- CN117076882A CN117076882A CN202311013131.9A CN202311013131A CN117076882A CN 117076882 A CN117076882 A CN 117076882A CN 202311013131 A CN202311013131 A CN 202311013131A CN 117076882 A CN117076882 A CN 117076882A
- Authority
- CN
- China
- Prior art keywords
- resource
- model
- cloud service
- data
- prediction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007726 management method Methods 0.000 title claims abstract description 42
- 238000012544 monitoring process Methods 0.000 claims abstract description 20
- 238000000034 method Methods 0.000 claims abstract description 17
- 238000007781 pre-processing Methods 0.000 claims abstract description 11
- 238000013468 resource allocation Methods 0.000 claims abstract description 11
- 238000012549 training Methods 0.000 claims abstract description 8
- 238000003860 storage Methods 0.000 claims description 8
- 238000013136 deep learning model Methods 0.000 claims description 6
- 238000010801 machine learning Methods 0.000 claims description 6
- 238000013528 artificial neural network Methods 0.000 claims description 4
- 238000002790 cross-validation Methods 0.000 claims description 4
- 239000002699 waste material Substances 0.000 claims description 4
- 101001095088 Homo sapiens Melanoma antigen preferentially expressed in tumors Proteins 0.000 claims description 3
- 102100037020 Melanoma antigen preferentially expressed in tumors Human genes 0.000 claims description 3
- 230000006978 adaptation Effects 0.000 claims description 3
- 238000004140 cleaning Methods 0.000 claims description 3
- 125000004122 cyclic group Chemical group 0.000 claims description 3
- 238000007405 data analysis Methods 0.000 claims description 3
- 238000013480 data collection Methods 0.000 claims description 3
- 238000013499 data model Methods 0.000 claims description 3
- 238000003066 decision tree Methods 0.000 claims description 3
- 238000011049 filling Methods 0.000 claims description 3
- 230000007787 long-term memory Effects 0.000 claims description 3
- 230000015654 memory Effects 0.000 claims description 3
- YHXISWVBGDMDLQ-UHFFFAOYSA-N moclobemide Chemical compound C1=CC(Cl)=CC=C1C(=O)NCCN1CCOCC1 YHXISWVBGDMDLQ-UHFFFAOYSA-N 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 3
- 238000007637 random forest analysis Methods 0.000 claims description 3
- 230000006403 short-term memory Effects 0.000 claims description 3
- 230000002159 abnormal effect Effects 0.000 claims description 2
- 238000001514 detection method Methods 0.000 claims description 2
- 230000006872 improvement Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 238000013439 planning Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000001965 increasing effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000013450 outlier detection Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application relates to the technical field of cloud computing, in particular to a cloud service resource dynamic prediction management method. The method comprises the following steps: collecting cloud service historical resource usage data; analyzing the collected data, exploring the mode and trend of resource use, and selecting a proper prediction model to predict future resource demands; preprocessing and feature engineering are carried out on the data according to the selected prediction model requirements; training the selected prediction model by using historical data, and optimizing and adjusting parameters of the model; predicting future resource demands by using the trained prediction model; continuously monitoring the actual resource use condition, and comparing and analyzing with the predicted result; an automatic resource management system is established, and self-adaptive management of resources is realized. The application provides a cloud service resource dynamic prediction management method, which can improve the utilization efficiency of resources on the premise of ensuring the service quality, so that the cloud service resource allocation and configuration are more reasonable to meet the demands of users.
Description
Technical Field
The application relates to the technical field of cloud computing, in particular to a cloud service resource dynamic prediction management method.
Background
Cloud computing, which is the development of technologies such as distributed computing, grid computing, utility computing, network storage and virtualization, has been regarded as the third technical revolution in the IT field, and will thoroughly change the habit of people for using IT resources traditionally. As a brand new technology and business model, the application of cloud computing breeds huge market potential and business opportunities, so that the cloud computing is widely concerned, and various enterprises invest in increasing the research and development force and the application and popularization force of the cloud computing technology, so as to try to preempt the front edge of the technology and the market.
Cloud computing provides shared software and hardware resources and information to various terminals as needed, converts the internet infrastructure into a utility, and enterprises can deploy their own infrastructure through the internet and use the computing resources and storage resources of the network. The cloud computing center can provide various customized services according to the request of the user, timely and fast response.
The elastic service mode of cloud computing brings great popularity to the cloud service market, which is full of cloud services with similar functions. However, devices providing cloud services are usually limited or mobile in resources, which may cause degradation of service quality or no response to the service, so that in the process of resource management, the use efficiency of resources must be improved as much as possible under the premise of ensuring service quality.
Disclosure of Invention
The application aims to provide a cloud service resource dynamic prediction management method, which can improve the utilization efficiency of resources on the premise of ensuring the service quality, so that the cloud service resource allocation and configuration are more reasonable, and the requirements of users are met.
The technical scheme adopted by the application is as follows: a cloud service resource dynamic prediction management method comprises the following steps:
step one: data collection and acquisition: collecting cloud service historical resource usage data;
step two: data analysis and model selection: analyzing the collected data, exploring the mode and trend of resource use, and selecting a proper prediction model to predict future resource demands;
step three: data preprocessing and feature engineering: preprocessing and feature engineering are carried out on the data according to the selected prediction model requirements;
step four: model training and optimizing: training the selected prediction model by using historical data, and optimizing and adjusting parameters of the model;
step five: prediction and decision: predicting future resource demands by using the trained prediction model;
step six: monitoring and feedback: continuously monitoring the actual resource use condition, and comparing and analyzing with the predicted result;
step seven: automation and adaptation: an automatic resource management system is established, and self-adaptive management of resources is realized.
As a further improvement of the present application, the data collected in the first step includes the number of virtual machine instances, storage capacity, network traffic, CPU and memory usage, and may be obtained by means of a monitoring platform, logging and monitoring tool.
As a further improvement of the application, the prediction model in the second step comprises a time sequence model, a machine learning model and a deep learning model; wherein the time series model comprises ARIMA and Prouphet, the machine learning model comprises decision tree and random forest, and the deep learning model comprises a cyclic neural network and a long-term and short-term memory network.
As a further improvement of the application, the data preprocessing method in the third step comprises data cleaning, missing value filling, outlier detection and processing, and simultaneously extracting, converting and selecting the characteristics related to the resource requirements according to the requirements of the model.
As a further improvement of the present application, cross-validation, grid search, RMSE, MAPE metrics may be used in step four to evaluate the performance of the model and select the best model parameters.
As a further improvement of the present application, in the fifth step, a resource allocation policy and a scheduling policy are formulated according to the prediction result, including automatically expanding or reducing the number of virtual machine instances, adjusting the storage capacity, and allocating network bandwidth, so as to meet the demands of users while minimizing the resource waste and cost.
In the sixth step, the monitoring result is fed back to the model, and the model is adjusted and improved to improve the accuracy of prediction and the utilization efficiency of resources.
As a further improvement of the application, the resource management system in the step seven can automatically adjust the resource allocation and configuration according to the predicted demand and the actual resource use condition, so as to realize the self-adaptive management of the resources.
The application has the beneficial effects that:
(1) By accurately predicting and managing the resource demand, the resource waste and idle can be avoided, the resource adjustment is carried out according to the change of the demand, the resource utilization rate can be improved to the maximum extent, and the resource cost is reduced;
(2) By predicting the resource demand and timely adjusting the resource allocation, the cloud service can be ensured to have enough computing capacity and bandwidth in the peak period so as to support the high-load business demand, thereby improving the service performance and response speed and enhancing the user satisfaction;
(3) The method can automatically perform elastic expansion to cope with sudden business load increase, and simultaneously can perform capacity planning, purchase and promote resources in advance through long-term prediction of resource demands, so that the risk of shutdown or performance reduction caused by resource shortage is reduced when the business is increased;
(4) The automatic management of resource demands and distribution can be realized through an automatic resource management system, manual intervention is reduced, and meanwhile, the efficiency and the accuracy of resource management can be improved through the application of a prediction model and a scheduling algorithm;
(5) Through predictive management, appropriate countermeasures can be taken when faults occur or resources are reduced, and continuous execution of critical tasks is ensured.
In summary, the cloud service resource dynamic prediction management can realize reasonable allocation and planning of resources, improve the resource utilization efficiency and the cost control, and simultaneously enhance the service performance and the user satisfaction; through automation and optimization, the efficiency and reliability of resource management can be improved, and the method is suitable for continuously changing service demands.
Drawings
Fig. 1 is a flowchart of a cloud service resource dynamic prediction management method of the present application.
Detailed Description
In order to make the technical problems, technical schemes and beneficial effects to be solved more clear, the application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The application provides a cloud service resource dynamic prediction management method, which comprises the following steps:
step one: data collection and acquisition: collecting cloud service historical resource usage data;
step two: data analysis and model selection: analyzing the collected data, exploring the mode and trend of resource use, and selecting a proper prediction model to predict future resource demands;
step three: data preprocessing and feature engineering: preprocessing and feature engineering are carried out on the data according to the selected prediction model requirements;
step four: model training and optimizing: training the selected prediction model by using historical data, and optimizing and adjusting parameters of the model;
step five: prediction and decision: predicting future resource demands by using the trained prediction model;
step six: monitoring and feedback: continuously monitoring the actual resource use condition, and comparing and analyzing with the predicted result;
step seven: automation and adaptation: an automatic resource management system is established, and self-adaptive management of resources is realized.
The data collected in the first step in the application comprises the number of virtual machine instances, the storage capacity, the network flow, the CPU and the memory utilization rate, and can be obtained by means of a monitoring platform, log record and a monitoring tool.
The prediction model in the second step comprises a time sequence model, a machine learning model and a deep learning model; wherein the time series model comprises ARIMA and Prouphet, the machine learning model comprises decision tree and random forest, and the deep learning model comprises a cyclic neural network and a long-term and short-term memory network.
The data preprocessing method in the third step comprises data cleaning, missing value filling, abnormal value detection and processing, and meanwhile, extracting, converting and selecting characteristics related to resource requirements according to the requirements of the model.
In the fourth step of the present application, cross-validation, grid search, RMSE, MAPE metrics may be used to evaluate the performance of the model and select the best model parameters.
In the fifth step, a resource allocation strategy and a scheduling strategy are formulated according to the prediction result, wherein the method comprises the steps of automatically expanding or reducing the number of virtual machine instances, adjusting the storage capacity and allocating network bandwidth so as to meet the demands of users and simultaneously reduce the resource waste and the cost as much as possible.
In the sixth step, the model is fed back to the model according to the monitoring result, and the model is adjusted and improved so as to improve the accuracy of prediction and the utilization efficiency of resources.
The resource management system in the seventh step can automatically adjust the resource allocation and configuration according to the predicted demand and the actual resource use condition, and realize the self-adaptive management of the resources.
Examples:
firstly, acquiring using data of cloud service history resources through a monitoring platform, a log record and a monitoring tool; then analyzing the collected data, and constructing a neural network prediction model according to the mode and trend of resource use; preprocessing data, and extracting characteristics related to resource requirements according to the requirements of the model; training the model by using historical data, evaluating the performance of the model by adopting a cross-validation mode, and selecting optimal model parameters; predicting future resource demands by using the trained model, and formulating a resource allocation strategy and a scheduling strategy according to the prediction result; monitoring the actual resource use condition, feeding back to the model according to the monitoring result, and adjusting and improving the model; an automatic resource management system is established, and self-adaptive management of resources is realized.
According to the embodiment, the cloud service resource dynamic prediction management method can realize reasonable allocation and planning of resources, improve the resource utilization efficiency and the cost control, and enhance the service performance and the user satisfaction; through automation and optimization, the efficiency and reliability of resource management can be improved, and the method is suitable for continuously changing service demands.
The above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims (8)
1. A cloud service resource dynamic prediction management method is characterized in that: the method comprises the following steps:
step one: data collection and acquisition: collecting cloud service historical resource usage data;
step two: data analysis and model selection: analyzing the collected data, exploring the mode and trend of resource use, and selecting a proper prediction model to predict future resource demands;
step three: data preprocessing and feature engineering: preprocessing and feature engineering are carried out on the data according to the selected prediction model requirements;
step four: model training and optimizing: training the selected prediction model by using historical data, and optimizing and adjusting parameters of the model;
step five: prediction and decision: predicting future resource demands by using the trained prediction model;
step six: monitoring and feedback: continuously monitoring the actual resource use condition, and comparing and analyzing with the predicted result;
step seven: automation and adaptation: an automatic resource management system is established, and self-adaptive management of resources is realized.
2. The cloud service resource dynamic prediction management method according to claim 1, wherein the method comprises the following steps: the data collected in the first step include the number of virtual machine instances, storage capacity, network traffic, CPU and memory usage, and can be obtained by means of a monitoring platform, log record and monitoring tool.
3. The cloud service resource dynamic prediction management method according to claim 1, wherein the method comprises the following steps: the prediction model in the second step comprises a time sequence model, a machine learning model and a deep learning model; wherein the time series model comprises ARIMA and Prouphet, the machine learning model comprises decision tree and random forest, and the deep learning model comprises a cyclic neural network and a long-term and short-term memory network.
4. The cloud service resource dynamic prediction management method according to claim 1, wherein the method comprises the following steps: the data preprocessing method in the third step comprises data cleaning, missing value filling, abnormal value detection and processing, and meanwhile, extracting, converting and selecting characteristics related to resource requirements according to the requirements of the model.
5. The cloud service resource dynamic prediction management method according to claim 1, wherein the method comprises the following steps: in the fourth step, cross-validation, grid search, RMSE, MAPE indexes may be used to evaluate the performance of the model and select the best model parameters.
6. The cloud service resource dynamic prediction management method according to claim 1, wherein the method comprises the following steps: and fifthly, formulating a resource allocation strategy and a scheduling strategy according to the prediction result, wherein the resource allocation strategy and the scheduling strategy comprise automatically expanding or reducing the number of virtual machine instances, adjusting the storage capacity and allocating network bandwidth so as to meet the demands of users and simultaneously reducing the resource waste and the cost as much as possible.
7. The cloud service resource dynamic prediction management method according to claim 1, wherein the method comprises the following steps: and step six, feeding back the monitoring result into the model, and adjusting and improving the model to improve the accuracy of prediction and the utilization efficiency of resources.
8. The cloud service resource dynamic prediction management method according to claim 1, wherein the method comprises the following steps: in the seventh step, the resource management system can automatically adjust the resource allocation and configuration according to the predicted demand and the actual resource use condition, so as to realize the self-adaptive management of the resources.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311013131.9A CN117076882A (en) | 2023-08-11 | 2023-08-11 | Dynamic prediction management method for cloud service resources |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311013131.9A CN117076882A (en) | 2023-08-11 | 2023-08-11 | Dynamic prediction management method for cloud service resources |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117076882A true CN117076882A (en) | 2023-11-17 |
Family
ID=88703452
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311013131.9A Pending CN117076882A (en) | 2023-08-11 | 2023-08-11 | Dynamic prediction management method for cloud service resources |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117076882A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117539648A (en) * | 2024-01-09 | 2024-02-09 | 天津市大数据管理中心 | Service quality management method and device for electronic government cloud platform |
-
2023
- 2023-08-11 CN CN202311013131.9A patent/CN117076882A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117539648A (en) * | 2024-01-09 | 2024-02-09 | 天津市大数据管理中心 | Service quality management method and device for electronic government cloud platform |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Askarizade Haghighi et al. | An energy-efficient dynamic resource management approach based on clustering and meta-heuristic algorithms in cloud computing IaaS platforms: Energy efficient dynamic cloud resource management | |
| CN109324875B (en) | Data center server power consumption management and optimization method based on reinforcement learning | |
| CN104317658A (en) | MapReduce based load self-adaptive task scheduling method | |
| TWI725744B (en) | Method for establishing system resource prediction and resource management model through multi-layer correlations | |
| CN112332404B (en) | Intelligent management system and method for heating service | |
| CN115934333A (en) | Historical data perception-based cloud computing resource scheduling method and system | |
| CN117076882A (en) | Dynamic prediction management method for cloud service resources | |
| CN102075944A (en) | Multi-Agent cooperation-based available resource management system and method in cognitive network | |
| CN114327811A (en) | Task scheduling method, device and equipment and readable storage medium | |
| Li et al. | Cost-aware automatic scaling and workload-aware replica management for edge-cloud environment | |
| CN115543624A (en) | Heterogeneous computing power arrangement scheduling method, system, equipment and storage medium | |
| CN117472587A (en) | Resource scheduling system of AI intelligent computation center | |
| He et al. | Energy-efficient framework for virtual machine consolidation in cloud data centers | |
| CN105022823A (en) | Cloud service performance early warning event generating method based on data mining | |
| Kumar et al. | A Hybrid Eagle’s Web Swarm Optimization (EWSO) technique for effective cloud resource management | |
| CN113806094A (en) | Cloud platform resource dynamic scheduling method based on deep learning | |
| CN117591285A (en) | Computing power resource scheduling and distributing system and method | |
| CN116361006B (en) | Method and system for collaborative scheduling of computing network resources oriented to memory computing separation | |
| CN111988412A (en) | Intelligent prediction system and method for multi-tenant service resource demand | |
| CN116974994A (en) | High-efficiency file collaboration system based on clusters | |
| CN116028193A (en) | Big data task dynamic high-energy-efficiency scheduling method and system for mixed part cluster | |
| CN113517988B (en) | Method and system for scheduling charging flow based on dynamic scene | |
| CN114741200A (en) | Data center station-oriented computing resource allocation method and device and electronic equipment | |
| Caux et al. | Smart datacenter electrical load model for renewable sources management | |
| Li et al. | Multi-resource collaborative optimization for adaptive virtual machine placement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |