WO2013185175A1 - Analytique prédictive pour l'approvisionnement de ressources dans un nuage informatique hybride - Google Patents

Analytique prédictive pour l'approvisionnement de ressources dans un nuage informatique hybride Download PDF

Info

Publication number
WO2013185175A1
WO2013185175A1 PCT/AU2013/000624 AU2013000624W WO2013185175A1 WO 2013185175 A1 WO2013185175 A1 WO 2013185175A1 AU 2013000624 W AU2013000624 W AU 2013000624W WO 2013185175 A1 WO2013185175 A1 WO 2013185175A1
Authority
WO
WIPO (PCT)
Prior art keywords
demand
data
input stream
implemented method
computer implemented
Prior art date
Application number
PCT/AU2013/000624
Other languages
English (en)
Inventor
Kevin Lee
Anna Liu
Original Assignee
National Ict Australia Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from AU2012902517A external-priority patent/AU2012902517A0/en
Application filed by National Ict Australia Limited filed Critical National Ict Australia Limited
Publication of WO2013185175A1 publication Critical patent/WO2013185175A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H04L41/147—Network analysis or design for predicting network behaviour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3442—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for planning or managing the needed capacity
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3447—Performance evaluation by modeling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466—Performance evaluation by tracing or monitoring
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/16—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks using machine learning or artificial intelligence
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/50—Network service management, e.g. ensuring proper service fulfilment according to agreements
    • H04L41/508—Network service management, e.g. ensuring proper service fulfilment according to agreements based on type of value added network service under agreement
    • H04L41/5096—Network service management, e.g. ensuring proper service fulfilment according to agreements based on type of value added network service under agreement wherein the managed service relates to distributed or central networked applications

Definitions

  • the computer file may include complex instructions for processes executed on the server 102 such as an online shop. These processes require computing power and the required computation power depends on the number of users that access the computer file 1 12. It is difficult for the web designer to decide how much investment into computing power is necessary to provide reliable web presence.
  • the server 310 is controlled by an administrator 340 using display device 342 and input device 344.
  • the scaling effector 324 communicates with the cloud provider API and controller 304 via the Internet to control the deployment of resources, such as the number of application nodes, such as cloud virtual machines 306, to perform the application.
  • the predictive analytics engine 322 communicates with monitoring client 308 to receive demand data from the cloud.
  • the local site 310 is where the processing of demand data takes place; it is also where scaling decisions are made and actions are being triggered to accommodate the scaling decision.
  • the remote site 302 is where the cloud provider resides and where demand data is being collected and transferred back to the local site.
  • the network 300 can be used by enterprise organisations wishing to deploy applications into the cloud, specifically with focus on e-commerce applications that are deployed by providers for infrastructure as a service (IaaS). In other examples, other application types such as platform as a service (PaaS) is used. Based on the resource requirement of the application 306 (which is expected to change over time), the predictive analytics engine 322 will adapt to the changing needs by instructing the scaling effector 324 to acquire more (or less) resources from the cloud providers.
  • IaaS infrastructure as a service
  • PaaS platform as a service
  • the specific resources that the scaling effector 324 acquires from the cloud provider are virtual instances with the application 306 deployed on them. These resources are more generally referred to as nodes. By dynamically provision and deprovision resources, the need for the consumer to perform manual capacity planning is removed.
  • the monitoring client 308 is a program that operates on a virtual machine and is responsible for collecting monitoring information from individual application nodes 306 in the remote cloud site 302. The collection of this information is via the virtual cloud network. That is, monitoring information is transferred within the remote cloud site 302 from individual application nodes 306 to the monitoring client 308.
  • Each application node 306 contains a data collection agent which is a piece of software that collects system information such as CPU utilisation, computing time, number of API calls, number of messages sent or received, amount of bandwidth used, memory usage, network data throughput, etc.
  • the pull method is similar to the push method but with the exception that information is polled periodically from the monitoring client 308.
  • the monitoring server 320 issues a poll periodically.
  • the frequency of the poll is defined in the monitoring server 320 and could be changed to accommodate the need of different cloud consumers (e.g., by adjusting the granularity of the monitoring data).
  • An example value of the polling period is 30 seconds. The reason for choosing this value is to ensure that the monitoring server 320 is capable of keeping up with the monitoring data workload (that is the data size will not be growing too rapidly to flood the database server) while without compromising too much on the granularity.
  • the default value for Amazon EC2 is 60 seconds, hence polling in 30 seconds intervals is sufficient for many applications.
  • the format of the information is kept in its original form (i.e., no processing is done on the remote site 302) until it reaches the monitoring server 320 where there is a common translation module 404 to convert the monitoring data into a supported format.
  • the purpose of the translation module 404 is to resolve discrepancies that could arise form different types of monitoring clients, and to translate the data into a common data format.
  • the formatting module 504 is responsible for converting the dataset into a format that can be used as input to the learning module. In one example, this is in some sort of vector format.
  • the vector format can be stored in memory if the dataset is of reasonable size, otherwise it can be dumped into a file (e.g., in comma-delimited format).
  • the formatted data is transferred to the learning module 506 for further processing.
  • the data from the monitoring server 320 comprises data from multiple different data sources, such as monitoring data translation module 404, human hints receiver 407 or internet traffic model receiver 410 of Fig. 4.
  • the data is stored on database server 406 in Fig.
  • the monitoring data comes from the monitoring module 320 and it is periodically updated based on information collected from each of the instances 306. For convenience, we use the monitoring data below in this example:
  • vl [20.63,26.98,29.37]
  • v2 [25.02,20.63,26.98]
  • v3 [17.56,25.02,20.63].
  • the first two values of each vector are treated as inputs to the vector and the third value, which is the most recent value, as the output.
  • the learning algorithm will allow us to find relationship between the inputs and the output, where the output will be the estimated value.
  • a query vector comprising historical demand data is received and a demand pattern is selected such that the input values of the demand pattern, that is all values except the most recent value, are closest to the historical demand data of the query vector. The most recent value of the demand pattern is then used to predict the future demand.
  • the received query vector is also depicted in Fig. 9 with reference numeral 910.
  • Fig. 10 illustrates the input vectors 901 to 906 of Fig. 9. Note that, the learning modules for the two data sets do not have to be the same as long as each learning module at the end returns one or more demand patterns based on the corresponding input data.
  • the method 700 associates the following vectors to demand pattern estimates:
  • Suitable computer readable media may include volatile (e.g. RAM) and/or non-volatile (e.g. ROM, disk) memory, carrier waves and transmission media.
  • Exemplary carrier waves may take the form of electrical, electromagnetic or optical signals conveying digital data steams along a local network or a publically accessible network such as the internet.

Abstract

La présente invention concerne la prédiction de demande d'un système informatique. Un processeur prédit la demande d'un système informatique par une détermination itérative de plusieurs modèles de demande sur la base d'un flux d'entrée de données de demandes historiques. Au cours de chaque itération, le processeur fusionne des sous-chaînes du flux d'entrée sur la base d'une différence entre de multiples estimations de modèles de demandes de l'itération précédente et les sous-chaînes du flux d'entrée. Le processeur détermine ensuite une prédiction de demande sur la base d'une différence entre une requête de demande et chacun des modèles de demande. De cette façon, le processeur fait l'apprentissage de modèles d'après des données historiques et prédit la demande sur la base des données apprises. Ces modèles de demande représentent des modèles qui apparaissent de manière invariable durant une période de temps prolongée.
PCT/AU2013/000624 2012-06-15 2013-06-12 Analytique prédictive pour l'approvisionnement de ressources dans un nuage informatique hybride WO2013185175A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
AU2012902517 2012-06-15
AU2012902517A AU2012902517A0 (en) 2012-06-15 Predictive Analytics for Resource Provisoning in Hybrid Cloud

Publications (1)

Publication Number Publication Date
WO2013185175A1 true WO2013185175A1 (fr) 2013-12-19

Family

ID=49757329

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/AU2013/000624 WO2013185175A1 (fr) 2012-06-15 2013-06-12 Analytique prédictive pour l'approvisionnement de ressources dans un nuage informatique hybride

Country Status (1)

Country Link
WO (1) WO2013185175A1 (fr)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9336245B2 (en) 2013-12-24 2016-05-10 Sap Se Systems and methods providing master data management statistics
US9356883B1 (en) 2014-05-29 2016-05-31 Amazon Technologies, Inc. Allocating cloud-hosted application resources using end-user metrics
WO2016176414A1 (fr) * 2015-04-28 2016-11-03 Solano Labs, Inc. Optimisation de coûts pour des ressources d'informatique en nuage
US9606840B2 (en) 2013-06-27 2017-03-28 Sap Se Enterprise data-driven system for predictive resource provisioning in cloud environments
GB2542870A (en) * 2015-06-30 2017-04-05 British Telecomm Local and demand driven QoS models
US9961675B1 (en) 2017-05-25 2018-05-01 At&T Intellectual Property I, L.P. Multi-layer control plane automatic resource allocation system
US9967327B2 (en) 2010-08-24 2018-05-08 Solano Labs, Inc. Method and apparatus for clearing cloud compute demand
US10171377B2 (en) 2017-04-18 2019-01-01 International Business Machines Corporation Orchestrating computing resources between different computing environments
US10509682B2 (en) 2017-05-24 2019-12-17 At&T Intellectual Property I, L.P. De-allocation elasticity application system
US10728157B2 (en) 2015-06-30 2020-07-28 British Telecommunications Public Limited Company Local and demand driven QoS models
US10963294B2 (en) 2018-07-02 2021-03-30 International Business Machines Corporation Cognitive cloud migration optimizer
US11150931B2 (en) 2018-10-30 2021-10-19 Hewlett Packard Enterprise Development Lp Virtual workload migrations

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5704012A (en) * 1993-10-08 1997-12-30 International Business Machines Corporation Adaptive resource allocation using neural networks
US20040122950A1 (en) * 2002-12-20 2004-06-24 Morgan Stephen Paul Method for managing workloads in an autonomic computer system for improved performance
US20050102398A1 (en) * 2003-11-12 2005-05-12 Alex Zhang System and method for allocating server resources
US20060092851A1 (en) * 2004-10-29 2006-05-04 Jeffrey Forrest Edlund Method and apparatus for communicating predicted future network requirements of a data center to a number of adaptive network interfaces
US20060218278A1 (en) * 2005-03-24 2006-09-28 Fujitsu Limited Demand forecasting system for data center, demand forecasting method and recording medium with a demand forecasting program recorded thereon
US7308687B2 (en) * 2002-02-07 2007-12-11 International Business Machines Corporation Method and system for managing resources in a data center

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5704012A (en) * 1993-10-08 1997-12-30 International Business Machines Corporation Adaptive resource allocation using neural networks
US7308687B2 (en) * 2002-02-07 2007-12-11 International Business Machines Corporation Method and system for managing resources in a data center
US20040122950A1 (en) * 2002-12-20 2004-06-24 Morgan Stephen Paul Method for managing workloads in an autonomic computer system for improved performance
US20050102398A1 (en) * 2003-11-12 2005-05-12 Alex Zhang System and method for allocating server resources
US20060092851A1 (en) * 2004-10-29 2006-05-04 Jeffrey Forrest Edlund Method and apparatus for communicating predicted future network requirements of a data center to a number of adaptive network interfaces
US20060218278A1 (en) * 2005-03-24 2006-09-28 Fujitsu Limited Demand forecasting system for data center, demand forecasting method and recording medium with a demand forecasting program recorded thereon

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
ARDAGNA, D. ET AL.: "Dual time-scale distributed capacity allocation and load redirect algorithms for cloud systems", J. PARALLEL DISTRIB. COMPUT., vol. 72, 14 March 2012 (2012-03-14), pages 796 - 808 *
GONG, Z. ET AL.: "PAC: Pattern-driven Application Consolidation for Efficient Cloud Computing", INTERNATIONAL SYMPOSIUM ON MODELLING, ANALYSIS & SIMULATION OF COMPUTER AND TELECOMMUNICATION SYSTEMS (MASCOTS), IEEE, 17 August 2010 (2010-08-17), pages 24 - 33 *

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9967327B2 (en) 2010-08-24 2018-05-08 Solano Labs, Inc. Method and apparatus for clearing cloud compute demand
US9606840B2 (en) 2013-06-27 2017-03-28 Sap Se Enterprise data-driven system for predictive resource provisioning in cloud environments
US9336245B2 (en) 2013-12-24 2016-05-10 Sap Se Systems and methods providing master data management statistics
US9356883B1 (en) 2014-05-29 2016-05-31 Amazon Technologies, Inc. Allocating cloud-hosted application resources using end-user metrics
AU2016254043B2 (en) * 2015-04-28 2020-12-17 Solano Labs, Inc. Cost optimization of cloud computing resources
US10026070B2 (en) 2015-04-28 2018-07-17 Solano Labs, Inc. Cost optimization of cloud computing resources
WO2016176414A1 (fr) * 2015-04-28 2016-11-03 Solano Labs, Inc. Optimisation de coûts pour des ressources d'informatique en nuage
US10728157B2 (en) 2015-06-30 2020-07-28 British Telecommunications Public Limited Company Local and demand driven QoS models
GB2542870A (en) * 2015-06-30 2017-04-05 British Telecomm Local and demand driven QoS models
US10735345B2 (en) 2017-04-18 2020-08-04 International Business Machines Corporation Orchestrating computing resources between different computing environments
US10171377B2 (en) 2017-04-18 2019-01-01 International Business Machines Corporation Orchestrating computing resources between different computing environments
US10509682B2 (en) 2017-05-24 2019-12-17 At&T Intellectual Property I, L.P. De-allocation elasticity application system
US9961675B1 (en) 2017-05-25 2018-05-01 At&T Intellectual Property I, L.P. Multi-layer control plane automatic resource allocation system
US10963294B2 (en) 2018-07-02 2021-03-30 International Business Machines Corporation Cognitive cloud migration optimizer
US11150931B2 (en) 2018-10-30 2021-10-19 Hewlett Packard Enterprise Development Lp Virtual workload migrations

Similar Documents

Publication Publication Date Title
WO2013185175A1 (fr) Analytique prédictive pour l'approvisionnement de ressources dans un nuage informatique hybride
US11106560B2 (en) Adaptive thresholds for containers
US10810052B2 (en) Methods and systems to proactively manage usage of computational resources of a distributed computing system
US11392416B2 (en) Automated reconfiguration of real time data stream processing
Imdoukh et al. Machine learning-based auto-scaling for containerized applications
US10447772B2 (en) Managed function execution for processing data streams in real time
US20230205675A1 (en) Agentless distributed monitoring of microservices through a virtual switch
US11204811B2 (en) Methods and systems for estimating time remaining and right sizing usable capacities of resources of a distributed computing system
US10841241B2 (en) Intelligent placement within a data center
US10320891B2 (en) Node selection for message redistribution in an integrated application-aware load balancer incorporated within a distributed-service-application-controlled distributed computer system
US8359223B2 (en) Intelligent management of virtualized resources for cloud database systems
US20210141900A1 (en) Methods and systems for troubleshooting applications using streaming anomaly detection
Nguyen et al. {AGILE}: elastic distributed resource scaling for {infrastructure-as-a-service}
CN109117252B (zh) 基于容器的任务处理的方法、系统及容器集群管理系统
US20190317817A1 (en) Methods and systems to proactively manage usage of computational resources of a distributed computing system
KR20150030332A (ko) 데이터 분산 처리 시스템 및 이의 동작 방법
US20180165693A1 (en) Methods and systems to determine correlated-extreme behavior consumers of data center resources
TW201308094A (zh) 藉由使用者歷史紀錄將網頁爬行最佳化
CN103713935A (zh) 一种在线管理Hadoop集群资源的方法和装置
KR20220129085A (ko) 통계 수집 오프로딩
Mohamed et al. A survey of big data machine learning applications optimization in cloud data centers and networks
Liu et al. OnlineElastMan: self-trained proactive elasticity manager for cloud-based storage services
CN114443310A (zh) 资源调度方法、装置、设备、介质和程序产品
Shi et al. Nodens: Enabling Resource Efficient and Fast {QoS} Recovery of Dynamic Microservice Applications in Datacenters
US10110502B1 (en) Autonomous host deployment in managed deployment systems

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13804112

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13804112

Country of ref document: EP

Kind code of ref document: A1