KR20230089509A - Bidirectional Long Short-Term Memory based web application workload prediction method and apparatus - Google Patents

Abstract

According to the present invention, disclosed are a method and a device for predicting a web application workload based on Bi-long short term memory (LSTM). According to the present invention, the device comprises: a processor; and a memory connected to the processor. The device of the present invention stores program instructions executed by the processor to output a first hidden state by inputting, into a first LSTM, an input sequence which is time series data on an application matrix including application throughput and average response time requested by a user and pod information on all currently running nodes, in a forward direction, to output a second hidden state by inputting the input sequence into a second LSTM in a backward direction, to calculate a future HTTP request workload using the first and second hidden states through an auto-scaler, and to calculate the number of pods required for provisioning or de-provisioning based on the future HTTP request workload through the auto-scaler. Therefore, a predicted speed faster than ARIMA can be achieved.

Description

Bidirectional Long Short-Term Memory based web application workload prediction method and apparatus based on Bi-LSTM

The present invention relates to a method and apparatus for predicting a Web application workload based on Bi-LSTM.

A key element of cloud computing is virtualization technology, which allows cloud providers to run multiple operating systems and applications on the same computing device, such as a server.

Traditionally, hypervisor-based virtualization is a technology used to create virtual machines (VMs) using specific resources (eg CPU cores, RAM and network bandwidth) and a guest operating system.

Container-based virtualization is an alternative to virtual machines, which takes less startup time and consumes fewer resources than virtual machines.

A key feature of cloud computing is elasticity, which allows application owners to provision and de-provision resources on demand to increase performance while reducing costs to manage the unpredictable workload inherent in Internet-based services.

Autoscaling is a process of dynamically acquiring and releasing resources and can be classified into reactive and proactive.

Choosing the right autoscaling can affect the quality of parameters such as CPU utilization, memory and response time.

While many approaches related to hypervisor-based auto-scaling solutions have been proposed, work related to container-based solutions is still in its infancy.

Many existing reactive auto-scaling solutions use rules with thresholds based on infrastructure-level metrics such as CPU utilization and memory. Although these solutions are easy to implement, choosing the right value for the threshold is difficult because the workload constantly fluctuates based on user behavior. Many methods have been proposed that use time-series data analysis for proactive solutions, but only a few use machine learning algorithms for autoscaler design.

KR Patent Publication No. 10-2021-0066697

In order to solve the problems of the prior art, the present invention proposes a Bi-LSTM based web application workload prediction method and apparatus capable of increasing resource estimation accuracy and handling workload bursts well.

In order to achieve the above object, according to an embodiment of the present invention, an apparatus for predicting a web application workload using a bidirectional short-term memory is provided, comprising: a processor; And a memory connected to the processor, including a first LSTM (Long Short Term Memory) for application metrics including application throughput and average response time requested by the user and pod information of all currently running nodes A first hidden state is output by inputting an input sequence that is time series data in a forward direction, and a second hidden state is output by inputting the input sequence in a reverse direction to a second LSTM, and the first hidden state and the first hidden state are output through an auto scaler 2 Program instructions executed by the processor to calculate a future HTTP request workload using the hidden state and calculate the number of pods required for provisioning or deprovisioning based on the future HTTP request workload through the auto scaler A web application workload prediction apparatus for storing is provided.

A cooling down time (CDT) may be set in the auto scaler after determining the scaling for calculating the required number of pods.

The auto scaler may remove surplus pods according to a resource removal strategy (RSS) when the required number of pods is smaller than the current number of pods.

The auto scaler may include an adaptation manager service that calculates the future HTTP request workload by taking an output sequence obtained by combining the first hidden state and the second hidden state as an input.

The Kubernetes engine of the Kubernetes master node can change the number of pods by receiving commands from the adaptation manager service.

According to another aspect of the present invention, a web application workload prediction system using bi-directional long-term short-term memory, comprising: an application metric collector for collecting application metrics including application throughput and average response time requested by a user; a monitoring server for aggregating time-series data including the application metrics and pod information of all currently running nodes retrieved from the master node and storing them in a time-series database; And a first LSTM outputting a first hidden state by inputting an input sequence corresponding to the time series data in a forward direction, a second LSTM outputting a second hidden state by inputting the input sequence in a reverse direction, and the first hidden state; A proactive auto scaler including an adaptation manager service that calculates a future HTTP request workload using the second hidden state and calculates the number of pods required for provisioning or deprovisioning based on the future HTTP request workload A web application workload prediction system is provided.

According to another aspect of the present invention, a method for predicting a web application workload using a bidirectional short-term memory including a processor and a memory, including application throughput and average response time requested by a user in a first Long Short Term Memory (LSTM) outputting a first hidden state by forwardly inputting an input sequence that is time-series data for application metrics and pod information of all currently running nodes; outputting a second hidden state by reversely inputting the input sequence to a second LSTM; Calculating a future HTTP request workload using the first hidden state and the second hidden state through an auto scaler; and calculating the number of pods required for provisioning or deprovisioning based on the future HTTP request workload through the auto scaler.

According to the present invention, ARIMA and standard feed forward LSTM outperform statistical methods in prediction accuracy, and have the advantage of achieving faster prediction speed than ARIMA.

에서 HTTP 요청 수를 제공한 후 스케일링 전략을 나타낸 알고리즘이다.
도 4는 본 발명의 일 실시예에 따른 양방향 LSTM 네트워크 구조를 도시한 도면이다. 1 is a diagram illustrating the architecture of a workload prediction system according to a preferred embodiment of the present invention.
2 is an architecture of an auto scaler according to this embodiment, and is a diagram showing each step of a MAPE loop.
3 shows that the predictor according to this embodiment is the next level

It is an algorithm that shows the scaling strategy after providing the number of HTTP requests in .
4 is a diagram illustrating a bidirectional LSTM network structure according to an embodiment of the present invention.

Since the present invention can make various changes and have various embodiments, specific embodiments are illustrated in the drawings and described in detail.

However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

In this embodiment We use a recurrent neural network with bidirectional long short-term memory (Bi-LSTM) to predict future HTTP requests.

In general, LSTMs take time-series data in the forward direction. Bi-LSTM is an extension of the LSTM model that applies two LSTMs to the input data. The first LSTM is trained by receiving the time series data in the forward direction, and the second LSTM is trained by receiving the time series data in the backward direction.

In the Bi-LSTM architecture according to this embodiment, two LSTM networks are stacked on top of each other. The first direction runs the input in both directions, from the past to the future, and the second direction from the future to the past. Unlike forward LSTM, using backward LSTM helps to protect future information.

By combining the two hidden states, we can protect past and future information. Applying the LSTM twice leads to improved long-term dependence of machine learning and thus improves the accuracy of the model.

1 is a diagram illustrating the architecture of a workload prediction system according to a preferred embodiment of the present invention.

The workload prediction system according to the present embodiment is based on a MAPE (monitor-analyze-plan-execute) loop.

Referring to FIG. 1, the system according to this embodiment includes a load balancer (Load balancer, 100), an application metrics collector (102), a monitoring server 104, a Kubernetes master (K8S Master 106), and a plurality of of Kubernetes minions (K8S Minion #1 to #n, 108-n).

Load balancer 100 provides support for containerized applications. It serves as a gateway to receive TCP/HTTP requests input from end users and distribute them to a plurality of Kubernetes minions 106-n.

In this embodiment, HAProxy, a fast and reliable open source technology for load balancing and proxying of TCP/HTTP-based applications, is used.

HAProxy is located outside of the Kubernetes cluster, and we configure HAproy to load balance all HTTP requests to all minion nodes in the Kubernetes cluster via the external node IP and node port as shown in Figure 1.

The application metric collector 102 collects application metrics including application throughput and average response time requested by the user, and transmits the collected information to the monitoring server 104 .

The monitoring server 104 receives monitored information from the application metric collector 102 and transmits it to the time series database 110 .

Time series data is a series of data points indexed in chronological order.

The metrics described above can be used for decision making, and the collected values can be exposed by API services and accessed by the ProActed Custom Auto Scaler 112.

In Kubernetes, each cluster consists of one master and multiple minion nodes.

In practice, a cluster can have multiple master nodes to keep the system stable by replacing one master node when it fails.

The master node controls the cluster with four components: etcd, kube-scheduler, kubecontrol-manager and kube-apiserver.

etcd stores all configuration data for the cluster.

kube-scheduler searches for created and unscheduled pods to assign to nodes. kube-scheduler ranks the nodes taking into account constraints and available resources and assigns pods to the appropriate nodes.

kube-control-manager monitors that the cluster is running in the desired state. For example, if your application is running with 5 pods and 1 pod is down or missing after some time, kube-control-manager will create a new clone for the missing pod.

kube-apiserver validates and manages cluster components including pods, services, and replication controllers. Each change in cluster state needs to go through the kube-apiserver, which manages the minion nodes via kubelet.

Minions 108 host pods and execute them according to the instructions of the master 106 .

The kubelet is an agent that runs on each node of the cluster, directed by the master's 106's kube-apiserver and keeps the containers well maintained.

kube-proxy controls the network rules of nodes. These rules allow network communication from the cluster's internal or external network to the Pod.

Kubernetes is one of the most common orchestration platforms for containerized applications. In this embodiment, Docker, the most widely used container engine for Kubernetes, is selected for its light weight and portability.

NodeIP is the external IP address of the minion node, and this IP address is accessible from the internet.

A NodePort is an open port on all minion nodes in the cluster and exposes applications to each minion node. The value of NodePort can have a predefined range between 30,000 and 32,767.

As described above, the workload prediction system according to the present embodiment is based on the MAPE loop and includes a Bi-LSTM prediction service (analysis step) and an adaptive manager service (planning step) centered on the auto scaler 112. do.

2 is an architecture of an auto scaler according to this embodiment, and is a diagram showing each step of a MAPE loop.

Hereinafter, each step of the MAPE loop according to this embodiment will be described in detail.

(1) Monitor phase

The monitoring server 104 according to this embodiment continuously receives various types of data for analysis and planning steps.

Monitoring server 104 can collect data from two sources.

The monitoring server 104 monitors the networking data of the load balancer 100, such as the number of HTTP requests per second through the application metrics collector 102, and the number of pod replicas discovered on the master node (kube-apiserver) of all currently running nodes. Pod information can be received.

After receiving the above information, the monitoring server 104 aggregates them and transmits them to the time series database 110 .

According to this embodiment, it is possible to improve prediction accuracy by adding the time series database 110 to maintain all collected data as past records and training a workload prediction model through this.

(2) Analysis phase - Bi-LSTM prediction service

In the analysis step, the Bi-LSTM prediction service period obtains the most recently collected metric data with window size w through Prometheus' restful API.

를 이용하여

순서의 다음 데이터를 예측한다. 본 실시예에 따르면, Bi-LSTM 신경망 모델은 미래 HTTP 요청의 워크로드를 예측하는데 사용된다. Next, the trained model uses the latest data

using

Predict the next data in the sequence. According to this embodiment, the Bi-LSTM neural network model is used to predict the workload of future HTTP requests.

In general, LSTM is a type of deep RNN used in various fields such as sentiment analysis, speech recognition, and time series analysis.

Unlike feed-forward neural networks, RNNs build networks using current and previous time-step output data as input, and have networks with loops that allow them to maintain historical data. However, conventional RNNs suffer from long-term dependencies due to the vanishing gradient problem.

In RNNs, gradients scale exponentially over time. To solve this problem, LSTM, a special type of RNN, has been proposed, but LSTM has limitations because it can process information only in one direction and ignores continuous data changes.

Bi-LSTM is an extension of the LSTM model where two LSTMs are applied to the input data.

The first LSTM is learned by receiving time-series data in a forward direction, and the second LSTM is learned by receiving time-series data in a reverse direction.

That is, the first LSTM receives time-series data in a forward direction from the past to the present, and the second LSTM receives time-series data in a reverse order from the present to the past, and learning proceeds.

The Bi-LSTM architecture is two LSTM networks stacked on top of each other, running inputs in both directions. The first direction is from the past to the future, and the second direction is from the future to the past. The difference from forward LSTM is that running backward helps preserve information for the future. Bi-LSTM can preserve past and future information by combining two hidden states. Applying the LSTM twice improves the learning long-term dependencies and consequently improves the accuracy of the model.

The Bi-LSTM equation is represented by Equations 1 to 3 below.

,

,

,

,

및

는 서로 다른 가중치 행렬이다. here,

,

,

,

,

and

are different weight matrices.

,

,

는 각각 순방향, 역방향 및 출력 레이어 바이어스이다. f represents the activation function (eg Tanh sigmoid or logistic sigmoid). and,

,

,

are the forward, reverse and output layer biases, respectively.

가 주어지면, Bi-LSTM은 t=1에서 T까지 순방향 레이어를 반복하여 순방향 은닉 상태

, t=T에서 1까지 역방향 레이어를 반복하여 역방향 은닉 상태

, 출력 레이어를 업데이트 하여 출력 시퀀스

를 계산한다. input sequence

Given , the Bi-LSTM iterates through the forward layers from t = 1 to T, resulting in a forward hidden state

, the reverse hidden state by iterating the reverse layer from t=T to 1

, update the output layer so that the output sequence

Calculate

를 입력으로 처리한다. At the tth time step, the two parallel hidden layers in the forward and backward directions are

is treated as an input.

및

는 각각 시간 단계 t에서 순방향 및 역방향의 두 개의 은닉 레이어의 출력 결과이다. hidden state

and

is the output result of the two hidden layers in the forward and backward directions at time step t, respectively.

는 수학식 3에 설명된 것처럼 은닉 상태

및

의 합에 따라 달라진다. output value

is the hidden state as described in Equation 3

and

depends on the sum of

(3) Planning phase - Adaption Manager Service

In the planning phase, the adaptive manager service calculates the number of pods needed to provision or deprovision based on the future HTTP request workload predicted in the previous phase. It is designed to scale pods to meet future workloads.

에서 HTTP 요청 수를 제공한 후 스케일링 전략을 나타낸 알고리즘이다. 3 shows that the predictor according to this embodiment is the next level

It is an algorithm that shows the scaling strategy after providing the number of HTTP requests in .

Due to the vibration problem, the auto scaler 112 frequently performs the opposite operation within a short time, so resources and costs are wasted. To solve this, the cooling down time (CDT) is set to 60 seconds after each scaling decision, which is subdivided compared to the container scenery.

를 포드가 1분 동안 처리할 수 있는 최대 워크로드로 표시하고,

를 유지해야 하는 최소 포드 수로 표시하며, 시스템은 이 값보다 낮은 포드 수로 축소할 수 없다.

represents the maximum workload the pod can handle in one minute,

is the minimum number of pods that must be maintained, and the system cannot scale down to a lower number of pods than this value.

)를 계산하고, 현재 포드 수(

)와 비교한다. The number of pods required for the next step in every minute while the system is running (

), and the current number of pods (

) compared to

If the required number of pods exceeds the current number of pods, a scale out command is triggered, and the scaler will increase the number of pods to meet resource demand in the near future.

를 제거하여 시스템을 안정화하면서 워크로드는 다음 간격에 발생시킨다. However, if the required number of pods is less than the current number of pods, the scaler uses a resource removal strategy (RSS) to remove excess pods.

to stabilize the system while the workload occurs at the next interval.

값을 업데이트 하기 위해

과

값 중 더 높은 값을 선택한다. According to this embodiment, through the max function

to update the value

class

Choose the higher of the values.

보다 낮은 수로 포드 수를 축소할 수 없기 때문에 이러한 작업을 수행해야 한다. the system

We have to do this because we can't scale the number of pods down to a lower number.

The number of surplus pods following the resource removal strategy is calculated as shown in Equation 4.

에서 이전 단계에서 얻은 잉여 포드의 값

를 사용하여

의 값을 업데이트한다. Then the current pod's value

the value of the surplus pod obtained in the previous step from

use with

update the value of

This way, you only need to remove some of the surplus resources, not all of them.

의 값으로 스케일링 인(scale in) 명령을 실행한다. last last updated

Execute the scale in command with the value of

The resource elimination strategy not only stabilizes the system with a low workload, but also allows it to adapt faster to bursts of workload as it spends less time creating and updating the number of pods.

(4) Execution phase

In this final phase of the MAPE loop, the Kubernetes engine (kube-apiserver) receives commands from the adaptive manager service in the planning phase to change the pod replica count.

4 is a diagram illustrating a bidirectional LSTM network structure according to an embodiment of the present invention.

Referring to FIG. 4, the Bi-LSTM model according to the present embodiment includes two layers moving in forward and backward directions.

Each hidden layer contains an input layer with 10 neural cells for 10 time steps, with 30 hidden units for each neural cell.

In both the forward and reverse directions, the final output of the hidden layer is concatenated and used as the input to the dense layer.

A dense layer is a fully connected layer.

That is, each neuron in a dense layer receives input from all neurons in the previous layer, and the output of a dense layer is affected by the specified number of neurons.

In this embodiment, a prediction result is output using a dense layer.

Since the model according to this embodiment is a nonlinear regression model, the hidden layer uses the ReLU activation function.

Although the above has been described with reference to preferred embodiments of the present invention, those skilled in the art can make various modifications to the present invention within the scope not departing from the spirit and scope of the present invention described in the claims below. It will be appreciated that modifications and changes may be made.

Claims (7)

As a web application workload prediction device using bidirectional long and short term memory,
processor; and
Including a memory coupled to the processor,
In the first LSTM (Long Short Term Memory), application metrics including application throughput and average response time requested by the user and input sequences, which are time-series data for pod information of all currently running nodes, are entered in the forward direction to control 1 output the hidden state,
Outputting a second hidden state by inputting the input sequence in a reverse direction to a second LSTM;
Calculate a future HTTP request workload using the first hidden state and the second hidden state through an auto scaler;
To calculate the number of pods required for provisioning or deprovisioning based on the future HTTP request workload through the auto scaler,
A web application workload predictor for storing program instructions executed by the processor. According to claim 1,
A web application workload prediction device in which a cooling down time (CDT) is set in the auto scaler after the scaling decision for calculating the required number of pods. According to claim 1,
The auto scaler removes surplus pods according to a resource removal strategy (RSS) when the required number of pods is less than the current number of pods. According to claim 1,
The auto scaler includes an adaptation manager service that calculates the future HTTP request workload by taking an output sequence obtained by combining the first hidden state and the second hidden state as an input. According to claim 4,
A web application workload prediction device in which the Kubernetes engine of the Kubernetes master node receives a command from the adaptation manager service and changes the number of pods. A web application workload prediction system using bidirectional long-term short-term memory,
an application metric collector that collects application metrics including application throughput and average response time requested by the user;
a monitoring server for aggregating time-series data including the application metrics and pod information of all currently running nodes retrieved from the master node and storing them in a time-series database; and
A first LSTM that outputs a first hidden state by inputting an input sequence corresponding to the time series data in a forward direction, a second LSTM that outputs a second hidden state by inputting the input sequence in a reverse direction, and the first hidden state and the A web including an adaptation manager service that calculates a future HTTP request workload using a second hidden state and calculates the number of pods required for provisioning or deprovisioning based on the future HTTP request workload Application workload forecasting system. A web application workload prediction method using bi-directional long-term short-term memory including a processor and memory,
In the first LSTM (Long Short Term Memory), application metrics including application throughput and average response time requested by the user and input sequences, which are time-series data for pod information of all currently running nodes, are entered in the forward direction to control outputting 1 hidden state;
outputting a second hidden state by reversely inputting the input sequence to a second LSTM;
Calculating a future HTTP request workload using the first hidden state and the second hidden state through an auto scaler; and
and calculating the number of pods required for provisioning or deprovisioning based on the future HTTP request workload through the auto scaler.

Images