KR102448789B1 - Method for scheduling worker in cloud computing system and apparatus using the same - Google Patents

Abstract

A method for scheduling a worker in a cloud computing system and an apparatus therefor are disclosed. The worker scheduling method according to an embodiment of the present invention generates a template worker (TEMPLATE WORKER) in advance to distribute the worker execution preparation load before a worker assignment request for performing a function occurs ( PRE-CREATION) load balancing step 1; estimating the number of worker assignments in advance in consideration of a change in worker assignment request period for each function; and a second step of load balancing of pre-allocating ready workers (READY WORKER) by up-scaling the template workers by the number of workers corresponding to the pre-allocated number of workers (WORKER UP SCALING).

Description

Cloud computing system worker scheduling method and device therefor

The present invention relates to a worker scheduling technique in a serverless cloud computing environment, and more particularly, to a technique for allocating workers to perform a function in a microfunction platform that provides a function-level microservice.

Recently, due to Google's Deep Mind AI (AlphaGo), artificial intelligence is emerging as a hot trend in the IT field. AlphaGo’s computational power was possible because of the distributed cloud computing-based infrastructure that can implement machine learning. Artificial intelligence workloads from autonomous driving and personalized medicine to superhuman speech recognition are gaining popularity in the cloud, providing optimal performance and flexibility for virtually any accelerated workload including deep learning training, inference, advanced analytics, and high-performance computing. can do.

Cloud computing refers to a technology that processes data through virtualization on another computer connected to the network, not on one's own computer or server. Google DeepMind made extensive use of the Google Cloud Platform to give researchers working on artificial intelligence and machine learning access to computing, storage and networking technologies whenever they needed them. IBM's cognitive computing system Watson, another powerhouse in artificial intelligence, also operates based on cloud computing. Jeopardy, which was IBM's early artificial intelligence, was developed at a time when cloud computing technology was not commercially available and implemented on a single general server. In fact, IBM is connecting enterprise hybrid cloud services based on Watson.

FaaS (Function as a Service) is being widely used in the cloud computing field due to the development of the cloud computing paradigm and the increase in demand for an event-based computing model. A worker performing a function in the FaaS computing platform loads a function in a lightweight virtual environment instance such as a container of a compute node, executes the function, and returns the processing result.

At this time, worker scheduling is a process of finding a computation node with an optimal resource to perform a function and preparing a function execution environment, and consists of the following three steps. First, a computational node with optimal resources for creating a worker is selected and assigned. Second, a lightweight virtual environment instance such as a container is created and initialized in the allocated compute node. Finally, the function and related packages and library files are loaded into the worker instance from the repository. In case of initial allocation, all three steps must be performed before allocating a worker. However, if there is a worker whose assignment has been completed after executing a function, the worker can be used again. In this case, the worker assignment response time is processed at the minimum cost.

The scheduler of the existing FaaS computing platform processes worker assignment based on the two-step worker state as shown in FIGS. 1 to 2 . For example, if the worker scheduler performs the task of allocating compute nodes and resources, the compute node creates and initializes a physical worker instance and loads and prepares functions to be executed.

At this time, Figure 2 shows the preparatory load required for the actual execution of the function from the assignment of the function execution worker on the FaaS platform to the loading of the function by testing the execution of the function in the FaaS prototype to check the cost of the worker scheduling based on the state of the two-step worker. corresponds to the results of the analysis. Referring to FIG. 2 , it can be seen that, even when a lightweight container is used, a worker creation cost of 1,750 ms is required for initial worker assignment and 470 ms for reallocation in the same node thereafter. In this way, the worker assignment process takes a lot of time to initially create and start a lightweight virtualized container that is a physical worker instance. Therefore, processing worker assignment at the time of request has a problem of further delaying function execution.

In other words, the worker scheduling algorithm based on the two-step worker state causes a problem that the initial worker creation cost occupies too much of the worker allocation and the execution time increases for short-term functions that require fast processing. In addition, when the load of the function service request is rapidly increased or fluctuated, continuous allocation requests may occur before the previously created worker is executed. In this case, the request-based worker creation method leads to an increase in response time to assignment, and it becomes difficult to quickly process worker assignment for function services.

Korean Patent Publication No. 10-2016-0136489, published on November 30, 2016 (Title: Virtualization-based resource management method for cloud services)

SUMMARY OF THE INVENTION It is an object of the present invention to provide a worker scheduling technique for quickly allocating container-based workers performing function services and minimizing response costs.

Another object of the present invention is to provide a worker scheduling technique capable of actively responding to a function service request load change and minimizing the allocation delay time of a short-term function service requiring a fast response time.

In addition, it is an object of the present invention to provide a new worker scheduling technology that makes the FaaS platform more suitable for a real-time processing environment by properly distributing the execution preparation load of workers related to function execution in advance.

In addition, it is an object of the present invention to distribute worker assignment tasks in a two-step pre-allocation method before request generation, and to allocate workers required for service in advance using the function request period and execution time to respond to worker assignment requests. to minimize scheduling costs.

In addition, it is an object of the present invention to provide a worker scheduler with significantly improved worker allocation cost that can be applied to a system requiring a resource-isolated virtual cloud infrastructure service and a multi-institutional medical intelligence specialized learning and diagnosis sharing platform.

In a worker scheduling method of a cloud computing system according to the present invention for achieving the above object, before a worker assignment request for performing a function occurs, a template worker ( 1st step of load balancing to pre-create (PRE-CREATION) TEMPLATE WORKER; estimating the number of worker assignments in advance in consideration of a change in worker assignment request period for each function; and a second step of load balancing of pre-allocating ready workers (READY WORKER) by up-scaling the template workers by the number of workers corresponding to the pre-allocated number of workers (WORKER UP SCALING).

At this time, when a worker assignment request for performing a function occurs, allocating the ready worker as an active worker, and changing the status of the de-allocated ready worker in consideration of the function service request load. can do.

At this time, the step of initializing the pre-allocated ready workers to the worker down-scaling (WORKER DOWN SCALING) a ready worker that is not used during a preset idle period may further include the step of initializing as the template worker.

In this case, the template worker may correspond to a temporary worker configured to correspond to the basic image and resource capacity to be commonly used by all functions.

In this case, the ready worker may correspond to a worker in a function execution preparation state in which the resource information setting and resource allocation for performing the function have been completed.

At this time, the state of the ready worker corresponds to any one of an active-ready state in which resource allocation and loading of the function file are completed, and an inactive-ready state in which the resource is not allocated but loading of the function file is completed. can do.

At this time, when the active worker usage ratio (ACTIVE WORKER USAGE RATIO) exceeds a preset active upper reference value, performing ready worker up-scaling (READY WORKER UP SCALING) to change the template worker to the ready worker; and when the active worker usage ratio is less than a preset active lower reference value, changing the active-ready ready-worker to the inactive-ready state.

In this case, the active worker usage ratio may correspond to a ratio of the active worker among workers in which the active worker and the ready worker in the active-ready state are combined.

At this time, if the inactive worker ratio (INACTIVE WORKER RATIO) exceeds a preset inactive upper reference value, the ready worker in the inactive ready state is changed to the template worker ready worker down scaling (READY WORKER DOWN SCALING) It may further include the step of performing.

At this time, if the number of template workers is less than a preset allowable limit compared to the maximum number of workers that can be created, performing TEMPLATE WORKER UP SCALING in the background to additionally create the template workers. have.

At this time, the worker allocation request period for each function is calculated based on the number of workers required to be allocated during the unit time for each function, and the number of pre-allocated workers is the number of workers required for the unit time for each function in the active-ready state It may correspond to a value obtained by subtracting the number of in-ready workers and the active workers.

In this case, the number of workers required during the unit time for each function may correspond to a value obtained by dividing the average execution time of the workers executing the function by the worker assignment request period for each function.

In addition, the worker scheduling apparatus of the cloud computing system according to an embodiment of the present invention, before a worker assignment request for performing a function occurs, a template worker ( TEMPLATE WORKER) is created in advance (PRE-CREATION), the number of pre-worker assignments is predicted in consideration of the change in the worker assignment request period for each function, and the template workers are up-scaled by the number of workers corresponding to the pre-worker assignments (WORKER) UP SCALING) to pre-allocate a ready worker (READY WORKER) (PRE-ALLOCATION); and a memory for storing a worker pool (WORKER POOL) that manages a worker for performing a function.

In this case, the processor allocates the ready worker as an active worker when a worker assignment request for performing the function occurs, and considers the function service request load. Can change the status of the released ready worker. .

In this case, the processor may initialize a ready worker not used for a preset idle period among the pre-allocated ready workers as the template worker by WORKER DOWN SCALING.

In this case, the template worker may correspond to a temporary worker configured to correspond to the basic image and resource capacity so that all functions can use it in common.

In this case, the ready worker may correspond to a worker in a function execution preparation state in which the resource information setting and resource allocation for performing the function have been completed.

At this time, the status of the ready worker may correspond to any one of an active ready (ACTIVE READY) state in which resource allocation and loading of the function file are completed, and an INACTIVE READY state in which the loading of the function file is completed although the resource is not allocated. can

At this time, when the active worker usage ratio (ACTIVE WORKER USAGE RATIO) exceeds a preset active upper reference value, the processor performs READY WORKER UP SCALING to change the template worker to the ready worker, and , when the active worker usage ratio is less than a preset active low reference value, the ready worker in the active-ready state may be changed to the in-active-ready state.

In this case, the active worker usage ratio may correspond to a ratio of the active worker among workers in which the active worker and the ready worker in the active-ready state are combined.

At this time, when the inactive worker ratio (INACTIVE WORKER RATIO) exceeds a preset inactive upper reference value, the processor changes the ready worker in the inactive ready state to the template worker. Ready worker down scaling (READY WORKER DOWN SCALING) ) can be done.

In this case, when the number of template workers is less than or equal to a preset allowable limit compared to the maximum number of workers that can be created, the processor may additionally create the template workers by performing TEMPLATE WORKER UP SCALING in the background.

At this time, the worker allocation request period for each function is calculated based on the number of workers required to be allocated during the unit time for each function, and the number of pre-allocated workers is the number of workers required for the unit time for each function in the active-ready state. It may correspond to a value obtained by subtracting the number of ready workers and the active workers.

In this case, the number of workers required during the unit time for each function may correspond to a value obtained by dividing the average execution time of the workers executing the function by the worker assignment request period for each function.

According to the present invention, it is possible to provide a worker scheduling technique for quickly allocating a container-based worker performing a function service and minimizing a response cost.

In addition, the present invention can provide a worker scheduling technique capable of actively responding to a function service request load change and minimizing the allocation delay time of a short-term function service requiring a fast response time.

In addition, the present invention can provide a new worker scheduling technology that makes the FaaS platform more suitable for a real-time processing environment by properly distributing the execution preparation load of workers related to function execution in advance.

In addition, the present invention provides scheduling for worker assignment requests by distributing worker assignment tasks in a two-step pre-allocation method prior to request generation, and pre-allocating workers required for service using the function request period and execution time. cost can be minimized.

In addition, the present invention can provide a worker scheduler with significantly improved worker allocation cost that can be applied to a system requiring a resource-isolated virtual cloud infrastructure service and a multi-institutional medical intelligence specialized learning and diagnosis sharing platform.

1 to 2 are diagrams illustrating an example of two-step worker allocation state transition and execution cost analysis.
3 is a diagram illustrating an example of a micro-function service platform system according to the present invention.
4 is an operation flowchart illustrating a worker scheduling method of a cloud computing system according to an embodiment of the present invention.
5 to 6 are diagrams illustrating an example of a three-step worker allocation state transition and execution cost analysis according to the present invention.
7 is a diagram illustrating an example of a worker pool according to the present invention.
8 is a diagram illustrating an example of worker instance transition according to the present invention.
9 is a diagram illustrating an example of worker state transition according to the present invention.
10 is a diagram illustrating an example of a unit time section according to the present invention.
11 is a diagram illustrating an example of a ready-walker scaling time point according to the present invention.
12 to 14 are diagrams illustrating an example of a ready-walker up-scaling process according to the present invention.
15 is a detailed operation flowchart illustrating a method for scheduling a worker according to an embodiment of the present invention.
16 is a detailed operation flowchart illustrating a template walker up-scaling process according to an embodiment of the present invention.
17 is an operation flowchart illustrating a process of performing up-scaling of a ready-walker according to an active worker usage ratio according to an embodiment of the present invention.
18 is a detailed operation flowchart illustrating a ready-walker up-scaling process according to an embodiment of the present invention.
19 is a detailed operation flowchart illustrating a template walker up-scaling process according to an embodiment of the present invention.
20 is a block diagram illustrating a worker scheduling apparatus of a cloud computing system according to an embodiment of the present invention.
21 is a detailed block diagram illustrating an example of a micro-function service platform and worker scheduler internal components according to the present invention.

The present invention will be described in detail with reference to the accompanying drawings as follows. Here, repeated descriptions, well-known functions that may unnecessarily obscure the gist of the present invention, and detailed descriptions of configurations will be omitted. The embodiments of the present invention are provided in order to more completely explain the present invention to those of ordinary skill in the art. Accordingly, the shapes and sizes of elements in the drawings may be exaggerated for clearer description.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

3 is a diagram illustrating an example of a micro-function service platform system according to the present invention.

Referring to Figure 3, the serverless computing platform system provided with the function service according to the present invention is a client (Client) 310, a serverless computing platform (Serverless Computing Platform) 320, a repository (Repository) (330) and It is composed of a computation node (Node) 340 .

The client 310 is a consumer of the function service that requests the function service from the serverless computing platform 320 through an application program or a web console and receives the result. The function file and related libraries and packages may be stored and managed in the repository 330 .

The calculation node 340 may refer to a physical resource in which a function is actually performed using container instances called workers 341 and 342 to which physical computing resources such as CPU and memory are allocated.

In this case, in the serverless computing platform system, the function service by calling may be performed by the following procedure.

First, a function service request may be called from the client 310 to the serverless computing platform 320 through an interface such as RestAPI.

Thereafter, the serverless computing platform 320 may search for a function called from the repository 330 .

Thereafter, the serverless computing platform 320 may allocate workers 341 and 342 using resource information required to perform a function and current allocation state information.

After that, it is possible to perform a distribution operation of loading a function to be performed on the workers 341 and 342 assigned to the computation node 340 and a library and package necessary for execution.

Thereafter, the workers 341 and 342 may perform a function and transmit a result of the function service processing to the client 310 through the serverless computing platform 320 .

4 is an operation flowchart illustrating a worker scheduling method of a cloud computing system according to an embodiment of the present invention.

Referring to FIG. 4 , in the method for scheduling a worker in a cloud computing system according to an embodiment of the present invention, before a worker assignment request for performing a function occurs, in order to distribute the worker execution preparation load. A template worker (TEMPLATE WORKER) is pre-created (PRE-CREATION) (S410).

At this time, the present invention proposes a three-step worker instance-based worker scheduling algorithm through two-step pre-worker assignment in order to solve the problem of large response time due to the initial assignment of workers and load changes in the two-step worker instance-based worker scheduling. do.

First, a worker corresponds to a higher-level logical resource corresponding to a container object of a computation node performing a micro-function, and may be allocated a computing resource of an actual physical node in the allocation step for executing the function. In this case, the worker according to an embodiment of the present invention may correspond to a basic unit of scheduler allocation and scaling, and may be classified into three stages as shown in FIG. 5 .

5 to 6 , the template worker 510 may correspond to an intermediate temporary worker configured to correspond to a basic image and resource capacity to be commonly used by all functions. In the present invention, as shown in FIG. 6 , the template worker may be utilized for pre-creation that supports quick worker allocation processing and load change processing through distribution of worker allocation and initialization execution times.

The process of creating the template worker 510 corresponds to the process of creating a worker that requires a lot of execution time in the two-step worker instance-based worker scheduling, but it is not created when a worker assignment request is made, but when worker scheduling is first started or It can be executed when the number of template workers falls below a certain percentage. That is, in the present invention, the worker creation process having 70 to 80% of the execution time in the worker assignment and initialization process can be made in the template worker creation step.

At this time, if the number of template workers is less than the preset allowable limit compared to the maximum number of workers that can be created, template workers can be additionally created by performing template worker up-scaling in the background.

For example, template worker upscaling can be performed to create template workers by about 20% of the maximum number of workers that can be created by the total number of compute nodes in the system. At this time, the template worker upscaling is performed in the background, and since it is a temporary worker that is created in advance regardless of workers executing the function, it can operate efficiently so as not to delay the execution of the function or provide a load.

Therefore, when making a general worker assignment request, you only need to perform the process of changing the template worker created in advance to a ready worker through resource information setting and function loading. can be done

In addition, the worker scheduling method of the cloud computing system according to an embodiment of the present invention predicts the number of worker assignments in advance in consideration of the change in the worker assignment request period for each function ( S420 ).

In this case, the present invention relates to a worker scheduling technique that actively responds to a function service request load, and it is possible to determine whether to scale a worker to manage a worker who will execute a function through a change in the request period rather than a worker assignment request.

For example, if the worker assignment request cycle is shorter than the function execution time, the worker can be dynamically increased in the ready state to execute the function through worker scaling. To this end, the number of workers required for scaling, that is, the number of workers allocated in advance, can be calculated using the execution time of the function and the function service request period. can decide

Through this method of worker scaling, it is possible to improve the response time to a worker assignment request by preparing a worker to perform a function in advance.

In this case, the worker assignment request period for each function may be calculated based on the number of workers required to be assigned during the unit time for each function.

For example, referring to FIG. 10 , the worker assignment request period Tf for each function may be calculated based on a dynamic flexible sliding window (DSW). At this time, the dynamic sliding window 1020 shown in FIG. 10 may represent a unit time period Δt that means a unit time for each function, and the worker assignment request period for each function is a worker within the unit time period Δt. It may be calculated by the number of allocation requests 1010 .

In this case, the unit time period Δt may be dynamically and flexibly changed for each function. That is, if there are many worker allocation requests 1010, the unit time period Δt by the dynamic sliding window can be reduced by half, and if there is no worker allocation request 1010 during the unit time period Δt, the corresponding period It can be increased by two times. In this case, the maximum value of the unit time period Δt may correspond to the timeout value of the short-term execution function.

In addition, the worker scheduling method of the cloud computing system according to an embodiment of the present invention pre-allocates the ready worker (READY WORKER) by up-scaling the template worker by the worker up-scaling corresponding to the number of pre-allocated workers (PRE-) ALLOCATION) (S430).

5 to 6 , the ready worker 520 may correspond to a worker in a function execution preparation state in which resource information setting and resource allocation for function execution have been completed. That is, the ready worker is a worker in the stage where assignment is made to the function, and may correspond to the worker in the execution preparation stage in which the actually executed function is loaded.

In addition, the active worker 530 shown in FIG. 5 corresponds to a worker that actually performs a function after allocation by the scheduler is completed, and the ready worker 520, which will be described later, is in an active run state. It can also mean walker.

In this case, the workers of the present invention shown in FIG. 5 may configure a separate list for each worker instance type, and all lists may be managed as one worker pool as shown in FIG. 7 .

At this time, referring to FIG. 7 , a worker pool according to an embodiment of the present invention may be created when a worker scheduler or a worker scheduling device is started, and only the template worker list 710 may initially exist.

In this case, the configuration and resource information of the node may be registered by the node registration request of the node agent provided in each computation node. In addition, the number of template workers that can be created in the computation node can be determined by the memory capacity among the resource information of the node. At first, template workers corresponding to about 30% of the total number of workers that can be created in the node are created and added to the worker pool. can be added

In addition, although not shown in FIG. 4 , the method for scheduling a worker in the cloud computing system according to an embodiment of the present invention allocates a ready worker as an active worker according to a worker assignment request for performing a function, and provides a function service The status of the deallocated ready worker can be changed in consideration of the request load.

For example, referring to FIG. 7 , the template worker (TWK) included in the template worker list 710 may be changed to a ready worker according to a worker assignment request for performing a function.

At this time, the state of the ready worker may correspond to either an active ready state in which resource allocation and loading of the function file are completed, and an inactive ready state in which the resource is not allocated but loading of the function file is completed. can

That is, the active-ready state may refer to a state of a worker instance in a standby state that is not assigned to a worker, but must be maintained in a ready state when considering the function service request load situation, and the inactive-ready state is deallocated. It may mean the state of a worker instance to maintain until the ready-made worker becomes completely unnecessary.

For example, the execution time cost of changing from a template worker to a ready worker is relatively low compared to the cost of creating a template worker, but it still requires 10 to 15% of the execution time. However, changing the active and inactive states of the ready worker simply changes the active-ready list 730 and the inactive-ready list 720 and the state flag shown in FIG. 7 , and requires little execution time.

That is, in the present invention, in order to minimize the situation in which the template worker is changed to a ready worker when an assignment request for a function is made, the lazy deallocation method that maintains the worker created as a ready worker even when the worker is not used is applied to allocate workers. cost can be minimized.

At this time, in the present invention, the actual worker assignment request is performed by preemptively assigning and releasing workers through worker scaling that responds to changes in the usage status of supported system resources, worker availability status, and worker assignment request load changes. You can change and manage worker instances to minimize costs.

In this case, worker scaling according to an embodiment of the present invention may refer to a process of creating, changing, or deleting a plurality of workers when a specific condition or situation is satisfied.

For example, referring to FIG. 8 , worker scaling according to an embodiment of the present invention is a template worker scaling that creates or deletes a plurality of template workers and a ready-made worker that changes a plurality of template workers to a ready worker It can be divided into ready worker up scaling and ready worker down scaling, which changes a ready worker back to a template worker.

At this time, template worker scaling is a process of pre-allocation (up-scaling) or de-allocation (down-scaling) of template workers to offset the total creation cost of workers, which is 70-80% of the total worker allocation cost can correspond to At this time, the template worker may mean a worker to which an image and minimum resources that all workers can use in common are allocated. can At this time, after the worker pool is created, the first template worker can be created in about 20% of the workers that can be created in the entire system. For example, template worker scaling can add or remove template workers from the template worker list managed by the worker pool by creating or deleting template workers of the compute node.

In this case, the ready worker up-scaling may correspond to a process of increasing the number of ready workers by preemptively responding to a function service request. In this case, the ready worker up-scheduling may be performed based on the function assignment request cycle in order to guarantee a quick response and execution time to the worker assignment request. For example, you can change a template worker to a ready worker that can perform functions and add it to the ready worker list.

At this time, ready worker downscaling may correspond to the process of changing to a template worker that can be used in other function assignment requests by initializing the ready worker in an inactive ready state that is assigned to a specific function and not used for a long time. . For example, for a ready worker in an inactive-ready state, resource setting and loading information of a function may be initialized to be changed to a template worker, and the changed template worker may be added to the template worker list.

In this case, the worker scheduling method according to an embodiment of the present invention can manage various worker states as shown in FIG. 9 in the process of worker scaling, worker assignment and release, and change requests or conditions due to function execution can also change the state of the worker.

For example, referring to FIG. 5 , an empty state among five states of a worker may indicate a state in which a worker does not exist as it is.

In addition, the qualified (Qualified) state shown in FIG. 5 means a state in which a worker is created as a base image, which may mean a template worker, and a state in which only a physical container is created in a node without resource allocation. may mean Template workers in this state may be created by template worker scaling or by changing a ready worker in an inactive-ready state to a template worker by down-scaling a ready-walker.

In addition, the inactive ready state shown in FIG. 5 corresponds to a state that a ready worker that has been allocated and used for a function can have, and although resource allocation is not made, the function may mean a loaded state. .

In other words, a significant portion of the worker allocation cost can be preemptively handled by template worker scaling, but 15-20% of the execution cost is incurred even in the process of changing template workers and ready workers, so the response time of worker allocation is still delayed The problem may not be resolved. However, since there is no cost in the process of changing a ready worker from an inactive-ready state to an active-ready state, a quick response to worker assignment requests is made by maintaining an inactive state without changing the currently unused ready worker to a template worker. reply can be returned.

In addition, the active ready state may mean a state in which resource allocation and loading of the function file are completed, so that the function can be performed. At this time, a template worker can be changed to a ready worker through ready worker upscaling, or a ready worker in an active ready state can be created by changing the ready worker state in an inactive ready state.

In addition, the active run state refers to a state in which the worker is actually executing a function, and may be changed to the active run state by a request of worker assignment for a ready worker in the active-ready state.

As described above, by dividing worker creation and initialization in the worker allocation phase and processing them as separate tasks, the response time delay overhead of the worker allocation operation can be reduced. That is, the worker allocation time can be significantly reduced by preparing the worker creation process, which takes most of the time in the worker allocation step, in advance rather than at the time of the worker allocation request.

For example, referring to FIGS. 12 to 14 , in the worker scheduling apparatus according to an embodiment of the present invention, the worker control module 1210 that manages the RestAPI request processing of the external component and the internal component request processing requests the worker assignment For (Fa), worker scaling may be requested to the worker scaler module 1220 for worker pre-allocation to increase response speed. In this case, the worker scaler module 1220 may predict the number of worker assignments in advance based on the worker assignment request period. If it is assumed that there are three ready workers 1340 in the inactive-ready state assigned to the function Fa in the state where the predicted number of assigned pre-workers is 5, as shown in FIGS. 13 to 14, template workers By changing the two 1310s to the ready workers 1320 in the active-ready state, the active workers 1330 may be allocated to correspond to the pre-allocated number of workers.

That is, through active transit, which changes a ready worker in an inactive-ready state to a ready-walker in an active-ready state, it is possible to prepare a ready-worker in an allocable state first at a minimum cost. After that, if the prior worker assignment is not satisfied through active transit, a ready worker upscaling process of changing a template worker to a ready worker may be performed.

In this case, the number of pre-allocated workers may correspond to a value obtained by subtracting the number of active-ready ready workers and active-running workers from the number of workers required for a unit time for each function.

For example, when a worker scaling request occurs to manage the workers required to execute a function, it is executed during the unit time period (Δt) based on the average execution time of the worker (Tx) and the worker allocation request period (Tf). It is possible to calculate the SI (Scaling Index) value, which is the number of required workers.

In this case, the number of workers required during the unit time for each function may correspond to a value obtained by dividing the average execution time (Tx) of the workers executing the function by the worker assignment request period (Tf) for each function. For example, the average execution time (Tx) may be used as the average value of the execution times recorded by the Fn executor at worker termination.

In this case, the number of workers for which worker scaling is required (Incremental Count, IC) may mean the number of pre-allocated workers additionally required except for the currently waiting ready worker and the worker currently executing the function. Accordingly, the IC may correspond to a value obtained by subtracting the number of workers currently in an active-ready state and the number of workers in an active-run state from the SI value, as shown in Equation (1).

[Equation 1]

IC _t = (Tx _t-1 / Tf _t) - (N _rw + N _aw )

At this time, (Tx _t-1 / Tf _t) of the equation may mean SI _t , which is the number of workers required during the unit time period (Δt), and N _rw and N _aw are each of the current active-ready state. It may mean the number of ready workers and the number of workers currently in an active run state.

In addition, although not shown in FIG. 4 , in the method for scheduling a worker in a cloud computing system according to an embodiment of the present invention, when an active worker usage ratio exceeds a preset active upper reference value, the template worker You can perform ready worker upscaling by changing to a ready worker.

In addition, although not shown in FIG. 4 , the worker scheduling method of the cloud computing system according to an embodiment of the present invention sets the active-ready-ready-ready worker in the inactive-ready state when the active worker usage ratio is less than or equal to a preset active low reference value. state can be changed.

At this time, the active worker usage ratio (active worker usage ratio) may correspond to the ratio of active workers among workers in the active-ready state and active workers combined. That is, it may mean a ratio value of workers in an active run state that are actually performing a function among all workers in an active state to which resources are currently allocated (active ready + active run).

For example, referring to FIG. 11 , the worker ratio value of the active run state corresponding to the preset active upper reference value 1110 corresponds to 80%, and the active run corresponding to the preset active lower reference value 1120 . We can assume that the state's worker percentage value is 20%.

In this case, when the measured active worker usage ratio exceeds a preset active upper reference value 1110, ready-walker up-scaling may be performed. That is, when the active worker usage ratio exceeds 80%, most workers are allocated to function execution, so it is determined that additional worker preparation is necessary and ready worker upscaling can be performed.

In addition, when the measured active worker usage ratio is less than or equal to the preset active lower reference value 1120, an inactive transit may be performed. In this case, the inactive transit may correspond to a process of changing the state of the ready worker to the inactive-ready state by releasing the resources allocated to the ready-worker in the active-ready state.

In addition, although not shown in FIG. 4 , in the method for scheduling a worker in a cloud computing system according to an embodiment of the present invention, a ready worker that is not used for a preset idle time can be initialized as a template worker based on the worker downscaling. .

For example, assuming that the preset idle time is IdleTime, among the ready workers in the inactive state assigned to the function, the ready worker downscaling is performed on the worker that is not used to perform the function during IdleTime, and it can be changed to a template worker. .

In addition, although not shown in FIG. 4 , the worker scheduling method of the cloud computing system according to an embodiment of the present invention is inactive when an inactive worker ratio exceeds a preset inactive upper reference value. Ready worker down scaling can be performed by changing a ready worker in a ready state to a template worker.

At this time, the inactive worker ratio (inactive worker ratio) means the ratio value of the ready workers in the inactive-ready state in which the resource allocation is released after performing the function among the ready workers allocated to the function. For example, if it is assumed that the preset inactive upper reference value is 80%, if the inactive worker ratio exceeds 80%, it is determined that the assigned worker is in a stage in which function execution is reduced, and unused ready workers are selected. Ready worker downscaling to convert to template worker can be done.

At this time, although not shown in FIG. 4 , the method for scheduling a worker in a cloud computing system according to an embodiment of the present invention may store various information generated in the above-described worker scheduling process in a separate storage module.

By using such a worker scheduling method, it is possible to quickly allocate container-based workers that perform function services and minimize response costs.

In addition, it is possible to actively respond to function service request load fluctuations, to minimize the allocation delay time for short-term function services that require a fast response time, and to properly distribute the execution load of workers related to function execution in advance so that the FaaS platform It can be made more suitable for a real-time processing environment.

In addition, by applying such a worker scheduling method to a medical AI diagnosis/learning cloud platform based on a cloud infrastructure shared by multiple medical institutions, scheduling that can efficiently support a priority task such as a diagnosis service may be possible.

15 is a detailed operation flowchart illustrating a method for scheduling a worker according to an embodiment of the present invention.

Referring to FIG. 15 , in the method for scheduling a worker according to an embodiment of the present invention, when the apparatus for scheduling a worker receives a request for worker assignment for executing a function, the worker assignment request time information may be recorded ( S1510 ).

In this case, the request time information may be used to predict and calculate the number of workers required to be pre-allocated together with the execution time information of the function.

Thereafter, the worker scheduling apparatus may determine whether the worker assignment request is the first worker assignment request for the function (S1515).

As a result of the determination of step (S1515), if the corresponding worker assignment request is not the first worker assignment request for the function, search for as many active ready workers as requested based on the active ready list managed in the worker pool (S1520) and assignable ready It may be determined whether a worker exists (S1525).

As a result of the determination of step (S1525), if there are ready workers that can be allocated as many as corresponding to the worker allocation request, a ready worker can be selected as an assignment target from the active-ready list of the worker pool (S1530).

Thereafter, the selected ready worker may be changed to an active run state, and information about allocation may be reflected in metadata (S1540).

Thereafter, information on the selected worker may be transmitted in response to the allocation request (S1550).

In addition, if it is determined in step S1515 that the corresponding worker assignment request is the first assignment request for a function, the ready worker in the active-ready state is pre-allocated through the ready-worker upscaling (S1560), and the list created in the worker pool A ready worker in an allocable active-ready state may be selected as an assignment target worker by periodically searching for them (S1570).

In this case, in the case of the first allocation request for a function, it may mean that there is no active-ready ready worker that can be allocated to the worker pool yet. Therefore, when the first allocation request for a function occurs, pre-allocation of launching template workers as many as the basic allocation number as ready workers in an active-ready state can be performed through ready-worker upscaling based on background tasks.

At this time, in the present invention, if the ready workers in the active-ready state are pre-allocated through the ready-walker up-scaling as in step S1560, as shown in FIG. can be inspected (S1605).

If it is determined in step S1605 that the number of template workers managed by the worker pool is less than or equal to a preset allowable limit, template workers may be additionally created and allocated through template worker scaling (S1610).

For example, template workers may be created by 20% of the maximum number of workers that can be created and additionally allocated, and template worker scaling may be performed as a background task.

Through the process shown in FIG. 16 , the system may be able to perform creation and initialization of a worker, which takes the longest processing time in the worker assignment step, before the worker assignment request.

In this case, if the number of template workers managed by the worker pool is not less than the preset allowable limit as a result of the determination in step S1605, it may be terminated without performing an additional operation.

In addition, as a result of the determination of step S1525, if there are no ready workers that can be allocated as much as the worker allocation request, after pre-allocating the ready workers in the active-ready state through ready-walker upscaling (S1560), the worker pool A ready worker in an allocable active-ready state may be selected as an assignment target worker by periodically searching the lists created in . Thereafter, the process from step S1540 to step S1550 may be performed to complete the worker assignment task.

In addition, in the present invention, when the number of active workers corresponding to the active run state increases due to the worker assignment in step S1540, as shown in FIG. 17 , it is checked whether the active worker usage ratio exceeds a preset active upper reference value. Can be (S1705).

If it is determined in step S1705 that the active worker usage ratio exceeds a preset active upper reference value, ready workers may be additionally pre-allocated through ready-walker upscaling (S1710).

By additionally pre-allocating ready workers through the process shown in FIG. 17, when a worker assignment request is received, ready workers can be assigned immediately.

18 is a detailed operation flowchart illustrating a ready-walker up-scaling process according to an embodiment of the present invention.

Referring to FIG. 18, the ready-walker up-scaling process according to an embodiment of the present invention performs the first worker assignment for performing a function or the active worker usage ratio exceeds a preset upper reference value, so the ready-walker up-scaling request is When it occurs (S1810), it is possible to calculate the worker allocation request period (Tf) in order to obtain the number of pre-allocation workers (Incremental Count, IC) requiring ready-walker upscaling (S1820).

In this case, the worker assignment request period Tf may be calculated based on the number of worker assignment requests recorded during the unit time period Δt based on the current time, that is, the number of function requests.

Thereafter, the number of workers (Scaling Index, SI) required for the function service may be calculated by dividing the average execution time (Tx) of the workers executing the function by the worker assignment request period (Tf) (S1830).

In this case, the average execution time Tx may correspond to an average value of execution times recorded as the worker terminates.

Thereafter, the number of workers allocated in advance (IC) may be calculated by subtracting the number of workers in the active state currently allocated to the corresponding function from the number of workers (SI) required to perform the function (S1840).

In this case, the worker in the active state may mean a ready worker in an active-ready state and an active worker in an active run state.

Thereafter, it may be determined whether there is an allocable worker in an inactive ready list of the worker pool ( S1845 ).

At this time, since the ready workers of the inactive ready list are workers in a state in which only resource allocation is released after the execution of the function is terminated, they can be used as ready workers in the active ready state as soon as resource allocation is made. Therefore, in the present invention, it is possible to prepare a worker in an allocable state at a minimum cost through an active transit that changes a ready worker in an inactive-ready state to a ready-walker in an active-ready state.

As a result of the determination of step (S1845), if there is an assignable worker in the inactive-ready list, the state of the worker is changed to the active-ready state through the active transit (S1850), and all the workers are pre-assigned by the number of pre-allocated workers (IC). It may be determined whether or not the assignment has been made (S1855).

As a result of the determination of step S1855, if workers are pre-allocated as much as the number of pre-allocated workers (IC), the ready-walker up-scaling may be terminated.

That is, if all the workers as many as the number of pre-allocated workers (IC) are allocated through active transit, the ready worker upscaling may be terminated.

However, if there is no worker assignable to the inactive ready list as a result of the determination of step S1845, or if the number of pre-allocated workers is not pre-allocated as a result of the determination of step S1855 (IC), step S1860 A ready worker upscaling process of launching a template worker and changing it to a ready worker may be performed as in step S1890.

First, the worker scheduling apparatus according to an embodiment of the present invention may request environment information and function storage information for performing a function from a node agent provided in a computation node (S1860).

Thereafter, the template worker can be launched using the function execution environment information as a parameter through the container manager and changed to the ready worker (S1870).

In this case, the node agent may perform the task of launching the worker based on the container manager or the lightweight virtual instance manager using the function execution environment information as a parameter.

Thereafter, the ready worker can be prepared by downloading the function file to the storage of the launched template worker and performing a pre-processing task for executing the function (S1880).

After that, the ready worker upscaling is terminated when the ready worker in the active ready state corresponding to the pre-allocated number of workers (IC) is ready by changing the state of the prepared ready worker to the active ready state and reflecting the allocation information of the corresponding ready worker. can be (S1890).

19 is a detailed operation flowchart illustrating a template walker up-scaling process according to an embodiment of the present invention.

Referring to FIG. 19 , in the template worker upscaling process according to an embodiment of the present invention, when a calculation node is first registered or the number of template workers available in the system by ready worker upscaling becomes less than or equal to a preset allowable limit, Template worker upscaling may be requested in the background by the worker scheduling device (S1910).

Thereafter, the worker scheduling apparatus may select a target node requiring template worker upscaling and calculate the number of template workers to be generated in the target node ( S1920 ).

For example, the number of template workers generated may be calculated to correspond to 20% of the number of workers that can be created in the target node.

After that, when the worker scheduling device requests information for template worker creation from the node agent of the target node with the number of template workers created as a parameter (S1930), the node agent sends an instance of the template worker through the container manager or lightweight virtual instance manager. An in-container may be created (S1940).

Thereafter, the worker scheduling apparatus may change the state of the created template worker to Qualified, add it to the template worker list of the worker pool (S1950), and reflect creation information of the worker.

After that, although not shown in FIG. 19, after checking whether the template worker creation for all the calculation nodes has been completed, if there is a calculation node for generating the template worker, the process from step S1910 to step S1950 is performed It can be performed repeatedly for each node. After that, when the creation of template workers for all computation nodes is completed, template worker upscaling may be terminated.

20 is a block diagram illustrating a worker scheduling apparatus of a cloud computing system according to an embodiment of the present invention.

Referring to FIG. 20 , a worker scheduling apparatus of a cloud computing system according to an embodiment of the present invention includes a processor 2010 and a memory 2020 .

The processor 2010 pre-creates (PRE-CREATION) a template worker (TEMPLATE WORKER) to distribute the worker execution preparation load before a worker assignment request for performing a function (FUNCTION) occurs.

At this time, the present invention proposes a three-step worker instance-based worker scheduling algorithm through two-step pre-worker assignment in order to solve the problem of large response time due to the initial assignment of workers and load changes in the two-step worker instance-based worker scheduling. do.

First, a worker corresponds to a higher-level logical resource corresponding to a container object of a computation node performing a micro-function, and may be allocated a computing resource of an actual physical node in the allocation step for executing the function. In this case, the worker according to an embodiment of the present invention may correspond to a basic unit of scheduler allocation and scaling, and may be classified into three stages as shown in FIG. 5 .

5 to 6 , the template worker 510 may correspond to an intermediate temporary worker configured to correspond to a basic image and resource capacity to be commonly used by all functions. In the present invention, as shown in FIG. 6 , the template worker may be utilized for pre-creation that supports quick worker allocation processing and load change processing through distribution of worker allocation and initialization execution times.

The process of creating the template worker 510 corresponds to the process of creating a worker that requires a lot of execution time in the two-step worker instance-based worker scheduling, but it is not created when a worker assignment request is made, but when worker scheduling is first started or It can be executed when the number of template workers falls below a certain percentage. That is, in the present invention, the worker creation process having 70 to 80% of the execution time in the worker assignment and initialization process can be made in the template worker creation step.

At this time, if the number of template workers is less than the preset allowable limit compared to the maximum number of workers that can be created, template workers can be additionally created by performing template worker up-scaling in the background.

For example, template worker upscaling can be performed to create template workers by about 20% of the maximum number of workers that can be created by the total number of compute nodes in the system. At this time, the template worker upscaling is performed in the background, and since it is a temporary worker that is created in advance regardless of workers executing the function, it can operate efficiently so as not to delay the execution of the function or provide a load.

Therefore, when making a general worker assignment request, you only need to perform the process of changing the template worker created in advance to a ready worker through resource information setting and function loading. can be done

Also, the processor 2010 predicts the number of worker assignments in advance in consideration of the change in the worker assignment request period for each function.

In this case, the present invention relates to a worker scheduling technique that actively responds to a function service request load, and it is possible to determine whether to scale a worker to manage a worker who will execute a function through a change in the request period rather than a worker assignment request.

For example, if the worker assignment request cycle is shorter than the function execution time, the worker can be dynamically increased in the ready state to execute the function through worker scaling. To this end, the number of workers required for scaling, that is, the number of workers allocated in advance, can be calculated using the execution time of the function and the function service request period. can decide

Through this method of worker scaling, it is possible to improve the response time to a worker assignment request by preparing a worker to perform a function in advance.

In this case, the worker assignment request period for each function may be calculated based on the number of workers required to be assigned during the unit time for each function.

For example, referring to FIG. 10 , the worker assignment request period Tf for each function may be calculated based on a dynamic flexible sliding window (DSW). At this time, the dynamic sliding window 1020 shown in FIG. 10 may represent a unit time period Δt that means a unit time for each function, and the worker assignment request period for each function is a worker within the unit time period Δt. It may be calculated by the number of allocation requests 1010 .

In this case, the unit time period Δt may be dynamically and flexibly changed for each function. That is, if there are many worker allocation requests 1010, the unit time period Δt by the dynamic sliding window can be reduced by half, and if there is no worker allocation request 1010 during the unit time period Δt, the corresponding period It can be increased by two times. In this case, the maximum value of the unit time period Δt may correspond to the timeout value of the short-term execution function.

In addition, the processor 2010 pre-allocates the ready worker (READY WORKER) by up-scaling the template worker by the worker up-scaling corresponding to the number of pre-allocated workers (PRE-ALLOCATION).

5 to 6 , the ready worker 520 may correspond to a worker in a function execution preparation state in which resource information setting and resource allocation for function execution have been completed. That is, the ready worker is a worker in the stage where assignment is made to the function, and may correspond to the worker in the execution preparation stage in which the actually executed function is loaded.

In addition, the active worker 530 shown in FIG. 5 corresponds to a worker that actually performs a function after allocation by the scheduler is completed, and the ready worker 520, which will be described later, is in an active run state. It can also mean walker.

In this case, the workers of the present invention shown in FIG. 5 may configure a separate list for each worker instance type, and all lists may be managed as one worker pool as shown in FIG. 7 .

At this time, referring to FIG. 7 , a worker pool according to an embodiment of the present invention may be created when a worker scheduler or a worker scheduling device is started, and only the template worker list 710 may initially exist.

In this case, the configuration and resource information of the node may be registered by the node registration request of the node agent provided in each computation node. In addition, the number of template workers that can be created in the computation node can be determined by the memory capacity among the resource information of the node. At first, template workers corresponding to about 30% of the total number of workers that can be created in the node are created and added to the worker pool. can be added

In addition, the processor 2010 may allocate a ready worker as an active worker according to a worker assignment request for performing a function, and change the state of the released ready worker in consideration of the function service request load.

For example, referring to FIG. 7 , the template worker (TWK) included in the template worker list 710 may be changed to a ready worker according to a worker assignment request for performing a function.

At this time, the state of the ready worker may correspond to either an active ready state in which resource allocation and loading of the function file are completed, and an inactive ready state in which the resource is not allocated but loading of the function file is completed. can

That is, the active-ready state may refer to a state of a worker instance in a standby state that is not assigned to a worker, but must be maintained in a ready state when considering the function service request load situation, and the inactive-ready state is deallocated. It may mean the state of a worker instance to maintain until the ready-made worker becomes completely unnecessary.

For example, the execution time cost of changing from a template worker to a ready worker is relatively low compared to the cost of creating a template worker, but it still requires 10 to 15% of the execution time. However, changing the active and inactive states of the ready worker simply changes the active-ready list 730 and the inactive-ready list 720 and the state flag shown in FIG. 7 , and requires little execution time.

That is, in the present invention, in order to minimize the situation in which the template worker is changed to a ready worker when an assignment request for a function is made, the lazy deallocation method that maintains the worker created as a ready worker even when the worker is not used is applied to allocate workers. cost can be minimized.

At this time, in the present invention, the actual worker assignment request is performed by preemptively assigning and releasing workers through worker scaling that responds to changes in the usage status of supported system resources, worker availability status, and worker assignment request load changes. You can change and manage worker instances to minimize costs.

In this case, worker scaling according to an embodiment of the present invention may refer to a process of creating, changing, or deleting a plurality of workers when a specific condition or situation is satisfied.

For example, referring to FIG. 8 , worker scaling according to an embodiment of the present invention is a template worker scaling that creates or deletes a plurality of template workers and a ready-made worker that changes a plurality of template workers to a ready worker It can be divided into ready worker up scaling and ready worker down scaling, which changes a ready worker back to a template worker.

At this time, template worker scaling is a process of pre-allocation (up-scaling) or de-allocation (down-scaling) of template workers to offset the total creation cost of workers, which is 70-80% of the total worker allocation cost can correspond to At this time, the template worker may mean a worker to which an image and minimum resources that all workers can use in common are allocated. can At this time, after the worker pool is created, the first template worker can be created in about 20% of the workers that can be created in the entire system. For example, template worker scaling can add or remove template workers from the template worker list managed by the worker pool by creating or deleting template workers of the compute node.

In this case, the ready worker up-scaling may correspond to a process of increasing the number of ready workers by preemptively responding to a function service request. In this case, the ready worker up-scheduling may be performed based on the function assignment request cycle in order to guarantee a quick response and execution time to the worker assignment request. For example, you can change a template worker to a ready worker that can perform functions and add it to the ready worker list.

At this time, ready worker downscaling may correspond to the process of changing to a template worker that can be used in other function assignment requests by initializing the ready worker in an inactive ready state that is assigned to a specific function and not used for a long time. . For example, for a ready worker in an inactive-ready state, resource setting and loading information of a function may be initialized to be changed to a template worker, and the changed template worker may be added to the template worker list.

At this time, the processor according to an embodiment of the present invention can manage the states of various workers as shown in FIG. 9 in the process of worker scaling, worker assignment and release, and by changing a request or condition by executing a function. It can also change the state of the walker.

For example, referring to FIG. 5 , an empty state among five states of a worker may indicate a state in which a worker does not exist as it is.

In addition, the qualified (Qualified) state shown in FIG. 5 means a state in which a worker is created as a base image, which may mean a template worker, and a state in which only a physical container is created in a node without resource allocation. may mean Template workers in this state may be created by template worker scaling or by changing a ready worker in an inactive-ready state to a template worker by down-scaling a ready-walker.

In addition, the inactive ready state shown in FIG. 5 corresponds to a state that a ready worker that has been allocated and used for a function can have, and although resource allocation is not made, the function may mean a loaded state. .

In other words, a significant portion of the worker allocation cost can be preemptively handled by template worker scaling, but 15-20% of the execution cost is incurred even in the process of changing template workers and ready workers, so the response time of worker allocation is still delayed The problem may not be resolved. However, since there is no cost in the process of changing a ready worker from an inactive-ready state to an active-ready state, a quick response to worker assignment requests is made by maintaining an inactive state without changing the currently unused ready worker to a template worker. reply can be returned.

In addition, the active ready state may mean a state in which resource allocation and loading of the function file are completed, so that the function can be performed. At this time, a template worker can be changed to a ready worker through ready worker upscaling, or a ready worker in an active ready state can be created by changing the ready worker state in an inactive ready state.

In addition, the active run state refers to a state in which the worker is actually executing a function, and may be changed to the active run state by a request of worker assignment for a ready worker in the active-ready state.

As described above, by dividing worker creation and initialization in the worker allocation phase and processing them as separate tasks, the response time delay overhead of the worker allocation operation can be reduced. That is, the worker allocation time can be significantly reduced by preparing the worker creation process, which takes most of the time in the worker allocation step, in advance rather than at the time of the worker allocation request.

For example, referring to FIGS. 12 to 14 , in the worker scheduling apparatus according to an embodiment of the present invention, the worker control module 1210 that manages the RestAPI request processing of the external component and the internal component request processing requests the worker assignment For (Fa), worker scaling may be requested to the worker scaler module 1220 for worker pre-allocation to increase response speed. In this case, the worker scaler module 1220 may predict the number of worker assignments in advance based on the worker assignment request period. If it is assumed that there are three ready workers 1340 in the inactive-ready state assigned to the function Fa in the state where the predicted number of assigned pre-workers is 5, as shown in FIGS. 13 to 14, template workers By changing the two 1310s to the ready workers 1320 in the active-ready state, the active workers 1330 may be allocated to correspond to the pre-allocated number of workers.

That is, through active transit, which changes a ready worker in an inactive-ready state to a ready-walker in an active-ready state, it is possible to prepare a ready-worker in an allocable state first at a minimum cost. After that, if the prior worker assignment is not satisfied through active transit, a ready worker upscaling process of changing a template worker to a ready worker may be performed.

In this case, the number of pre-allocated workers may correspond to a value obtained by subtracting the number of active-ready ready workers and active-running workers from the number of workers required for a unit time for each function.

For example, when a worker scaling request occurs to manage the workers required to execute a function, it is executed during the unit time period (Δt) based on the average execution time of the worker (Tx) and the worker allocation request period (Tf). It is possible to calculate the SI (Scaling Index) value, which is the number of required workers.

In this case, the number of workers required during the unit time for each function may correspond to a value obtained by dividing the average execution time (Tx) of the workers executing the function by the worker assignment request period (Tf) for each function. For example, the average execution time (Tx) may be used as the average value of the execution times recorded by the Fn executor at worker termination.

In this case, the number of workers for which worker scaling is required (Incremental Count, IC) may mean the number of pre-allocated workers additionally required except for the currently waiting ready worker and the worker currently executing the function. Accordingly, the IC may correspond to a value obtained by subtracting the number of workers currently in an active-ready state and the number of workers in an active-run state from the SI value, as shown in Equation (1).

[Equation 1]

IC _t = (Tx _t-1 / Tf _t) - (N _rw + N _aw )

At this time, (Tx _t-1 / Tf _t) of the equation may mean SI _t , which is the number of workers required during the unit time period (Δt), and N _rw and N _aw are each of the current active-ready state. It may mean the number of ready workers and the number of workers currently in an active run state.

In addition, when the active worker usage ratio exceeds a preset active upper reference value, the processor 2010 performs ready worker up-scaling to change the template worker to the ready worker. can

Also, when the active worker usage ratio is less than or equal to a preset active low reference value, the processor 2010 may change the ready worker in the active-ready state to the in-active-ready state.

At this time, the active worker usage ratio (active worker usage ratio) may correspond to the ratio of active workers among workers in the active-ready state and active workers combined. That is, it may mean a ratio value of workers in an active run state that are actually performing a function among all workers in an active state to which resources are currently allocated (active ready + active run).

For example, referring to FIG. 11 , the worker ratio value of the active run state corresponding to the preset active upper reference value 1110 corresponds to 80%, and the active run corresponding to the preset active lower reference value 1120 . We can assume that the state's worker percentage value is 20%.

In this case, when the measured active worker usage ratio exceeds a preset active upper reference value 1110, ready-walker up-scaling may be performed. That is, when the active worker usage ratio exceeds 80%, most workers are allocated to function execution, so it is determined that additional worker preparation is necessary and ready worker upscaling can be performed.

In addition, when the measured active worker usage ratio is less than or equal to the preset active lower reference value 1120, an inactive transit may be performed. In this case, the inactive transit may correspond to a process of changing the state of the ready worker to the inactive-ready state by releasing the resources allocated to the ready-worker in the active-ready state.

Also, the processor 2010 may initialize a ready worker not used for a preset idle time as a template worker based on the worker downscaling.

For example, assuming that the preset idle time is IdleTime, among the ready workers in the inactive state assigned to the function, the ready worker downscaling is performed on the worker that is not used to perform the function during IdleTime, and it can be changed to a template worker. .

In addition, the processor 2010, when the inactive worker ratio (inactive worker ratio) exceeds a preset inactive upper reference value, ready worker down scaling (ready worker down) to change the inactive-ready ready worker to the template worker scaling) can be performed.

At this time, the inactive worker ratio (inactive worker ratio) means the ratio value of the ready workers in the inactive-ready state in which the resource allocation is released after performing the function among the ready workers allocated to the function. For example, if it is assumed that the preset inactive upper reference value is 80%, if the inactive worker ratio exceeds 80%, it is determined that the assigned worker is in a stage in which function execution is reduced, and unused ready workers are selected. Ready worker downscaling to convert to template worker can be done.

The memory 2020 stores a worker pool that manages a worker for performing a function.

In addition, the memory 2020 may support a function for scheduling a worker according to an embodiment of the present invention as described above. In this case, the memory 2020 may operate as a separate mass storage and may include a control function for performing an operation.

On the other hand, the worker scheduling device may be equipped with a memory to store information in the device. For one implementation, the memory is a computer-readable medium. In one implementation, the memory may be a volatile memory unit, and in another implementation, the memory may be a non-volatile memory unit. In one embodiment, the storage device is a computer-readable medium. In various different implementations, the storage device may include, for example, a hard disk device, an optical disk device, or some other mass storage device.

Through such a worker scheduling device, it is possible to quickly allocate a container-based worker that performs a function service and minimize the response cost.

In addition, it is possible to actively respond to function service request load fluctuations, to minimize the allocation delay time for short-term function services that require a fast response time, and to properly distribute the execution load of workers related to function execution in advance so that the FaaS platform It can be made more suitable for a real-time processing environment.

21 is a detailed block diagram illustrating an example of a micro-function service platform and worker scheduler internal components according to the present invention.

Referring to FIG. 21 , the worker scheduler 2140 according to the present invention transfers the resources registered by the node agent component provided in the computing node or the computing nodes 2151 to 2153 by itself without the help of the node agent component. By managing it, you can allocate workers required to execute the function.

In this case, the worker scheduler 2140 may mainly operate a worker controller that manages the RestAPI request processing of the external component and the internal component request processing.

In addition, the worker scheduler 2140 may register the resource used for performing the function based on the node agent component at the initial stage of execution in the resource allocator, and deduct the resource by itself before allocating the worker, or release the worker It can be increased or decreased again later.

In addition, the worker scheduler 2140 may allocate and manage workers in advance in order to increase the response speed to the worker assignment request of the API gateway 2110 .

For example, the worker scheduler 2140 may pre-allocate more than a required number of workers through a worker scaler module in order to reduce the starting load of the workers. For this, a worker pool manager module This note agent component can request startup and shutdown of a plurality of predefined workers.

In this way, the pre-allocated workers can be managed as a separate list for each instance type by the worker pool manager module.

In this case, the worker scaler module may play a role of automatically expanding or reducing the workers managed by the worker pool manager module according to the function request load. At this time, the worker scaler module can perform worker scaling that dynamically adjusts the state and number of workers managed by the worker pool manager module in response to the load requesting the function execution.

In this case, pre-allocation of workers performed by the worker scaler module may be performed in two steps. First, step 1 pre-allocation is the process of creating a template worker composed of a common base image and base resource capacity for worker execution, which can be handled by template worker scaling in the worker scaler module. . After that, step 2 pre-allocation is a process of changing the template worker created in step 1 to a ready worker to actually perform a function. can be processed.

In addition, the worker scheduler 2140 may manage operation information through a DB table called WS metadata to control the above operation.

As described above, the configuration and method of the embodiments described above may not be limitedly applicable to the method and apparatus for scheduling a worker in the cloud computing system according to the present invention, but various modifications may be made to the embodiments. All or part of each embodiment may be selectively combined and configured.

310: client 320: serverless computing platform
330: repository 340, 2151-2153: compute node
341, 342: workers 510, 1310: template workers
520: ready walker 530, 1330: active worker
710: template worker list 720: inactive ready list
730: active ready list 1110: preset active upper reference value
1120: preset active lower reference value
1210: walker control module 1220: walker scaler module
1320: ready-walker in active-ready state
1330: Ready walker in inactive ready state
2010: Processor 2020: Memory
2110: API GATEWAY 2120: Fn NAMAGER
2130: Fn EXECUTOR 2140: worker scheduler

Claims (20)

Load balancing step 1 of pre-creating (PRE-CREATION) a template worker (TEMPLATE WORKER) in order to distribute the load ready for worker execution before a worker assignment request for performing a function (FUNCTION) occurs;
estimating the number of worker assignments in advance in consideration of a change in worker assignment request period for each function; and
Load balancing step 2 of pre-allocating ready workers (READY WORKER) by up-scaling the template workers by the number of workers allocated in advance (WORKER UP SCALING)
including,
The worker scheduling method of the cloud computing system, characterized in that the worker execution preparation load is distributed through the first load balancing step and the second load balancing step. The method according to claim 1,
When a worker assignment request for performing the function occurs, allocating the ready worker as an active worker, and changing the status of the de-allocated ready worker in consideration of the function service request load A method for scheduling workers in a cloud computer system, characterized in that it is The method according to claim 1,
Worker scheduling method of the cloud computing system, characterized in that it further comprises the step of initializing the template worker by scaling down a worker that is not used for a preset idle period among the pre-allocated ready workers. The method according to claim 1,
The template worker is a worker scheduling method of a cloud computing system, characterized in that it corresponds to a temporary worker configured to correspond to a basic image and resource capacity so that all functions can use it in common. 3. The method according to claim 2,
The ready worker is a worker scheduling method of a cloud computing system, characterized in that it corresponds to a worker in a function execution preparation state in which the resource information setting and resource allocation for performing the function have been completed. 6. The method of claim 5,
The status of the ready worker is
A cloud computing system, characterized in that it corresponds to any one of an active-ready state in which the resource allocation and loading of the function file are completed, and an inactive-ready state in which the resource is not allocated but the loading of the function file is completed How to schedule workers. 7. The method of claim 6,
When the active worker usage ratio (ACTIVE WORKER USAGE RATIO) exceeds a preset active upper reference value, performing a ready worker up-scaling (READY WORKER UP SCALING) to change the template worker to the ready worker; and
When the active worker usage ratio is less than a preset active lower reference value, the worker scheduling method of the cloud computing system, characterized in that it further comprises the step of changing the active-ready ready worker to the inactive-ready state. 8. The method of claim 7,
The active worker usage ratio is
A worker scheduling method of a cloud computing system, characterized in that it corresponds to a ratio of the active worker among workers in the active worker in the active-ready state and the active worker. 7. The method of claim 6,
When the inactive worker ratio (INACTIVE WORKER RATIO) exceeds a preset inactive upper reference value, performing a ready-walker down-scaling (READY WORKER DOWN SCALING) to change the inactive-ready ready worker to the template worker Worker scheduling method of the cloud computing system, characterized in that it further comprises. The method according to claim 1,
When the number of template workers is less than a preset allowable limit compared to the maximum number of workers that can be created, performing TEMPLATE WORKER UP SCALING in the background to further create the template workers A method of scheduling workers in a cloud computing system. 7. The method of claim 6,
The worker assignment request period for each function is calculated based on the number of workers required to be assigned during a unit time for each function,
The number of pre-allocated workers corresponds to a value obtained by subtracting the number of ready workers and the active workers in the active-ready state from the number of workers required for a unit time for each function. 12. The method of claim 11,
The number of workers required during the unit time for each function is
A worker scheduling method for a cloud computing system, characterized in that it corresponds to a value obtained by dividing an average execution time of a worker executing a function by a worker assignment request period for each function. Before a worker assignment request to perform a function occurs, a TEMPLATE WORKER is pre-created (PRE-CREATION) to distribute the worker execution preparation load, and the worker assignment request cycle for each function Predicting the number of pre-worker assignments in consideration of the change in , and pre-allocating the ready workers (READY WORKER) by up-scaling the template workers by the number of worker assignments corresponding to the pre-worker assignments (PRE-ALLOCATION) processor; and
Memory to store the worker pool that manages workers for function execution
including,
Worker scheduling apparatus of a cloud computing system, characterized in that the worker execution preparation load is distributed through a process of pre-creating the template worker and a process of pre-allocating the ready worker. 14. The method of claim 13,
the processor
Cloud computing, characterized in that when a worker assignment request for performing the function occurs, the ready worker is assigned as an active worker, and the status of the de-allocated ready worker is changed in consideration of the function service request load The system's worker scheduling device. 14. The method of claim 13,
the processor
Worker scheduling apparatus of a cloud computing system, characterized in that the pre-allocated ready-workers not used for a preset idle period are initialized as the template workers by down-scaling the workers (WORKER DOWN SCALING). 15. The method of claim 14,
The template worker corresponds to a temporary worker configured to correspond to a basic image and resource capacity so that all functions can use it in common, and the ready worker is a worker in a function execution preparation state where resource information setting and resource allocation for function execution are completed. Worker scheduling device of the cloud computing system, characterized in that corresponding to. 17. The method of claim 16,
The status of the ready worker is
A cloud computing system, characterized in that it corresponds to any one of an active-ready state in which the resource allocation and loading of the function file are completed, and an inactive-ready state in which the resource is not allocated but the loading of the function file is completed Worker Scheduling Device. 18. The method of claim 17,
the processor
When the active worker usage ratio (ACTIVE WORKER USAGE RATIO) exceeds a preset active upper reference value, a ready worker up-scaling (READY WORKER UP SCALING) of changing the template worker to the ready worker is performed, and the active worker is used When the ratio is less than a preset active lower reference value, the worker scheduling device of the cloud computing system, characterized in that the change of the active-ready ready worker to the inactive-ready state. 19. The method of claim 18,
The active worker usage ratio is
The worker scheduling apparatus of a cloud computing system, characterized in that it corresponds to a ratio of the active workers among workers in the active worker in the active-ready state and the active workers. 18. The method of claim 17,
the processor
When the inactive worker ratio (INACTIVE WORKER RATIO) exceeds a preset inactive upper reference value, performing the ready-walker down-scaling (READY WORKER DOWN SCALING) to change the inactive-ready ready-worker to the template worker A worker scheduling device in a cloud computing system, characterized in that it.

Images