CN112860317B - Call pool method and system in serverless computing system - Google Patents
Call pool method and system in serverless computing system Download PDFInfo
- Publication number
- CN112860317B CN112860317B CN202011636030.3A CN202011636030A CN112860317B CN 112860317 B CN112860317 B CN 112860317B CN 202011636030 A CN202011636030 A CN 202011636030A CN 112860317 B CN112860317 B CN 112860317B
- Authority
- CN
- China
- Prior art keywords
- caller
- load
- state
- marked
- computing system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Telephonic Communication Services (AREA)
Abstract
A method for pooling callees in a serverless computing system includes setting a redirection table in the serverless computing system, after a controller of the serverless computing system selects a callee, randomly positioning a callee with an available state in the redirection table according to a group of the callees selected by the controller, and outputting information of the selected callee to replace initial selection information for control so as to realize load balance of the callees in the same pool. The invention can obviously improve the efficiency of the serverless computing system, save memory resources and reduce the waiting time of the application.
Description
Technical Field
The invention relates to a technology in the field of internet information processing, in particular to a caller pooling method and system in a serverless computing system.
Background
Serverless computing (serverless computing) means that a user does not need to pay attention to the deployment configuration of a server, only needs to pay attention to a specifically required service, and an operator is specifically responsible for an underlying operating system and hardware facilities and runs the service of the user in a software container (software container) environment.
The invoker is an entity for processing the invocation in the serverless computing, a current invoker exists in a serverless computing system, a current container is maintained in the invoker, and the application runs in the container in the invoker. However, different applications require different container environments that need to be configured before running, which is referred to as cold start of the container. The cold start requires a lot of time and occupies memory resources, which has a great influence on the efficiency of the containerized application.
Because the capacity of a caller is limited, and the task allocation mechanism of the current serverless computing system is in units of callers, the current serverless computing system cannot exhibit flexibility in the face of variable load strengths, which is specifically represented by: 1) when the load intensity is higher than the current system capacity, the container is cold started on a large scale, and the system efficiency is reduced. 2) The load intensity is lower than the current system capacity, a large number of containers are idle, and the system efficiency is reduced. Meanwhile, the current naive caller selection strategy also enables the same type of application to be more easily dispersed into a plurality of callers for processing, and reduces the efficiency of the system.
Disclosure of Invention
The invention provides a caller pooling method and system in a serverless computing system, aiming at the problems of resource waste, long processing delay and the like caused by cold start and idle occupation in the existing serverless computing system, realizing pooling management of the caller by a redirection table, breaking through the upper limit of the capacity of a single caller, realizing aggregation processing of the same type of application, and realizing dynamic and flexible adjustment of the caller pool capacity by copying, closing and load migration of the caller so as to reduce performance loss caused by cold start and idle waiting.
The invention is realized by the following technical scheme:
the invention relates to a caller pooling method in a serverless computing system, which is characterized in that a redirection table is arranged in the serverless computing system, after a controller of the serverless computing system selects a caller, the redirection table randomly positions a caller with an available state according to a group of the caller selected by the controller, and the information of the selected caller is output to replace the initial selection information for control, so that the load balance of the callers in the same pool is realized.
The redirection table comprises: a current trunk group caller for pooling management, each caller having states of available and unavailable, wherein: the redirection table selects only invokers whose state is available when redirecting the load.
The redirection table reflects whether the caller can be selected to execute the task of processing load, and the caller status table reflects whether the caller needs to be adjusted or can be adjusted with other callers.
The redirection table periodically updates the state of the caller in a pooling management mode, and the specific steps comprise:
1) periodically checking, by the capacity control module, the caller status table and obtaining callers in which a jam condition exists and which are marked as unavailable;
the state table of the calling device comprises: a blocked or idle state and a load type, wherein: the blocked or idle state refers to whether the length of the caller's load queue exceeds a set threshold and causes the task to be blocked or not.
The idle state is as follows: and the invoker without load is characterized in that the invoker is marked as unavailable in the redirection table, and the load type is marked as "-" in the invoker state table.
2) Copying container-related configuration information related to execution calls in the caller in the blocking state to the caller in the idle state;
3) and updating the redirection table.
The replication specifically includes: the method comprises the steps of initiating a copy request to a calling device in a blocking state in a serverless computing system, transferring container related configuration information subordinate to the calling device to a cache region, sending a request reading cache region to the calling device marked as unavailable of a target, and finally clearing the cache region.
The updating redirection table is as follows: and adding the target idle caller in the redirection table into the caller group where the source blocking caller is positioned, and marking the target idle caller as available.
The container-related configuration information is: the container handles the specific code environment that the load requires, such as a specific Python version that certain applications require, etc.
The idle no-load invoker is marked as an unavailable invoker in a redirection table. Only after participating in the copy operation will it be marked as an available caller.
Preferably, the load intensity of the current invoker is judged by judging the number of idle containers, and when the number of idle containers exceeds a set threshold, the low-load invoker is closed, and the state of the low-load invoker is marked as unavailable, that is, a new load is not accepted any more to avoid waste of resources.
It is further preferred that the shutdown operation is not performed when the low-load caller is the last caller in any caller group to avoid cold start.
Preferably, the load intensity of the current invoker is judged by judging the number of idle containers, when the number of idle containers exceeds a set threshold value, most of the containers belonging to the invoker are in an idle state to consume resources, the low-load invoker is closed, all containers in the invoker are closed and the states of the containers are marked as unavailable after the configuration information of the containers in the invoker is transferred to a cache region, and new loads are not accepted any more to avoid waste of the resources; and simultaneously transferring the information in the cache region to a new target caller with enough capacity, namely selecting a caller with enough capacity from other groups according to the caller state table and combining the two caller groups in the redirection table.
The invention relates to a pooling module for implementing the method, which comprises the following steps: capacity control module, buffer and communication module, wherein: the communication module periodically inquires the states of other callers in the serverless computing system, periodically updates a caller state table according to the periodically updated caller state table and transmits an instruction of the capacity control module to a specific caller, the capacity control module periodically checks load information, container operation information and load types of the callers in the caller state table, judges whether the caller needs to be copied, closed or load migrated, and sends out an update instruction of a redirection table according to the system state of the serverless computing system, the communication module updates the state of the caller in the redirection table according to the instruction of the capacity control module, and the cache area is used as a transfer cache when the file migration between the callers needs to occur, so that the direct communication between the callers is avoided.
The state table of the calling device comprises: the load strength, load type, number of caller pool, and internal container running state of each caller in the serverless computing system.
And when the low-load calling device inquired from the calling device state table by the capacity control module is not the last available calling device in the calling device group, closing the calling device, otherwise, needing a target migration calling device with enough capacity and then carrying out migration operation.
The migration operation is as follows: the capacity control module sends a reading request to the source calling device through the communication module, reads container configuration information in the source calling device into a cache region, then sends a closing instruction to the source calling device to close all idle containers, and when part of the containers are still in a running state, the containers are closed after the running is finished; the communication module sends a request to update the redirection table, marking the source invoker as unavailable.
The migration operation is as follows: and sending the container configuration information in the cache region to a target caller for the early initialization of the container, sending a request to update a redirection table by a communication module, marking a source caller as unavailable, and combining the caller groups of the source caller and the target caller in the redirection table.
Technical effects
The invention integrally solves the problems of resource waste, long processing delay and the like caused by cold start and idle occupation in a non-service period computing system. The invention adopts a pooling management mode for the invokers through the redirection table, and in order to realize the dynamic balance of load and capacity, the capacity management module performs operations of copying, closing, load migration and the like on the invokers. Compared with a native server-free computing system, the system with the added pooling management module has the advantages that the memory resource usage of the same task executed by the system is reduced by 25%, and the task waiting time is reduced by 19%.
Compared with the prior art, the method and the system realize the balance of the capacity and the load by calling the redirection table pooling management invoker and controlling the copying, closing and load migration of the invoker through the capacity control module, and the ratio of the cold start of the container to the idle time is reduced.
Drawings
FIG. 1 is a block diagram of the system of the present invention.
Fig. 2 shows the difference between the load balancing table and the redirection table in the original server-less computing system.
FIG. 3 is the caller state table.
Detailed Description
Fig. 1 shows a pooling module and a serverless computing system connected thereto according to the embodiment, where the serverless computing system includes a controller and a caller including a plurality of containers.
When the load reaches the controller of the server-free computing system, the controller selects a healthy caller without losing response due to network failure and the like for processing according to the load type, the selection of the controller is redirected through a redirection table in the pooling module to point to a new caller N, the controller puts specific execution information of the load into a message queue of the caller N to serve as an execution queue for waiting for processing by the caller N, when a container to which the controller belongs is idle, the load at the top of the queue is selected for processing, when the container is selected, the container depending on environment matching is preferentially selected for processing, after the load processing is finished, the caller returns a processing result, and processing records are also stored in a database in the system.
When a free container exists in a caller in the serverless computing system, the pooling module inquires a message queue marked with a caller N, when a load is in the queue, a proper container is selected for the load at the top of the queue, the highest priority is the free container matched with the load type, the second highest priority is the running container matched with the load type, the current load is run after waiting for the running of the running container, and finally a new container is cold started for processing; when the treatment is finished, the container environment is reserved for a set time, and after the threshold value is exceeded, the container is closed.
In addition, after receiving the information of the pooling module, corresponding operation is performed, and when the container information is copied or the container is closed, the container is still in a running state, and the operation is continued after the running is finished.
As shown in fig. 1, the pooling module according to the present embodiment includes: capacity control module, buffer and communication module, wherein: the capacity control module periodically checks the state table of the invokers, and when the load queue of a certain invoker exceeds a set threshold value, the invoker is marked as a blocked invoker. When the number of idle containers in a certain calling device exceeds a set threshold value, the calling device is marked as a low-load calling device, and when the calling device marked as blocked and the calling device marked as unavailable exist at the same time, the operation of copying the calling device is carried out, and a redirection table is updated correspondingly; when a certain calling device is marked as a low-load state and the only calling device which is not in the calling device group is marked as an available calling device, carrying out calling device closing operation; when a certain caller is marked as a low-load state and the caller is the only caller marked as available in the caller group, the capacity control module queries the caller state table to select a proper target caller and performs the caller migration operation.
The load corresponds to five different applications in the serverless computing system as follows: autocomplete, markdown, img-resize, sentient, ocr-img. In this embodiment, the numbers "0", "1", "2", "3", and "4" are sent to the system in a randomly selected manner.
In this embodiment, the serverless computing system is implemented based on the OpenWhisk, and includes 8 invokers, each memory is 1GB, the number of containers that can be accommodated by each invoker is 4, the memory of each container is 256MB, and 5 initial invoker groups are set for the redirection table.
The caller set in this embodiment determines whether the threshold of the queue length for sending the copy request is 5, and the caller set determines whether the threshold of the number of free containers for sending the close request is 3.
The period of time for updating the caller status table and checking the capacity control module is set to 1min in the embodiment.
In this embodiment, the period of each group of embodiments is 20min, and the load sending method is to send a random type load request to the system with a random load intensity, but the total number of sent loads is constant.
Through specific practical experiments, under the above specific environment setting, a request is sent to the system with a load intensity of 20 calls per second, and the obtained experimental data is as follows: under the same load processing condition, the average memory occupation of the system after the pooling management module is added is 5.6GB per second, and compared with 7.5GB per second of the original serverless computing system, the memory usage is reduced by 25%. Meanwhile, the average latency of the system load execution after the pooling management module is added is 575ms, which is about 19% lower than 709ms of the native serverless computing system.
In summary, the present invention changes the adjustment for a certain caller in the conventional serverless computing system into the adjustment for a certain group of callers by invoking the redirection table pooling management caller, thereby breaking through the upper limit of the capacity of a single caller and realizing the flexibility of the capacity. At the same time, aggregating applications of a type in the same set of invokers also reduces unnecessary cold starts. The invention realizes the balance of the capacity and the load by controlling the copying, closing and load migration of the calling device through the capacity control module, flexibly adjusts the capacity of the calling device group, avoids resource waste caused by overlarge capacity setting and performance deficiency caused by undersize capacity setting, realizes the reduction of the ratio of cold start and idle time of the container, and reduces the waiting time of application.
The foregoing embodiments may be modified in many different ways by those skilled in the art without departing from the spirit and scope of the invention, which is defined by the appended claims and all changes that come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.
Claims (9)
1. A caller pooling method in a serverless computing system is characterized in that a redirection table is arranged in the serverless computing system, after a controller of the serverless computing system selects a caller, the redirection table randomly positions a caller with an available state according to a group of the caller selected by the controller, and the information of the selected caller is output to replace the initial selection information for control, so as to realize the load balance of the callers in the same pool;
the redirection table comprises: a current trunk group caller for pooling management, each caller having states of available and unavailable, wherein: when the redirection table redirects the load, only the invoker with the available state in the invoker state table is selected;
the state table of the calling device comprises: a blocked or idle state and a load type, wherein: the blocked or idle state refers to whether the length of the load queue of the caller exceeds a set threshold value and causes the task to be blocked or not; initiating a replication request to a caller in a blocking state in a serverless computing system, transferring container related configuration information subordinate to the caller to a cache region, sending a request reading cache region to the caller marked as unavailable of a target, and finally clearing the cache region;
the redirection table is updated by adding the target idle caller in the table into the caller group where the source blocking caller is located and marking the target idle caller as available.
2. The method of claim 1, wherein the redirection table reflects whether a caller can be selected to perform a processing load task and the caller status table reflects whether it needs to be adjusted or whether it can be adjusted with other callers.
3. The method as claimed in claim 1, wherein the redirection table periodically updates the state of the caller in the redirection table in a pooling management manner, and comprises the following steps:
1) periodically checking, by the capacity control module, the caller status table and obtaining callers in which a jam condition exists and which are marked as unavailable;
2) copying container-related configuration information related to execution calls in the caller in the blocking state to the caller in the idle state;
3) and updating the redirection table.
4. The method of claim 1, wherein the idle state, i.e., unloaded, invokers are marked as unavailable invokers in the redirection table and are only marked as available invokers after participating in the copy operation.
5. The method as claimed in claim 1, wherein the caller pooling method in the serverless computing system is characterized in that the load intensity of the current caller is determined by determining the number of idle containers, and when the number of idle containers exceeds a set threshold, i.e. a low load, the low load caller is turned off and the status thereof is marked as unavailable, i.e. no new load is accepted to avoid wasting resources.
6. The method of claim 5, wherein when the low load caller is the last caller in any caller group, no shutdown is performed to avoid cold start.
7. The caller pooling method in the serverless computing system of claim 1, wherein the load intensity of the current caller is determined by determining the number of idle containers, when the number of idle containers exceeds a predetermined threshold, most of the containers belonging to the caller are in an idle state to consume resources, the caller is closed, after transferring the configuration information of the internal containers to the cache area, all the internal containers are closed and the states thereof are marked as unavailable, i.e., no new load is accepted to avoid waste of resources; and simultaneously transferring the information in the cache region to a new target caller with enough capacity, namely selecting a caller with enough capacity from other groups according to the caller state table and merging the current caller and the target caller in the redirection table.
8. A pooling module for implementing the method of any of the above claims 1-7, comprising: capacity control module, buffer and communication module, wherein: the communication module periodically inquires the states of other callers in the serverless computing system, periodically updates a caller state table according to the periodically updated caller state table and transmits an instruction of the capacity control module to a specific caller, the capacity control module periodically checks load information, container operation information and load types of the callers in the caller state table, judges whether the caller needs to be copied, closed or load migrated, and sends out an update instruction of a redirection table according to the system state of the serverless computing system, the communication module updates the state of the caller in the redirection table according to the instruction of the capacity control module, and the cache area is used as a transfer cache when the file migration between the callers needs to occur, so that the direct communication between the callers is avoided.
9. The pooling module of claim 8, wherein the capacity control module periodically checks the caller status table, marks a caller as a blocked caller when the load queue of the caller exceeds a set threshold, marks the caller as a low load caller when the number of free containers in the caller exceeds a set threshold, and performs caller replication and updates the redirection table accordingly when there are both a caller marked as blocked and a caller marked as unavailable; when a certain calling device is marked as a low-load state and the only calling device which is not in the calling device group is marked as an available calling device, carrying out calling device closing operation; when a certain caller is marked as a low-load state and the caller is the only caller marked as available in the caller group, the capacity control module queries the caller state table to select a proper target caller and performs the caller migration operation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011636030.3A CN112860317B (en) | 2020-12-31 | 2020-12-31 | Call pool method and system in serverless computing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011636030.3A CN112860317B (en) | 2020-12-31 | 2020-12-31 | Call pool method and system in serverless computing system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112860317A CN112860317A (en) | 2021-05-28 |
| CN112860317B true CN112860317B (en) | 2022-03-29 |
Family
ID=76000641
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011636030.3A Active CN112860317B (en) | 2020-12-31 | 2020-12-31 | Call pool method and system in serverless computing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112860317B (en) |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105430027A (en) * | 2014-09-04 | 2016-03-23 | 中国石油化工股份有限公司 | Load balance dynamic pre-allocating method based on a plurality of resource scales |
| CN111381936B (en) * | 2020-03-23 | 2023-03-31 | 中山大学 | Method and system for allocating service container resources under distributed cloud system-cloud cluster architecture |
| CN111683117B (en) * | 2020-05-11 | 2021-12-10 | 厦门潭宏信息科技有限公司 | Method, equipment and storage medium |
-
2020
- 2020-12-31 CN CN202011636030.3A patent/CN112860317B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN112860317A (en) | 2021-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9882975B2 (en) | Method and apparatus for buffering and obtaining resources, resource buffering system | |
| KR100383381B1 (en) | A Method and Apparatus for Client Managed Flow Control on a Limited Memory Computer System | |
| US7349970B2 (en) | Workload management of stateful program entities | |
| US6377975B1 (en) | Methods and systems to distribute client software tasks among a number of servers | |
| CN1095120C (en) | Computer system having client-server architecture | |
| US20070038837A1 (en) | Merging identical memory pages | |
| CN1256666C (en) | Dynamically downloading and executing system services on a wireless device | |
| KR20040105685A (en) | Processor system, task control method on computer system, computer program | |
| US11893029B2 (en) | Real-time streaming data ingestion into database tables | |
| Alonso et al. | Sharing jobs among independently owned processors | |
| CN109725920A (en) | A kind of update method of Service Instance, device and storage medium | |
| US20050021511A1 (en) | System and method for load balancing in database queries | |
| US20020095471A1 (en) | Method of transferring data between memories of computers | |
| CN112600761A (en) | Resource allocation method, device and storage medium | |
| CN112860317B (en) | Call pool method and system in serverless computing system | |
| CN111160911B (en) | Intelligent contract calling frequency control method for block chain | |
| US20220222013A1 (en) | Scheduling storage system tasks to promote low latency and sustainability | |
| KR20070014724A (en) | Memory controlling method for mobile communicationn terminal | |
| CN113946528A (en) | Method and device for switching system architecture | |
| EP1107554B1 (en) | Method and system for adaptively allocating call-related tasks | |
| JP5257077B2 (en) | Resource allocation apparatus, method and program | |
| CN1604528A (en) | Cooperative starting method for network computer | |
| EP0789881B1 (en) | System and method for processing of data and a communications system with such a system | |
| JP2006277047A (en) | Data processing device and dynamic substitution method of application program in multithread system | |
| CN114124680B (en) | File access control alarm log management method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |