WO2020124459A1

WO2020124459A1 - Method employing hybrid-cloud computing platform to provide serverless function as a service

Info

Publication number: WO2020124459A1
Application number: PCT/CN2018/122167
Authority: WO
Inventors: 吴楚楠; 刘阳; 温书豪; 马健; 赖力鹏
Original assignee: 深圳晶泰科技有限公司
Priority date: 2018-12-19
Filing date: 2018-12-19
Publication date: 2020-06-25

Abstract

The present invention relates to the technical field of serverless function services. Provided is a method employing a hybrid-cloud computing platform to provide serverless function as a service, comprising the following steps: a serverless function submission process; a serverless function operation process; and a serverless function result and state recovery process. The method employing a hybrid-cloud computing platform to provide serverless function as a service of the present invention is applicable to the field of high-performance computation. The method enables integration of different types of cloud computing services or establishment of an open-source containerized cluster, and provides a serverless function solution.

Description

Invention Name: Service method without service function based on hybrid cloud computing platform

[0001] The present invention belongs to the technical field of serviceless service services, and specifically relates to a serviceless service method based on a hybrid cloud computing platform.

Background technique

[0002] Today, serviceless architecture has become a very popular topic in software architecture design, including the world's three largest cloud service providers-Amazon, Google and Microsoft have invested relatively large R & D costs in the product and ecological construction of serviceless architecture And there are also many domestic products such as Alibaba Cloud Function Computing Platform and Tencent Cloud SCF Serviceless Platform for product promotion and practice on scientific computing and serviceless architecture. At the same time, we have gradually seen more and more literature, materials and open source communities begin to explore various types of serviceless architecture design. We can divide the typical serviceless architecture into two categories:

[0003] The first type is used to describe that some applications host the business logic of the server on a third-party cloud service, or directly establish a connection with the third-party cloud service through the client to complete the logic and state control of the server. Applications with "rich client" type mostly use this kind of architecture. For example, some mobile software implements a complete serviceless application architecture by hosting the data storage logic to Google Firebase service and the authentication logic to the third-party authentication service AuthO . This type of serviceless architecture is also commonly referred to as "(Mobile) Backend as a Service" (hereinafter collectively referred to as BaaS).

[0004] The second type of serviceless architecture is distinguished from the first type in that the logic on the server side is no longer fully hosted by a third-party cloud service provider, but can be designed and written by application developers. Different from the traditional server-side architecture design, this type of server is completely triggered by a stateless container environment through event-driven logic execution, and developers do not need to pay attention to the operation and maintenance of the container environment and resource scheduling issues. This method is usually called "Function as a Service" (hereinafter referred to as FaaS). For example, A WS Lambda service, Tencent Cloud SCF service and Alibaba Cloud function computing service are all encapsulation and implementation of this design.

[0005] Regardless of the above two methods, there are currently the following defects and deficiencies:

[0006] (1) Cloud platform or cloud service provider architecture binding, regardless of BaaS service or FaaS service, its platform design and implementation is strongly tied to the service provider's service stack, but for cross-service, cross-cloud and local storage Support is weak. This increases the difficulty of resource scaling and cost evaluation when using different types of cloud service providers at the same time.

[0007] (2) The requirements of the two types of products for the back-end operating environment are quite strict, the modification and management of environment-dependent configurations are relatively complicated, and the support for custom environments is insufficient.

[0008] (3) There is no unified standard for the data structure and storage model of the two types of products. Each cloud service provider has different solutions and storage services to provide users, resulting in users using different types of serviceless systems to build applications at the same time. It is difficult to uniformly configure and schedule computing resources and aggregation of computing results of different cloud services.

[0009] (4) Both types of cloud service products do not support docking with self-built containerized clusters, and cannot provide those with self-built containerized cluster systems (such as open source Mesos, Kubemetes container management platform) or special computing resources. Users with isolation requirements are provided with no-service architecture solutions.

Summary of the invention

technical problem

Solution to the problem

Technical solution

[0010] In view of the above technical problems, the present invention provides a service method based on a hybrid cloud computing platform without service function, applicable to the field of high-performance computing, through the method can integrate different types of cloud computing services or self-built open source containerization Cluster and provide solutions without service functions.

[0011] The specific technical solution is:

[0012] A service method without a service function based on a hybrid cloud computing platform includes the following steps: a submission process without a service function, an operation process without a service function, a result without service function, and a state recovery process.

[0013] Specifically, the submission process of the serviceless function includes the following steps:

[0014] (1) After the user writes the environment configuration script after the custom function and the function runtime, the function and parameters are converted into character strings through the object serialization method and stored in the form of files, and the definitions that need to be serialized are also defined The programming environment parameters of the function, used to restore the environment dependency during programming development when the server executes the function in reverse sequence

[0015] (2) Initialize the server type without service function, any specified server needs to be allocated with a storage service available; and then the serialized functions, parameters, environment configuration scripts and programming environment parameters The file is uploaded to the cloud storage service of the specified server, and the operation of the serviceless function is triggered OK; a successful start of operation will return a unique serviceless function storage ID, that is, the address of the storage instance of the serviceless function, which contains the serialized file of the serviceless function, operation status, and operation-related data.

[0016] The running process of the serviceless function includes the following steps:

[0017] (1) By parsing the service-free function storage ID in the submission parameters, download all user serialized upload files to the corresponding storage service;

[0018] (2) In accordance with the order of "restore programming development environment dependencies" -> "execution environment configuration script" -> "deserialize user-defined functions and parameters" -> "execute user-defined functions and parameters" This is executed, and the result of the custom function is serialized and stored as a file, and finally uploaded back to the address specified by the storage ID of the no-service function;

[0019] If the operation fails, the error information is captured and serialized, and passed back to the storage service in the same way

[0020] The serviceless function result and state recovery process includes the following steps:

[0021] Through the storage ID of the serviceless function, an instance of the serviceless function equivalent to the original submitter can be recovered, and then the running state of the current serviceless function can be queried through the instance;

[0022] If the operation is successful, the result file can be downloaded from the place specified by the storage ID and the content is obtained in reverse sequence; [0023] If the operation fails, the operation error information can be downloaded in the same manner.

Beneficial effects of invention

Beneficial effect

[0024] The service method based on a hybrid cloud computing platform without service functions provided by the present invention has the following technical effects:

[0025] (1) A variety of non-service products and open source container management platforms can be docked to realize the non-service function scheduling of the hybrid cloud computing platform.

[0026] (2) For dependency management, there is an independent dependency configuration module to support developers through typical Linux

Shell script implements the configuration management of the server-side environment; at the same time, when docking the containerized cluster, it is integrated into any container in the form of a plug-in, turning the container into a running environment without service functions.

[0027] (3) The unified object storage platform persists the state and operation results of all serviceless functions, and helps the serviceless function users to recover the calculation results through the unified object serialization method and SDK. [0028] (4) It can flexibly expand and support open source container management systems, such as Mesos, Kubernetes, or managed container management cloud services such as AWS ECS, etc., to encapsulate traditional containerized clusters into a serviceless function system.

Brief description of the drawings

BRIEF DESCRIPTION

[0029] FIG. 1 is a flowchart of a submission process of a serviceless function of the present invention;

[0030] FIG. 2 is a flowchart of the operation process of the serviceless function of the present invention;

[0031] FIG. 3 is a flowchart of the result of the serviceless function and state recovery process of the present invention;

[0032] FIG. 4 is a schematic diagram of an architecture for docking different cloud services and self-built container clusters of the present invention.

Invention Example

Embodiments of the invention

[0033] The specific technical solutions of the present invention will be described in conjunction with the embodiments.

[0034] A service method based on a hybrid cloud computing platform without a service function can be mainly divided into three processes: a submission process, a running process, a result, and a state recovery process of a serviceless function.

[0035] As shown in FIG. 1, the submission process of no service function:

[0036] First, after the user writes a custom function and an environment configuration script at the time of function runtime, the function and parameters are converted into character strings through object serialization and stored in the form of a file, and there are also definition functions that need to be serialized The programming environment parameter is used to restore the environment dependency during programming development when the server executes the function in reverse sequence. Secondly, the server type that initializes the serviceless function can be a specified third-party cloud service such as AWS

Lambda, Tencent Cloud SCF, etc., or self-built container clusters such as Kubernetes, Mesos, etc. Any specified server needs to be equipped with a storage service that can be used, which can be AWS S3, Tencent Cloud COS, or within the cluster Hard disk storage service. Furthermore, the serialized functions, parameters, environment configuration scripts and programming environment parameter files can be uploaded to the cloud storage service of the specified server, and trigger the operation of the serviceless function. A successful start of operation will return a unique serverless function storage ID, that is, the address of the storage instance of the serverless function, which contains the serialized file of the serverless function, running status, running results and other related data.

[0037] As shown in FIG. 2, the running process of the serviceless function: [0038] First, all user serialized upload files are downloaded to the corresponding storage service by parsing the service-free function storage ID in the submission parameter. Then, follow the order of "Restoring Programming Development Environment Dependencies"->"Execution Environment Configuration Script"->"Deserialize User-Defined Functions and Parameters"->"Execute User-Defined Functions and Parameters", and The result of the custom function is serialized and stored as a file, and finally uploaded back to the address specified by the storage ID of the no-service function. If the operation fails, the error information is captured and serialized, and sent back to the storage service in the same way.

[0039] As shown in FIG. 3, the result of no service function and the state recovery process:

[0040] Because the state recovery end of the serviceless function may not be the task submitting end, the serviceless function instance equivalent to the original submitting end can be recovered through the storage ID of the serviceless function, and then the current serviceless Query the running status of the function. If the operation is successful, the result file can be downloaded from the location specified by the storage ID and the content is obtained in reverse sequence; if the operation fails, the operation error information can be downloaded in the same way.

[0041] As shown in FIG. 4, a service method based on a hybrid cloud computing platform without a service function is an architecture that interfaces with different cloud services and self-built container clusters. Among them, the user serializes the custom function and the parameter list according to the description method of the submission process of the serviceless function, and uploads it to the specified cloud storage through the storage adapter module. The storage adapter module abstracts the addition, deletion, modification and inspection interfaces in the storage process to realize the connection of object storage services of different cloud service providers, or the hard disk storage process of self-built containerized clusters. At the same time, the scheduling adapter module is used to abstract the relevant interfaces of different cloud services and self-built containerized clusters in the process of task submission and management. Therefore, users can specify the cloud services or containerized cluster resources that need to be used during the initialization process of the storage adapter and the scheduling adapter, and the hybrid cloud computing platform without service functions can be supported without modifying custom functions and parameters.

[0042] This service method based on a hybrid cloud computing platform without service functions, mainly based on AWS Lambda and self-built Mesos containerized cluster, based on Python 2.7.13 developed without service function storage adapter and scheduling adapter to complete the above two A hybrid serviceless architecture for cloud computing platforms. Among them, AWS Lambda's cloud service storage uses AWS S3, and the self-built Mesos cluster is built on the AWS EC2 virtual machine, so it can also use AWS S3 as its storage service without service functions.

[0043] The user can configure and install the entire client through the Python environment. After the user finishes writing the custom functions and parameters of Python on the machine, the user can call the storage adapter stomgiengine of the client to change the current environment. Lay relationships, custom functions, custom parameter lists, and running environment configuration scripts are serially uploaded to the designated bucket of AW S S3. The directory name of the bucket is a randomly assigned global unique string ID, which is used to help users mark different Instance of a serviceless function call. After completing the above serviceless function storage process, you can submit the serviceless function through the scheduling adapter Executor. The executor encapsulates the underlying scheduling interface for docking with different cloud services and containerized clusters, and the scheduling interface is abstracted through Invoker and exposed to the Executor for invocation. . At this time, the user can submit the unique ID returned by the serverless function to monitor the status of the serverless function, and after the operation is successfully completed, download and deserialize the operation result of the serverless function through the storage adapter storage_engin e, that is, the user The local custom function is consistent with the result of the parameter execution. Otherwise, the operation error information is obtained to help the user to troubleshoot the function definition and runtime errors

Claims

Claims [Claim 1] A service method for a serviceless function based on a hybrid cloud computing platform, characterized in that it includes the following steps: submission process of a serviceless function, operation process of a serviceless function, results and status of a serviceless function Recycling process. [Claim 2] The service method without service functions based on the hybrid cloud computing platform according to claim 1, characterized in that the submission process of the serviceless functions includes the following steps:

(1) After the user has written a custom function and the environment configuration script at the time of function runtime, the function and parameters are converted into character strings by object serialization and stored in the form of files. At the same time, serialization and programming of defined functions are also required Environment parameter, used to restore the environment dependency during programming development when the function is executed in reverse sequence on the server side

(2) Initialize the server type without service function, any specified server needs to be allocated with a storage service that can be used; and then the serialized functions, parameters, environment configuration scripts and programming environment parameter files can be uploaded to Specify the server-side cloud storage service, and trigger the operation of the serviceless function; successful start will return a unique serviceless function storage ID, that is, the storage instance address of the serviceless function, which contains the serialized file of the serviceless function, Data related to operation status and operation results.

[Claim 3] The service method of service-free function based on a hybrid cloud computing platform according to claim 1, characterized in that the operation process of the service-free function includes the following steps:

(1) By parsing the service-free function storage ID in the submission parameters, download all user serialized upload files to the corresponding storage service;

(2) Perform this in the order of "Restoration of programming development environment dependencies" -> "Execution environment configuration script" -> "Deserialize user-defined functions and parameters" -> "Execute user-defined functions and parameters", Serialize the result of the custom function and store it as a file, and finally upload it back to the address specified by the storage ID of the no-service function;

If the operation fails, the error information is captured and serialized, and then passed back to the storage service in the same way.

[Claim 4] The service method of service-free function based on a hybrid cloud computing platform according to claim 1, characterized in that the process and state recovery process of the service-free function result includes the following steps Step:

Through the storage ID of the serviceless function, an instance of the serviceless function equivalent to the original submitter can be restored, and then the running status of the current serviceless function can be queried through the instance; if the operation is successful, the location specified by the storage ID can be retrieved Download the result file and get the content in reverse sequence;

If the operation fails, the operation error information can be downloaded in the same way.