CN113556375A

CN113556375A - Cloud computing service method and device, electronic equipment and computer storage medium

Info

Publication number: CN113556375A
Application number: CN202010340866.2A
Authority: CN
Inventors: 颜廷帅; 王涛; 赵宇
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2020-04-26
Filing date: 2020-04-26
Publication date: 2021-10-26

Abstract

The embodiment of the invention provides a cloud computing service method and device, electronic equipment and a computer storage medium. The cloud computing service method comprises the following steps: receiving a model service request initiated by a user; allocating an execution node for the model service request; processing the model service request by using a container management application instance on the execution node, wherein the container management application instance at least comprises the following containers: initializing a container and a model container; and returning the processing result to the user. In the scheme of the embodiment of the invention, the initialization container and the model container keep the independence of the containers and can be updated or maintained independently, so that the updating flexibility of the computing platform is improved.

Description

Cloud computing service method and device, electronic equipment and computer storage medium

Technical Field

The embodiment of the invention relates to the technical field of computers, in particular to a cloud computing service method and device, electronic equipment and a computer storage medium.

Background

Due to the reasoning capability of the artificial intelligence algorithm model, more and more users hope to improve the service value of the users by the aid of the capability of the algorithm model. Generally, a complete machine learning system involves multiple links including data capture, data cleaning, model training, model tuning, model servitization, and the like. Therefore, the platform of the algorithm model is provided through the network technology, the application service and the algorithm model service are deployed on one server and serve as a computing platform to provide services to the outside, and the deployment cost can be greatly reduced. However, this solution requires that the application code and the algorithm code be released simultaneously when updating the computing platform.

Disclosure of Invention

In view of this, embodiments of the present invention provide a cloud computing service method, apparatus, electronic device, and computer storage medium, so as to improve flexibility of updating a computing platform.

According to a first aspect of the embodiments of the present invention, there is provided a cloud computing service method, including: receiving a model service request initiated by a user; allocating an execution node for the model service request; processing the model service request by using a container management application instance on the execution node, wherein the container management application instance at least comprises the following containers: initializing a container and a model container; and returning the processing result to the user.

According to a second aspect of the embodiments of the present invention, there is provided a cloud computing service method, including: receiving a model service request sent aiming at a container management application instance, wherein the container management application instance comprises a service container and a model container, the service container is used for providing application service, and the model container is used for running a prediction model; responding to the model service request, and calling the prediction model in the model container for prediction through the service container; and returning the prediction result of the prediction model.

According to a third aspect of the embodiments of the present invention, there is provided a cloud computing service method, including: receiving a prediction model deployment request sent aiming at the container management application instance, wherein the container management application instance comprises a service container which is used for providing application service; responding to the prediction model deployment request, creating a second container and an initialization container in the container management application instance, wherein the model container is used for running a prediction model, and the initialization container is used for obtaining the prediction model; and acquiring the prediction model into the model container through the initialization container so as to call the prediction model in the model container for prediction through the service container.

According to a fourth aspect of the embodiments of the present invention, there is provided a cloud computing service method, including: receiving a neural network model service request sent aiming at a container management application instance, wherein the model service request comprises semantic data to be predicted, the container management application instance comprises a service container and a model container, the service container is used for providing application service, and the model container is used for operating a neural network prediction model; responding the model service request, calling a prediction model in the model container through the service container, and performing semantic processing on the semantic data to be predicted to obtain a semantic processing result; and returning a prediction result comprising the semantic processing result.

According to a fifth aspect of the embodiments of the present invention, there is provided a cloud computing service apparatus including: the request receiving module is used for receiving a model service request initiated by a user; the node distribution module is used for distributing execution nodes for the model service request; a request processing module, which processes the model service request by using a container management application instance on the execution node, wherein the container management application instance at least comprises the following containers: initializing a container and a model container; and the processing result returning module returns the processing result to the user.

According to a sixth aspect of the embodiments of the present invention, there is provided a cloud computing service apparatus including: the model service request receiving module is used for receiving a model service request sent aiming at a container management application instance, wherein the container management application instance comprises a service container and a model container, the service container is used for providing application service, and the model container is used for operating a prediction model; the container calling module responds to the model service request and calls a prediction model in the model container to predict through the service container; and the prediction result returning module returns the prediction result of the prediction model. .

According to a seventh aspect of the embodiments of the present invention, there is provided a cloud computing service apparatus including: the deployment request receiving module is used for receiving a prediction model deployment request sent by aiming at the container management application instance, wherein the container management application instance comprises a service container which is used for providing application service; the container creating module is used for responding to the prediction model deployment request by a container, creating a model container and an initialization container in the container management application instance, wherein the model container is used for running a prediction model, and the initialization container is used for acquiring the prediction model; and the model acquisition module acquires the prediction model into the model container through the initialization container so as to call the prediction model in the model container for prediction through the service container.

According to an eighth aspect of the embodiments of the present invention, there is provided a cloud computing service apparatus including: the device comprises a model service request receiving module, a neural network model service request processing module and a prediction module, wherein the model service request comprises semantic data to be predicted, the container management application instance comprises a service container and a model container, the service container is used for providing application service, and the model container is used for operating a neural network prediction model; the semantic processing module responds to the model service request, calls a prediction model in the model container through the service container, and performs semantic processing on the semantic data to be predicted to obtain a semantic processing result; and the prediction result returning module returns the prediction result comprising the semantic processing result.

According to a ninth aspect of embodiments of the present invention, there is provided an electronic apparatus, including: one or more processors; a computer readable medium configured to store one or more programs which, when executed by the one or more processors, cause the one or more processors to carry out the method of any one of the first to fourth aspects.

According to a tenth aspect of embodiments of the present invention, there is provided a computer readable medium, on which a computer program is stored, which when executed by a processor, implements the method according to any one of the first to fourth aspects.

In the scheme of the embodiment of the invention, the initialization container and the model container keep the independence of the containers and can be updated or maintained independently, so that the updating flexibility of the computing platform is improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the embodiments of the present invention, and it is also possible for a person skilled in the art to obtain other drawings based on the drawings.

Fig. 1A is a schematic diagram of a network architecture to which a cloud computing service method according to an embodiment of the present invention is applied;

FIG. 1B is a schematic flow chart diagram of a cloud computing service method according to another embodiment of the present invention;

fig. 1C is a schematic diagram of a service deployment phase of a cloud computing service method according to another embodiment of the present invention;

fig. 1D is a schematic diagram of a predicted service phase of a cloud computing service method according to another embodiment of the present invention;

FIG. 1E is a schematic flow chart diagram of a cloud computing service method according to another embodiment of the present invention;

FIG. 2A is a schematic flow chart diagram of a cloud computing service method according to another embodiment of the present invention;

fig. 2B is a schematic diagram illustrating a predicted service phase of a cloud computing service method according to another embodiment of the present invention;

FIG. 3 is a schematic flow chart diagram of a cloud computing service method of another embodiment of the present invention;

FIG. 4 is a schematic flow chart diagram of a cloud computing service method of another embodiment of the present invention;

fig. 5 is a schematic block diagram of a cloud computing service apparatus of another embodiment of the present invention;

fig. 6 is a schematic block diagram of a cloud computing service apparatus of another embodiment of the present invention;

fig. 7 is a schematic block diagram of a cloud computing service apparatus of another embodiment of the present invention;

fig. 8 is a schematic block diagram of a cloud computing service apparatus of another embodiment of the present invention;

FIG. 9 is a schematic block diagram of an electronic device of another embodiment of the present invention;

fig. 10 is a hardware configuration of an electronic device according to another embodiment of the present invention.

Detailed Description

In order to make those skilled in the art better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments of the present invention shall fall within the scope of the protection of the embodiments of the present invention.

Fig. 1A is a schematic diagram of a network architecture to which a cloud computing service method according to an embodiment of the present invention is applied. As shown in the figure, in the network architecture, the cloud computing service method may be implemented by the client 30 and the cloud computing server 10 and the server 20. In this example, the number of servers and the number of clients shown in the figure are merely exemplary, and it should be understood that the number of servers and clients may be arbitrary. In this example, the server 10 may be implemented as one or more data processing apparatuses or other computing devices, the system architecture of which includes server hardware 11, a hosting system 12, a cloud computing service container 131, and a cloud computing service container 132. It should be understood that the host system 12 may invoke a hardware interface, such as a hardware virtual layer interface, etc., of the server hardware 11. The cloud computing service container 131 and the cloud computing service container 132 are installed in the host system 12, and the containers may access the storage resources of the server 10 or may be configured with their own storage resources. For example, either one of the container 131 and the container 132 may access a storage resource of the server 10, and the other may access a storage resource configured by itself. Similarly, the server 20 may be implemented as one or more data processing devices or other computing devices. The host system 22 may invoke a hardware interface of the server hardware 21, schedule or control computing resources of the hardware 21. The cloud computing service container 23 is installed in the host system 22. The container 23 may access the storage resource of the server 20, or may be configured with its own storage resource. The server side hardware 11 and 12 includes, but is not limited to, a multiprocessor CPU architecture, a memory system, and the like. The multiprocessor CPU architecture described above includes a processor capable of performing vector calculation, scalar calculation, tensor calculation.

The host system 12 and the host system 22 may be the same operating system or different operating systems. The host system 12 and the host system 22 may be a server operating system such as LINUX. Although managed as a function of a server, the operating system described above may be a desktop operating system or an embedded operating system. In addition, host system 12 and host system 22

It should also be understood that each of the server 10 and the server 20 may be implemented as at least one physically separate plurality of servers. It may also be implemented as multiple virtual servers installed in the same physical server. For example, the server 10 and the server 20 may be installed in the same physical server as different virtual servers. In addition, the server 10 and the server 20 can be in different physical locations and may have different capabilities and computing architectures.

It should also be understood that the client 30 may be a desktop computer, a mobile terminal device, or the like. The operating system 32 may be a desktop operating system or an embedded operating system, among others. The client application may send a request to the server 10 or the server 20 and receive results returned by the server processing based on the request.

In this example, client 30 may communicate with server 10 and server 20 over network 40 through link 50. It should be understood that in other examples, client 30 may also communicate with server 10 and server 20 bypassing network 40. In addition, the server 10 and the server 20 may communicate through a network such as the network 40. For example, the server 10 and the server 20 can communicate through an internal network. The communications described above may be based on the same network, and may be heterogeneous, for example, communicating with each other or other devices through one or more gateways. Additionally, the network 40 includes, but is not limited to, a wired network such as ethernet or a wireless network such as WIFI (wireless fidelity).

In this example, the cloud computing service containers 131 and 132 of the server 10 and the cloud computing service container 23 of the server 20 may include the application service and the algorithm model therein. For example, the algorithm model may be a prediction model such as Natural Language Processing (NLP) and Machine Translation (MT), or may be a prediction model of computer vision services. The algorithmic model may include processing modules for semantic processing or computer vision processing. The application service may be responsible for data download, authentication, current limiting, statistics, and intervention functions. In addition, the container may further include an engine for tensor acceleration or vector acceleration, and may further include maintenance tools and development tools such as project management, compilation debugging, flow arrangement, and log management.

In this example, the user can customize the model through the client application 33, and can directly call an API (application program interface) of a general online service on the

server

10 or 20 to acquire capabilities such as natural language processing and machine translation, in other words, a prediction service based on the capabilities. In addition, the user can also provide data to train the prediction model, and then the model is subjected to service, so that further service customization is realized.

The following further describes specific implementation of the embodiments of the present invention with reference to the drawings.

Fig. 1B is a schematic flowchart of a cloud computing service method according to another embodiment of the present invention. The cloud computing service method provided by the embodiment of the invention can be applied to execution nodes (for example, resource object execution nodes) in a cloud computing service network. In a cloud computing services network, resource object execution nodes may communicate with clients to provide requested specific services (e.g., services provided based on a predictive model) for the clients. In addition, the server of the cloud computing services network may include an execution node and a management node (e.g., a resource object management node). The resource object management node may manage at least one resource object execution node. For example, a resource object execution node may obtain a service request (e.g., a model service request) from a client and generate a sub-request for the resource object execution node according to the service sentiment. The resource object execution node may return a service result (e.g., a predicted result) or an intermediate result to the resource object management node in response to the sub-request. The resource object execution node may forward the service result to the client; or processing the intermediate result to obtain a service result and returning the service result to the client. The method of FIG. 1B includes:

110: a model service request initiated by a user is received.

120: an execution node is allocated for the model service request.

130: processing the model service request by using a container management application instance on the execution node, wherein the container management application instance at least comprises the following containers: initializing the container and model container.

140: and returning the processing result to the user.

It should be understood that the cloud computing service methods herein may be implemented based on a cloud service platform, e.g., based on a cloud native service platform. The overall architecture of the cloud service platform can be divided into two layers of application service and model service. The application service is responsible for user-related service logic, such as data management, project management, user management, and the like. The cloud computing service method provided by the embodiment of the invention can be applied to cloud native computing service. The computing service is responsible for resource scheduling, model training, model deployment and model online service. The computing service is integrally deployed in the resource object management cluster network, and particularly provides the work of resource scheduling, service discovery, load balancing, health check and the like. For example, a resource object management cluster network may include resource object execution nodes and resource object management nodes (e.g., cloud-native object management hosts). The service container may be referred to as an application service container or a platform service container. The model container may be referred to as a predictive model container or a model service container.

A container management application instance herein may be an object for creating containers, deleting containers, organizing containers, managing containers, and the like. The container management application instance may be implemented as a specific abstraction layer in the execution node. The container in the container management application instance may also manage at least one container runtime (as an instance of the container) through the container engine. The container engine may be used to run at least one container, for example, the container engine may run the container by creating a container runtime or deleting a container runtime. Additionally, container management application instances may be installed in the container engine, enabling unified management of container runtime and containers, and enabling compatibility with existing container technologies (e.g., containers or container clouds). Alternatively, the container engine may be installed in a container management application instance, thereby enabling orchestration and management of containers with guaranteed independence of the container engine.

It should be appreciated that for receiving model service requests sent for a container management application instance, the model service requests may be obtained by a home agent of the management node. The request may be sent by the management node to a container management application instance of the executing node. In addition, the executing node may also directly receive the model service request. In addition, the model service request includes a first network address, which may indicate a network address of the executing node or a network address of a container management application instance in the executing node. For example, if there are multiple container management application instances in the execution node, the multiple container management application instances all have the first network address. In addition, different container management application instances in the same execution node may have multiple different virtual ports. The virtual port in combination with the first network address indicates a unique address of the container management application instance. In addition, different executing nodes (logical nodes) may have different network addresses. When a particular execution node is implemented by multiple physical servers, the multiple physical servers may have the same network address. The client may send a model service request based on the first network address, in other words, the client directly invokes a service provided by the container management application instance in the execution node. For example, the client may also send a model service request to the management node (or to the management node via an application server in the server), which may determine an address of the container management application instance, e.g., determine a first network address of the container management application instance, in response to the model service request. The management node may send a sub-request of the model service request to an execution node of the container management application instance based on the first network address or forward the model service request. The executing node may determine the container management application instance based on the virtual port of the container management application instance and forward the sub-request or model service request thereto. Alternatively, the management node may also determine a virtual port of the container management application instance and send a sub-request of the model service request to the container management application instance or forward the model service request.

For receiving a model service request sent by a management node for a container management application instance having a first network address, the model service request may be received through a first network address-based network interface of the container management application instance. The service container and the model container may be installed in a container management application instance. For example, the container management application instance may be installed in an operating system (e.g., may be a server operating system, etc.) of the resource object execution node, which may act as a host system for the container management application instance.

It should be appreciated that the model service request may be a request for a model prediction service, such as a neural network model prediction service. Data as input in the prediction model may be included in the model service request, for example, the data may be waiting prediction data such as voice data, audio data, semantic data, picture data, text data, video data, so that the model container performs prediction processing using the corresponding prediction model, returns text data such as a recognition result of the input audio data or video data, text data of a second language corresponding to the input text data of the first language, text data corresponding to the input picture data, and the like. For predictions made by a service container invoking a prediction model in a model container in response to a model service request, the service container and the model container may be accessible to each other through a call command. Alternatively, the service container and the model container may enable access to each other through inter-container communication. Additionally, the model service request may include data to be predicted. The model service request is a request sent or forwarded by a home agent of the management node. In addition, the predictive model may be run on a mirror IMAGE (IMAGE) of the model container configuration. The image may be a read-only program and one or more images may be pulled from the image repository in creating the container to configure the container.

The scheme of the embodiment of the invention has the beneficial effects that: because the initialization container and the model container keep the independence of the container, the updating or the maintenance can be independently carried out, and the flexibility of updating the computing platform is improved.

In another implementation of the present invention, processing the model service request with the container management application instance includes: the initialization container copies the data of the user to the local execution node; the model container provides the model service for the user by using the data of the user.

In one example, initializing the container retrieves the predictive model to the storage space of the container management application instance. In one example, the initialization container acquires the predictive model into the model container. For example, the initialization container may share storage space of the container management application instance with the model container, and the initialization container may retrieve the predictive model from the storage space of the container management application instance. For example, any manner may be used for initializing the container to obtain the prediction model from the storage space of the container management application instance, which is not limited by the embodiment of the present invention.

In another implementation manner of the present invention, the container management application instance further includes: a service container; processing the model service request with the container management application instance further comprises: the service container provides the corresponding performance service of the model service for the user.

The service container (i.e., the application service container) of the embodiment of the present invention is responsible for application service logic of the cloud computing service framework. The service container has the functions of caching, authentication, intervention, batch processing arrangement and the like. The model container (predictive model container) is responsible for online servicing of the model (i.e., the algorithmic model). It should also be understood that the user can define the required model development framework and corresponding language as desired. For example, the model development framework may be tensorflow, pyrrch, etc. For example, the language developed in the development framework may be, for example, python, c + +, java, or the like.

In other words, in the solution of the embodiment of the present invention, since the prediction model and the application service can be configured in different containers, for the prediction model container, a plurality of containers supporting different algorithm environments can be configured; in addition, for the application service model, a plurality of containers can be constructed for a plurality of versions of the application service codes, so that external dependency conflicts of different algorithm models and different application service codes are avoided.

In another implementation of the present invention, a model container for providing a model service to a user by using data of the user includes: the model container defines a corresponding model framework and language according to the requirements of the user, and provides model services for the user through the model framework and the language.

In another implementation of the invention, the initialization container, the model container, and the service container share a network and a disk.

In another implementation of the present invention, allocating an executing node for a model service request includes: in response to the model service request, a route of the executing node is allocated for the model service request based on the load states of the plurality of candidate executing nodes.

In another implementation manner of the present invention, a method for allocating a route of an executing node to a model service request based on load states of a plurality of candidate executing nodes in response to the model service request includes: responding to the model service request, and respectively sending a load state query request to a plurality of candidate execution nodes; and distributing the route of the executing node for the model service request by receiving the current load state information sent by a plurality of candidate executing nodes.

Fig. 1E is a schematic flowchart of a cloud computing service method according to another embodiment of the present invention. The method of FIG. 1E includes:

170: and receiving a model service request sent aiming at a container management application instance, wherein the container management application instance comprises a service container and a model container, the service container is used for providing application services, and the model container is used for running a prediction model.

180: and responding to the model service request, and calling a prediction model in the model container by the service container to perform prediction.

190: and returning the prediction result of the prediction model.

It should be appreciated that for returning the prediction results of the prediction model to the management node based on the first network address, the prediction results of the prediction model may be returned to the management node through a port of the first network address. The prediction results of the prediction model are returned, for example, by the home agent of the management node. For example, a port based on the first network address returns a prediction result of the prediction model through a home agent of the management node. It should be understood that the prediction results are the output of the prediction model, corresponding to the input data provided in response to the model service request.

It should be understood that services of embodiments of the present invention include, but are not limited to, various cloud technology services such as natural language processing and/or machine translation. Resource object management networks include, but are not limited to, container management networks such as kubernets (a container cluster management system). The container management application instance may include, but is not limited to, resource objects such as PODs (in kubernets, PODs are container-deploying units, i.e., container-deploying objects).

It should also be understood that the network address in embodiments of the present invention may be the network address of any network. For example, a network address based on the internet, or a network address of a mobile network, etc. The embodiment of the present invention is not limited thereto. The container in the embodiment of the present invention may be any virtualized object. For example, the container may be a portable container such as a dock (a type of container engine). The container in the embodiment of the invention can be a container in container cloud technology. The container management application instance in embodiments of the present invention may be at an upper layer of the host system. In one example, the container management application instance in embodiments of the present invention may be at an upper level of the container runtime. In another example, the container management application instance may be at a lower level of the container runtime. The embodiment of the present invention does not limit this, and preferably, the container management application instance is located at an upper layer of the container runtime, so as to implement tighter management and scheduling on the container.

It should also be appreciated that in the application scenario of the embodiment of the present invention, when a user initiates a request to a resource object management cluster network (e.g., a self-learning platform) through a client, the resource object management node may look up the current configuration of the container management application instance in each execution node and initiate a correct route. For example, the request may include a prediction model to be requested, and data to be predicted, and the resource object management node requests a prediction service from the prediction model based on an address of a container in which the prediction model is run. In addition, since there may be multiple requests initiated by multiple clients requesting the same or different prediction services, and the number and address of containers may change due to the object management node creating and deleting the containers, the load balancing process of the application service container and the prediction model container in the container management application instance may be performed before the requests reach the container where the prediction model is located. Then, when the request reaches the application service container, the application service container may call the container running the predictive model (i.e., the request reaches the predictive model service). The predictive model service then processes the request (e.g., performs natural language processing, translation processing, or computer vision processing, etc.) and returns the results (e.g., recognition results or translation results) to the application service. For example, the model service request is a command of a scheduling center of the management node (e.g., scheduler of a kubernets host). For example, the dispatch center would configure two containers into the same container management application instance (e.g., POD) by creating a container instruction.

In addition, in the scheme of the embodiment of the invention, the prediction model container and the application service model have independence. Thus, whether the platform operation and maintenance personnel want to update the platform application code in the application service model or the algorithm personnel update the algorithm code in the prediction model, they can issue each independently, i.e., without being coupled together and issued at the same time.

As one example, receiving a model service request sent for a container management application instance includes: receiving a model service request sent by a management node aiming at a container management application instance of a first network address, and returning a prediction result of a prediction model, wherein the model service request comprises: and returning a prediction result of the prediction model to the management node based on the first network address.

In this example, the service container and the model container employ network stacks based on the same execution node so that no delay is added when invoking the prediction model compared to the calling process in the same container. Namely, the prediction model and the application service are decoupled on the premise of ensuring low time delay.

In another implementation manner of the present invention, after the initializing container copies the data of the user to the local of the executing node, the method further includes: the initialized container is destroyed.

As one example, the service container has a first virtual port based on a first network address, the method further comprising: responding to a service container updating instruction sent by a management node, creating an updating service container in a container management application instance, and allocating a second virtual port based on a first network address to the updating service container, wherein the second virtual port is different from the first virtual port; the service container is deleted from the container management application instance.

As one example, the model container has a third virtual port based on the first network address, the method further comprising: in response to a model container updating instruction sent by the management node, creating an updated model container in the container management application instance, and allocating a fourth virtual port based on the first network address to the updated model container, wherein the fourth virtual port is different from the third virtual port; the model container is deleted from the container management application instance.

As one example, prior to creating an initialization container in the container management application instance in response to the model deployment request, the method further comprises: in response to a container management application instance creation instruction sent by the management node, a container management application instance is created and a first network address is assigned.

It should be understood that the cloud computing service method provided by the embodiment of the invention can be suitable for cloud service types such as public clouds, private clouds or private clouds. The above-described stage of performing prediction using the prediction model may be applicable to various types of cloud services. The cloud computing service method of this example may also include a model deployment phase and an application service deployment phase. In the model deployment phase, the method may further include: receiving a model deployment request sent by a management node aiming at a container management application instance; in response to the model deployment request, a model container and/or an initialization container is created in the container management application instance. In the application service deployment phase, the method may further include: receiving an application service deployment request sent by a management node aiming at a container management application instance; in response to the application service deployment request, a service container is created in the container management application instance.

The cloud computing service method provided by the embodiment of the invention further comprises a model training phase. In the model training stage, a user can provide a training sample in a specific field, and an initial model provided by a model service framework provided by a platform is used for training, so that a prediction model in the field is obtained. In one example, a model training container may be created with a container management application instance during a model training phase and the trained predictive model stored in one or more storage media.

In another implementation of the present invention, prior to receiving the model service request sent for the container management application instance, the method further comprises: responding to a prediction model deployment request sent aiming at a container management application instance, and creating an initialization container in the container management application instance for obtaining a prediction model; the predictive model is obtained into the model container by initializing the container.

As one example, prior to receiving the model service request sent by the management node for the container management application instance having the first network address, the method further comprises: receiving a model deployment request sent by a management node aiming at a container management application instance with a first network address; in response to the model deployment request, creating an initialization container in the container management application instance; the predictive model is obtained into the model container by initializing the container. For example, the initialization container, the service container, and the model container may share a network (network address) and a storage space (e.g., a disk). Because the initialized container also has the network address of the execution node where the container management application instance is located in the container management application instance, the three containers can be ensured to perform inter-container communication or scheduling based on the same network address, and the three containers can be ensured to be managed by the container management application instance.

Fig. 1C is a schematic diagram of a service deployment phase of a cloud computing service method according to another embodiment of the present invention. As shown in fig. 1C, the initialization container retrieves a pre-trained predictive model from at least one of the first storage medium, the second storage medium, and the third storage medium. For example, the first storage medium, the second storage medium, and the third storage medium may be different storage service platforms or different storage modes. For example, the Storage medium includes, but is not limited to, Storage media provided by Network Attached Storage (NAS), Object Storage Service (OSS), CEPH (a distributed Storage system that supports block Storage, file Storage, and Object Storage), and a monto database (a distributed and non-stored database).

For a proprietary cloud, model training may be performed for one or more containers. Multiple prediction models may be trained using one container, or multiple prediction models may be trained in multiple containers, respectively. In another example, the model may be trained in other ways and the trained model may be stored in one or more of the storage media described above to retrieve the trained model from the one or more storage media. Due to the fact that different storage media such as OSS, MONGO, CEPH or local disks are used, the storage flexibility can be improved, and model training services based on the proprietary cloud can be provided for different service objects conveniently. In addition, the initialization container of the embodiment of the invention can acquire the model from different storage media or storage platforms so as to predict based on the prediction model, thereby realizing the decoupling between the storage media or storage platforms and the update of the prediction model. Therefore, for the proprietary cloud, flexible updating and resource optimization configuration can be provided for a large number of proprietary cloud users through the configuration.

In addition, for private cloud services, model deployment, model training, and application services are typically configured in the same platform, which may include a hardware platform and a software platform. In this case, the solution configuration storage medium (or storage platform), the prediction model service, or the application service of the embodiment of the present invention may be disposed in the private cloud platform. The safety can be improved by realizing the various services on a single platform, and the optimization can be carried out in a targeted manner based on the characteristics of a prediction model. For example, based on the statistics of the number of various predictive model services, the amount of computations, the power of the computations, the number of hardware platforms, software platforms, the number of containers in each execution node, the type of storage medium, the number of storage media, etc. are configured. The predictive model service and the application service are respectively configured in different containers so that updating or upgrading can be performed for the application service or the predictive model service, respectively. In addition, the model is acquired from different types of storage media or storage platforms to the container for operating the prediction model, so that the media at the bottom layer can be shielded, the abstraction of the storage media is realized, and the safe and flexible management of the special cloud platform is facilitated.

Therefore, as described above, compared with a scheme of downloading data and models from a storage medium to the local, in a public cloud, a private cloud, or a private cloud scenario, the scheme of the embodiment of the present invention does not need to use different storage medium clients to adapt to different storage media, thereby achieving decoupling of the storage media.

In another implementation manner of the present invention, the initializing container and the model container share a storage space of the container management application instance, wherein obtaining the prediction model into the model container by the initializing container includes: and acquiring the prediction model from the storage medium into the storage space by initializing the container so that the model container acquires the prediction model from the storage space.

In another implementation of the present invention, a container management application instance having a first network address and a storage management object having a second network address is used for managing a storage medium, wherein obtaining a prediction model from the storage medium into a storage space by initializing a container includes: an access request is initiated to the storage medium by the initialization container based on the first network address to the second network address so that the storage management object returns the predictive model to the storage space in response to the access request. The storage management object is configured with a uniform storage medium access interface for a plurality of storage media. For example, the storage management objects described above include, but are not limited to, resource objects for storage that include persistent-volumes (PV). For an access request for a storage medium, the Persistent Volume class can be bound with the PV, and then various storage media managed by the PV can be accessed. Because the storage management object is configured with a uniform storage medium access interface for a plurality of storage media, the model stored in the storage medium can be conveniently acquired at different storage media, in other words, the storage management object is used for shielding the storage medium at the bottom layer.

In addition, in the scheme of the embodiment of the invention, the model drawing and the platform service are decoupled, and meanwhile, the model drawing and the platform service can be ensured to be scheduled as a unit and share physical resources.

It should be understood that the storage management object may be in the same management node as the container management application instance. The second network address is different from the first network address. The initialization container (or initialization container) realizes the acquisition of the prediction model with the storage medium in the storage management object through network communication and calling. Meanwhile, independence between different storage services or different storage media is guaranteed.

In another implementation of the present invention, after obtaining the prediction model from the storage medium into the storage space by initializing the container, the method includes: the initialized container is deleted from the container management application instance. For example, fig. 1D is a schematic diagram of a service phase prediction of a cloud computing service method according to another embodiment of the present invention. As shown in fig. 1D, the initialization container in the container management application instance may be deleted after the prediction model is obtained from outside the container management application instance, thereby increasing the storage space of the container management application instance. It should be appreciated that initializing the container after obtaining the predictive model from outside the container management application instance may mark the completion of the model deployment phase, after which the model prediction phase of predicting with the model may be entered. Thereafter, in one aspect, the application service container may make calls to the prediction models in the prediction model container by the management node sending instructions to the execution node in response to a user request. On the other hand, in the model prediction phase, maintenance personnel or model algorithm personnel can update the application service codes and the algorithm codes at any time.

In another implementation of the invention, the service container has a first virtual port based on the first network address, the method further comprising: responding to a service container updating instruction sent by a management node, creating an updating service container in a container management application instance, and allocating a second virtual port based on a first network address to the updating service container, wherein the second virtual port is different from the first virtual port; the service container is deleted from the container management application instance. In other words, initializing the container is first performed, and is responsible for copying data on the storage medium to the local by mounting the storage management object to the storage medium. After the copying of the data is completed, the initialization container no longer occupies resources.

It should be understood that, by using different virtual ports based on the same network address, dynamic allocation of ports is realized, and after a communication task is completed or a service container is updated, the ports can be cancelled or deleted, so that communication resources are saved while computing efficiency is ensured. In addition, the network address corresponds to a container management application instance, which may be created and deleted. Containers in the container management application instance may be created and deleted. As an example, prior to creating the initialization container in the container management application instance in response to the model deployment request, the method may further comprise: in response to a container management application instance creation instruction sent by the resource object management node, a container management application instance is created and a first network address is assigned. For example, when a container management application instance is updated, the first network address may be deleted, such that the first network address is re-available or assigned to other container management application instances. Therefore, the creation and deletion of the network address and the virtual port can better match the container management application instance and the virtualization of the container, so that the whole platform architecture is not dependent on the physical environment to manage, schedule and allocate resources to a certain extent.

It should be understood that, by using different virtual ports based on the same network address, dynamic allocation of ports is realized, and after a communication task is completed or a model container is updated, the ports can be cancelled or deleted, so that communication resources are saved while computing efficiency is ensured. In addition, the network address corresponds to a container management application instance, which may be created and deleted. Containers in the container management application instance may be created and deleted. Therefore, the creation and deletion of the network address and the virtual port can better match the container management application instance and the virtualization of the container, so that the whole platform architecture is not dependent on the physical environment to manage, schedule and allocate resources to a certain extent.

In another implementation of the invention, the method further comprises: the service container and the model container are created in the container management application instance in response to a container creation instruction sent by the resource object management host for the first network address.

FIG. 2A is a schematic block diagram of a cloud computing service method according to another embodiment of the present invention; the cloud computing service method of fig. 2A may be applied to an execution node (e.g., a resource object execution node) in a cloud computing service network. In a cloud computing services network, resource object execution nodes may communicate with clients to provide requested specific services (e.g., services provided based on a predictive model) for the clients. In addition, the server of the cloud computing services network may include an execution node and a management node (e.g., a resource object management node). The resource object management node may manage at least one resource object execution node. The resource object execution node may obtain a service request (e.g., a model service request) from a client and generate a sub-request for the resource object execution node according to the service sentiment. The resource object execution node may return a service result (e.g., a predicted result) or an intermediate result to the resource object management node in response to the sub-request. The resource object execution node may forward the service result to the client; or processing the intermediate result to obtain a service result and returning the service result to the client. The method of fig. 2A includes:

210: and receiving a model service request sent aiming at a service container management application instance, wherein the service container is included in the service container management application instance and is used for providing application service.

It should be understood that the container management application instances herein may be objects for creating containers, deleting containers, organizing containers, managing containers, and the like. The container management application instance may be implemented as a specific abstraction layer in the execution node. The container in the container management application instance may also manage at least one container runtime (as an instance of the container) through the container engine. The container engine may be configured to run at least one container. For example, the container engine may run the container by creating a container runtime or deleting a container runtime. Additionally, container management application instances may be installed in the container engine, enabling unified management of container runtime and containers, and enabling compatibility with existing container technologies (e.g., containers or container clouds). Alternatively, the container engine may be installed in a container management application instance, thereby enabling orchestration and management of containers with guaranteed independence of the container engine. For receiving the model service request sent by the management node for the container management application instance having the third network address, the model service request may be obtained by a home agent of the management node. The third network address may be included in the model service request. The container management application instance may be determined based on the model service request. The service container may enable invocation and access by invoking commands. For receiving the model service request sent by the management node for the container management application instance having the third network address, the model service request may be received through the first network address-based network interface of the container management application instance. The service container may be installed in a container management application instance. The container management application instance may be installed in an operating system of the server that is the hosting system. The container management application instance may be installed on top of the container engine. In one example, the container engine may be installed on top of the container management application instance. In another example, a container management application instance is installed under a container runtime and manages a container runtime (runtime) instance.

220: and responding to the model service request, determining a model container management application instance, wherein the model container management application instance comprises a model container which is used for running the prediction model.

It is to be appreciated that in response to the model service request, a model container management application instance is determined having a fourth network address, the model container management application instance being determined by the third network address in communication with the fourth network address. The model container management application instance may be determined by a home agent of the management node. The model container may enable calling and accessing through a call command. Wherein the model container may be installed in a container management application instance.

230: and calling a prediction model in the model container by the service container to predict.

It should be understood that for a prediction by a service container calling a prediction model in a model container in response to a model service request, the service container and the model container may be accessed from each other by calling a command. Alternatively, the service container and the model container may enable access to each other through inter-container communication. Additionally, the model service request may include data to be predicted. The model service request may be a request sent or forwarded by a home agent of the management node. In addition, the predictive model may be run on a mirror IMAGE (IMAGE) of the model container configuration. The image may be a read-only program and one or more images may be pulled from the image repository in creating the container to configure the container. For responding to a model service request, the model service request may include data to be predicted by the service container calling a prediction model in the model container for prediction. The model service request may be a request sent or forwarded by a home agent of the management node. The predictive model may be run on a mirror image of the model container. In addition, the service container calls the prediction model in the model container through communication of the third network address with the fourth network address. The service container may invoke the predictive model in the model container through a bridge with the model container. The service container may also call the predictive model in the model container by communicating with a second agent of the management node through the first agent of the management node. Wherein the first agent and the second agent may be managed by a management node. The service container may call the predictive model in the model container through a two-layer network with the model container. For example, the service container may have a port of a third network address and the model container may have a port of a fourth network address.

240: and returning the prediction result of the prediction model.

It should be appreciated that for returning the prediction results of the prediction model to the management node based on the first network address, the prediction results of the prediction model may be returned to the management node through a port of the first network address. The prediction results of the prediction model may be returned by the home agent of the management node. For example, the prediction results of the prediction model may be returned by the home agent of the management node based on the port of the first network address. Additionally, the prediction results may be the output of the prediction model, corresponding to input data provided in response to the model service request.

In other words, the solution of this embodiment puts the service container and the model container into one independent container management application instance, for example, the whole cluster shares the same set of service containers. Fig. 2B is a schematic diagram of a service phase prediction of a cloud computing service method according to another embodiment of the present invention. As shown in fig. 2B, the service container management application instance and the model container management application instance have different network addresses. Containers in different container management application instances may alternatively communicate using network addresses. On one hand, the flexibility of platform or algorithm updating is improved because containers in different container management application instances can still be updated or replaced independently. On the other hand, since the service container and the model container are in an incompatible container management application instance, the two containers have higher independence and further decoupling is achieved compared with the scheme in the above embodiment. For example, more service containers based on different services may be created with a single service container management application instance. Similarly, more different service-based model containers can be created using the model container management application instance. Therefore, on the cluster level, the service container based on different services and the model container scheduling based on different services are prevented from having disorder, so that the effective utilization of resources is integrally realized, and communication resources and scheduling resources are integrally saved.

In one example, the initialization container may be deleted after the predictive model is obtained from outside the container management application instance, thereby increasing the storage space of the container management application instance while masking differences in the storage media. It should be appreciated that initializing the container after obtaining the prediction model from outside the container management application instance may mark completion of the model deployment phase, after which, in one aspect, the platform may call the prediction model in the model container by sending instructions through the management node in response to a user request. On the other hand, the platform maintenance personnel or the model algorithm personnel can update the codes of the platform at any time.

FIG. 3 is a schematic flow chart diagram of a cloud computing service method of another embodiment of the present invention; the cloud computing service method of fig. 3 may be applied to an execution node (e.g., a resource object execution node) in a cloud computing service network. In a cloud computing services network, resource object execution nodes may communicate with clients to provide requested specific services (e.g., services provided based on a predictive model) for the clients. In addition, the server of the cloud computing services network may include an execution node and a management node (e.g., a resource object management node). The resource object management node may manage at least one resource object execution node. For example, a resource object execution node may obtain a service request (e.g., a model service request) from a client and generate a sub-request for the resource object execution node according to the service sentiment. The resource object execution node may return a service result (e.g., a predicted result) or an intermediate result to the resource object management node in response to the sub-request. The resource object execution node may forward the service result to the client; or processing the intermediate result to obtain a service result and returning the service result to the client. The method of fig. 3 includes:

310: and receiving a prediction model deployment request sent by aiming at a container management application instance, wherein the container management application instance comprises a service container which is used for providing application services.

320: and in response to the prediction model deployment request, creating a second container and an initialization container in the container management application instance, wherein the model container is used for running the prediction model, and the initialization container is used for obtaining the prediction model.

It should be appreciated that the initialization container may share storage space of the container management application instance with the model container, and the initialization container may retrieve the predictive model from the storage space of the container management application instance. In addition, any manner may be adopted for initializing the container to obtain the prediction model from the storage space of the container management application instance, which is not limited by the embodiment of the present invention.

330: and acquiring a prediction model into the model container by the initialization container so as to call the prediction model in the model container by the service container to perform prediction.

It should be appreciated that for a prediction by a service container calling a prediction model in a model container in response to a model service request, the service container and the model container may be accessible to each other by calling a command. Alternatively, the service container and the model container may enable access to each other through inter-container communication. Additionally, the model service request may include data to be predicted. For example, the model service request is a request sent or forwarded by a home agent of the management node. In addition, the predictive model may be run on a mirror IMAGE (IMAGE) of the model container configuration. The image may be a read-only program and one or more images may be pulled from the image repository in creating the container to configure the container.

It should be understood that the cloud computing service method of the embodiment of the present invention may further include an application service deployment phase. In the model deployment phase, for example, the method may further include: receiving an application service deployment request sent by a management node aiming at a container management application instance; in response to the application service deployment request, a service container is created in the container management application instance.

FIG. 4 is a schematic flow chart diagram of a cloud computing service method of another embodiment of the present invention; the cloud computing service method of fig. 4 may be applied to an execution node (e.g., a resource object execution node) in a cloud computing service network. In a cloud computing services network, resource object execution nodes may communicate with clients to provide requested specific services (e.g., services provided based on a predictive model) for the clients. In addition, the server of the cloud computing services network may include an execution node and a management node (e.g., a resource object management node). The resource object management node may manage at least one resource object execution node. For example, a resource object execution node may obtain a service request (e.g., a model service request) from a client and generate a sub-request for the resource object execution node according to the service sentiment. The resource object execution node may return a service result (e.g., a predicted result) or an intermediate result to the resource object management node in response to the sub-request. The resource object execution node may forward the service result to the client; or processing the intermediate result to obtain a service result and returning the service result to the client.

It should also be understood that in this example, the cloud computing service approach is directed to neural network computing services such as natural language processing or machine translation. The prediction model is a natural language processing model or a language recognition model or a machine translation module, and is obtained by training through a Recurrent Neural Network (RNN). Additionally, the client requesting the prediction service from the execution node or from the management node may be a machine translation engine, a machine translation application, a speech recognition application, or the like. The method of fig. 4 includes:

410: receiving a neural network model service request sent aiming at a container management application instance, wherein the model service request comprises semantic data to be predicted, the container management application instance comprises a service container and a model container, the service container is used for providing application service, and the model container is used for operating a neural network prediction model.

420: responding to the model service request, calling the prediction model in the model container through the service container, and performing semantic processing on the semantic data to be predicted to obtain a semantic processing result.

It should be appreciated that for a prediction by a service container calling a prediction model in a model container in response to a model service request, the service container and the model container may be accessible to each other by calling a command. Alternatively, the service container and the model container may enable access to each other through inter-container communication. Additionally, the model service request may include data to be predicted. The model service request may be a request sent or forwarded by a home agent of the management node. The predictive model may be run on a mirror IMAGE (IMAGE) of the model container configuration. The image may be read-only and is pulled from the image repository in the creation of the container.

430: and returning a prediction result comprising the semantic processing result.

Fig. 5 is a schematic block diagram of a cloud computing service apparatus of another embodiment of the present invention; the cloud computing service apparatus of fig. 5 includes:

a model service request receiving module 510, configured to receive a model service request sent for a container management application instance, where the container management application instance includes a service container and a model container, the service container is used to provide an application service, and the model container is used to run a prediction model;

the container calling module 520 responds to the model service request and calls a prediction model in the model container to predict through the service container;

and a prediction result returning module 530 for returning the prediction result of the prediction model.

In other words, the environment is not shared with the model service because the platform service is stand-alone into the service container. Therefore, only one piece of code and running environment of the platform need to be maintained. Meanwhile, the method is not limited by an algorithm environment, and more customized performances and expansion are realized. Accordingly, the model service is stand alone into the model container and does not share the environment with the platform service. The algorithm personnel can use the model framework, language and dependency more flexibly.

It should be appreciated that in aspects of embodiments of the present invention, even though the model container and platform service model are served externally at the same container management application instance (e.g., POD), the container management framework (e.g., the publishing mechanism (e.g., rolling update mechanism) of kubernets) may update only one container in the container management application instance, and thus the service container and the model container can be decoupled.

In another implementation of the present invention, the apparatus further comprises: the container creating module is used for responding to a prediction model deployment request sent aiming at the container management application instance, and creating an initialization container in the container management application instance for obtaining a prediction model; and the model acquisition module acquires the prediction model into the model container by initializing the container.

In another implementation manner of the present invention, the initialization container and the model container share a storage space of the container management application instance, where the model obtaining module is specifically configured to: and acquiring the prediction model from the storage medium into the storage space by initializing the container so that the model container acquires the prediction model from the storage space.

In another implementation manner of the present invention, the container management application instance has a first network address, and the storage management object having a second network address is used for managing the storage medium, wherein the model obtaining module is specifically configured to: an access request is initiated to the storage medium by the initialization container based on the first network address to the second network address so that the storage management object returns the predictive model to the storage space in response to the access request.

In another implementation of the present invention, after the obtaining of the prediction model from the storage medium into the storage space by initializing the container, the apparatus further includes: and the container deleting module is used for deleting the initialized container from the container management application example.

In another implementation manner of the present invention, the model service request receiving module is specifically configured to: receiving a model service request sent by a management node for a container management application instance of a first network address,

the prediction result returning module is specifically configured to: and returning a prediction result of the prediction model to the management node based on the first network address.

In another implementation of the present invention, the service container has a first virtual port based on the first network address, and the apparatus further comprises: the service container updating module is used for responding to a service container updating instruction sent by the management node, creating an updating service container in the container management application instance, and allocating a second virtual port based on the first network address to the updating service container, wherein the second virtual port is different from the first virtual port; the service container is deleted from the container management application instance.

In another implementation of the invention, the model container has a third virtual port based on the first network address, the apparatus further comprising: the model container updating module is used for responding to a model container updating instruction sent by the management node, creating an updated model container in the container management application instance, and allocating a fourth virtual port based on the first network address to the updated model container, wherein the fourth virtual port is different from the third virtual port; the model container is deleted from the container management application instance.

In another implementation of the present invention, before creating the initialization container in the container management application instance in response to the model deployment request, the apparatus further includes: and the container creating module responds to a container management application instance creating instruction sent by the management node, creates a container management application instance and allocates the first network address.

The apparatus of this embodiment is used to implement the corresponding method in the foregoing method embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described herein again. In addition, the functional implementation of each module in the apparatus of this embodiment can refer to the description of the corresponding part in the foregoing method embodiment, and is not described herein again.

Fig. 6 is a schematic block diagram of a cloud computing service apparatus of another embodiment of the present invention; the cloud computing service apparatus of fig. 6 includes:

a request receiving module 610, which receives a model service request initiated by a user;

a node allocation module 620, which allocates execution nodes for the model service request;

a request processing module 630, on the execution node, processing the model service request by using a container management application instance, where the container management application instance at least includes the following containers: initializing a container and a model container;

and a processing result returning module 640 for returning the processing result to the user.

In another implementation manner of the present invention, the request processing module is specifically configured to: the initialization container copies the data of the user to the local execution node; the model container provides the model service for the user by using the data of the user.

In another implementation manner of the present invention, the container management application instance further includes: a service container; the request processing module is further configured to: the service container provides the corresponding performance service of the model service for the user.

In another implementation manner of the present invention, after the initializing container copies the data of the user to the local of the executing node, the apparatus further includes: and the deleting module is used for destroying the initialized container.

In another implementation manner of the present invention, the request processing module is specifically configured to: the model container defines a corresponding model framework and language according to the requirements of the user, and provides model services for the user through the model framework and the language.

In another implementation manner of the present invention, the node allocating module is specifically configured to: in response to the model service request, a route of the executing node is allocated for the model service request based on the load states of the plurality of candidate executing nodes.

In another implementation manner of the present invention, the node allocating module is specifically configured to: responding to the model service request, and respectively sending a load state query request to a plurality of candidate execution nodes; and distributing the route of the executing node for the model service request by receiving the current load state information sent by a plurality of candidate executing nodes.

Fig. 7 is a schematic block diagram of a cloud computing service apparatus of another embodiment of the present invention; the cloud computing service device of fig. 7 includes:

a deployment request receiving module 710, configured to receive a prediction model deployment request sent for the container management application instance, where the container management application instance includes a service container, and the service container is used to provide an application service;

a container creating module 720, configured to create, in response to the prediction model deployment request, a model container and an initialization container in the container management application instance, where the model container is used to run a prediction model, and the initialization container is used to obtain the prediction model;

the model obtaining module 730 obtains the prediction model into the model container through the initialization container so as to call the prediction model in the model container for prediction through the service container.

Fig. 8 is a schematic block diagram of a cloud computing service apparatus of another embodiment of the present invention; the cloud computing service device of fig. 8 includes:

the model service request receiving module 810 is configured to receive a neural network model service request sent for a container management application instance, where the model service request includes semantic data to be predicted, the container management application instance includes a service container and a model container, the service container is used to provide application services, and the model container is used to run a neural network prediction model;

the semantic processing module 820 responds to the model service request, calls a prediction model in the model container through the service container, and performs semantic processing on the semantic data to be predicted to obtain a semantic processing result;

and a prediction result returning module 830 for returning a prediction result including the semantic processing result.

Fig. 9 is a schematic structural diagram of an electronic device according to another embodiment of the invention; the electronic device may include:

one or more processors 901;

a computer-readable medium 902, which may be configured to store one or more programs,

when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the cloud computing service method according to the above embodiment.

Fig. 10 is a hardware configuration of an electronic apparatus according to another embodiment of the present invention; as shown in fig. 10, the hardware structure of the electronic device may include: a processor 1001, a communication interface 1002, a computer-readable medium 1003, and a communication bus 1004;

wherein the processor 1001, the communication interface 1002, and the computer readable medium 1003 complete communication with each other through the communication bus 1004;

alternatively, the communication interface 1002 may be an interface of a communication module;

the processor 1001 may be specifically configured to: receiving a model service request sent aiming at a container management application instance, wherein the container management application instance comprises a service container and a model container, the service container is used for providing application service, and the model container is used for operating a prediction model; responding to the model service request, and calling a prediction model in the model container through the service container to predict; and returning the prediction result of the prediction model.

Alternatively, the processor 1001 may be specifically configured to: receiving a model service request sent aiming at a service container management application instance, wherein the service container management application instance comprises a service container which is used for providing application service; responding to the model service request, determining a model container management application instance, wherein the model container management application instance comprises a model container which is used for operating a prediction model; calling a prediction model in the model container through the service container to predict; and returning the prediction result of the prediction model.

Alternatively, the processor 1001 may be specifically configured to: receiving a prediction model deployment request sent aiming at a container management application instance, wherein the container management application instance comprises a service container which is used for providing application service; responding to a prediction model deployment request, creating a second container and an initialization container in a container management application instance, wherein the model container is used for operating a prediction model, and the initialization container is used for obtaining the prediction model; and acquiring a prediction model into the model container by the initialization container so as to call the prediction model in the model container by the service container to perform prediction.

Alternatively, the processor 1001 may be specifically configured to: receiving a neural network model service request sent aiming at a container management application instance, wherein the model service request comprises semantic data to be predicted, the container management application instance comprises a service container and a model container, the service container is used for providing application service, and the model container is used for operating a neural network prediction model; responding to the model service request, calling a prediction model in the model container through the service container, and performing semantic processing on semantic data to be predicted to obtain a semantic processing result; and returning a prediction result comprising the semantic processing result.

The Processor 1001 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The computer-readable medium 1003 may be, but is not limited to, a Random Access Memory (RAM), a Read-Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code configured to perform the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section, and/or installed from a removable medium. The computer program, when executed by a Central Processing Unit (CPU), performs the above-described functions defined in the method of the present application. It should be noted that the computer readable medium described herein can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access storage media (RAM), a read-only storage media (ROM), an erasable programmable read-only storage media (EPROM or flash memory), an optical fiber, a portable compact disc read-only storage media (CD-ROM), an optical storage media piece, a magnetic storage media piece, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Computer program code configured to carry out operations for the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may operate over any of a variety of networks: including a Local Area Network (LAN) or a Wide Area Network (WAN) -to the user's computer, or alternatively, to an external computer (e.g., through the internet using an internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions configured to implement the specified logical function(s). In the above embodiments, specific precedence relationships are provided, but these precedence relationships are only exemplary, and in particular implementations, the steps may be fewer, more, or the execution order may be modified. That is, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules described in the embodiments of the present application may be implemented by software or hardware. The names of these modules do not in some cases constitute a limitation of the module itself.

As another aspect, the present application also provides a computer-readable medium on which a computer program is stored, which when executed by a processor, implements the cloud computing service method as described in the above embodiments.

As another aspect, the present application also provides a computer-readable medium, which may be contained in the apparatus described in the above embodiments; or may be present separately and not assembled into the device. The computer readable medium carries one or more programs which, when executed by the apparatus, cause the apparatus to: receiving a model service request sent by a management node aiming at a container management application instance with a first network address, wherein the container management application instance comprises a service container and a model container; responding to the model service request, and calling a prediction model in the model container for prediction through the service container; returning a prediction result of the prediction model to the management node based on the first network address, or,

receiving a model service request sent by a management node aiming at service container management with a third network address, wherein the service container management application instance comprises a service container; responding to the model service request, determining a model container management application instance with a fourth network address, wherein the model container management application instance comprises a model container; based on the third network address, calling a prediction model in the model container from the fourth network address through the service container for prediction; and returning the prediction result of the prediction model to the management node based on the third network address.

The expressions "first", "second", "said first" or "said second" used in various embodiments of the present disclosure may modify various components regardless of order and/or importance, but these expressions do not limit the respective components. The above description is only configured for the purpose of distinguishing elements from other elements. For example, the first user equipment and the second user equipment represent different user equipment, although both are user equipment. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of the present disclosure.

When an element (e.g., a first element) is referred to as being "operably or communicatively coupled" or "connected" (operably or communicatively) to "another element (e.g., a second element) or" connected "to another element (e.g., a second element), it is understood that the element is directly connected to the other element or the element is indirectly connected to the other element via yet another element (e.g., a third element). In contrast, it is understood that when an element (e.g., a first element) is referred to as being "directly connected" or "directly coupled" to another element (a second element), no element (e.g., a third element) is interposed therebetween.

The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the invention. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.

Claims

1. A cloud computing service method, comprising:

receiving a model service request initiated by a user;

allocating an execution node for the model service request;

processing the model service request by using a container management application instance on the execution node, wherein the container management application instance at least comprises the following containers: initializing a container and a model container;

and returning the processing result to the user.

2. The method of claim 1, wherein the processing the model service request with the container management application instance comprises:

the initialization container copies the user's data to the execution node local;

and the model container provides model service for the user by utilizing the data of the user.

3. The method of claim 2, wherein the container management application instance further comprises: a service container;

the processing the model service request with the container management application instance further comprises:

the service container provides corresponding performance services of the model services for the user.

4. The method of claim 2, wherein after the initialization container copies the user's data locally to the execution node, the method further comprises:

and destroying the initialization container.

5. The method of claim 2, wherein the model container provides model services to the user using the user's data, comprising:

and the model container defines a corresponding model framework and language according to the requirements of the user, and provides model service for the user through the model framework and the language.

6. The method of claim 3, wherein the initialization container, the model container, and the service container share a network and a disk.

7. The method of any of claims 1-6, wherein the assigning an executing node to the model service request comprises:

and responding to the model service request, and distributing the route of the execution node for the model service request based on the load states of a plurality of candidate execution nodes.

8. The method of claim 7, wherein said assigning the route of the executing node to the model service request based on the load status of a plurality of candidate executing nodes in response to the model service request comprises:

responding to the model service request, and respectively sending load state query requests to a plurality of candidate execution nodes;

and distributing the route of the executing node for the model service request by receiving the current load state information sent by the candidate executing nodes.

9. A cloud computing service method, comprising:

the method comprises the steps that model service requests are sent aiming at container management application instances, wherein the container management application instances comprise service containers and model containers, the service containers are used for providing application services, and the model containers are used for running prediction models;

responding to the model service request, and calling the prediction model in the model container for prediction through the service container;

and returning the prediction result of the prediction model.

10. A cloud computing service method, comprising:

receiving a prediction model deployment request sent aiming at the container management application instance, wherein the container management application instance comprises a service container which is used for providing application service;

responding to the prediction model deployment request, creating a second container and an initialization container in the container management application instance, wherein the model container is used for running a prediction model, and the initialization container is used for obtaining the prediction model;

and acquiring the prediction model into the model container through the initialization container so as to call the prediction model in the model container for prediction through the service container.

11. A cloud computing service method, comprising:

receiving a neural network model service request sent aiming at a container management application instance, wherein the model service request comprises semantic data to be predicted, the container management application instance comprises a service container and a model container, the service container is used for providing application service, and the model container is used for operating a neural network prediction model;

responding the model service request, calling a prediction model in the model container through the service container, and performing semantic processing on the semantic data to be predicted to obtain a semantic processing result;

and returning a prediction result comprising the semantic processing result.

12. A cloud computing service apparatus, comprising:

the model service request receiving module is used for receiving a model service request sent aiming at a container management application instance, wherein the container management application instance comprises a service container and a model container, the service container is used for providing application service, and the model container is used for operating a prediction model;

the container calling module responds to the model service request and calls a prediction model in the model container to predict through the service container;

and the prediction result returning module returns the prediction result of the prediction model.

13. A cloud computing service apparatus, comprising:

the request receiving module is used for receiving a model service request initiated by a user;

the node distribution module is used for distributing execution nodes for the model service request;

a request processing module, which processes the model service request by using a container management application instance on the execution node, wherein the container management application instance at least comprises the following containers: initializing a container and a model container;

and the processing result returning module returns the processing result to the user.

14. A cloud computing service apparatus, comprising:

the deployment request receiving module is used for receiving a prediction model deployment request sent by aiming at the container management application instance, wherein the container management application instance comprises a service container which is used for providing application service;

the container creating module is used for responding to the prediction model deployment request by a container, creating a model container and an initialization container in the container management application instance, wherein the model container is used for running a prediction model, and the initialization container is used for acquiring the prediction model;

and the model acquisition module acquires the prediction model into the model container through the initialization container so as to call the prediction model in the model container for prediction through the service container.

15. A cloud computing service apparatus, comprising:

the device comprises a model service request receiving module, a neural network model service request processing module and a prediction module, wherein the model service request comprises semantic data to be predicted, the container management application instance comprises a service container and a model container, the service container is used for providing application service, and the model container is used for operating a neural network prediction model;

the semantic processing module responds to the model service request, calls a prediction model in the model container through the service container, and performs semantic processing on the semantic data to be predicted to obtain a semantic processing result;

and the prediction result returning module returns the prediction result comprising the semantic processing result.

16. An electronic device, the device comprising:

one or more processors;

a computer readable medium configured to store one or more programs,

when executed by the one or more processors, cause the one or more processors to implement a method as claimed in any one of claims 1-11.

17. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the method of any one of claims 1 to 11.