CN117270987A

CN117270987A - Application starting method and device, electronic equipment and computer readable storage medium

Info

Publication number: CN117270987A
Application number: CN202210674606.8A
Authority: CN
Inventors: 陈国龙
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2022-06-14
Filing date: 2022-06-14
Publication date: 2023-12-22

Abstract

The embodiment of the application provides an application starting method, an application starting device, electronic equipment and a computer readable storage medium, and relates to the fields of cloud technology and games. The method is applied to an application system, the application system comprises an application management server and an application server comprising at least two NUMA nodes, each NUMA node corresponds to a respective storage resource and is provided with at least one path of container, and the application management server executes the operations of: receiving an application starting request aiming at a target application, which is sent by a user terminal; determining a target container based on the application identifier in the application starting request; and sending an application starting request to a target NUMA node to which the target container belongs so that the target NUMA node controls the target container to start the target application based on the target container image file read from the corresponding storage resource. The embodiment of the application accelerates the data reading speed and the starting speed of the target application, and reduces the load pressure of a single NUMA node and the waiting time required for starting a plurality of target applications.

Description

Application starting method and device, electronic equipment and computer readable storage medium

Technical Field

The present application relates to the technical fields of cloud technology, games, and the like, and in particular, to an application starting method, an apparatus, an electronic device, a computer readable storage medium, and a computer program product.

Background

With the development of internet technology and the improvement of application demands, the requirements on the aspects of improving the utilization rate of network resources and storage resources, managing resources and the like are also increasing. Under such a demand, container technology has been developed and rapidly developed. For example, different applications may be launched through different containers during the application launch process.

When an application program is started through a container, the method generally comprises three steps of starting the container, starting an operating system corresponding to the application and starting the application in the container, and the three steps all need to be read and written on a magnetic disk. However, in the related art, in the same container management system, generally, each NUMA node (NUMA node) corresponds to a CPU, but only one NUMA node can be used for reading and writing a disk, and when the disk is read and written, for the NUMA node without a disk, only the disk can be read and written across the NUMA, which results in low disk reading and writing efficiency, slow container starting speed and slow starting speed of an application program, and needs to be improved.

Disclosure of Invention

The embodiment of the application provides an application starting method, an application starting device, electronic equipment, a computer readable storage medium and a computer program product, which can improve the disk reading and writing efficiency and accelerate the container starting speed and the application starting speed.

According to an aspect of an embodiment of the present application, there is provided an application launching method, where the method is applied to an application system, the application system includes an application management server and an application server, the application server includes at least two non-uniform memory access NUMA nodes, each NUMA node has a respective storage resource, and at least one path of container is created in each NUMA node, and the method is executed by the application management server, and includes:

receiving an application starting request aiming at a target application, wherein the application starting request comprises an application identifier of the target application, and the application starting request is sent by a user terminal;

determining a target container corresponding to the application starting request based on the application identifier;

and sending an application starting request to a target NUMA node to which the target container belongs so that the target NUMA node controls the target container to start the target application based on a target container image file of the target application, wherein the target container image file is read by the target NUMA node from a storage resource corresponding to the target NUMA node, and the target container image file comprises a first image file for running the target application.

According to another aspect of the embodiments of the present application, there is provided an application starting apparatus, where the apparatus is applied in an application system, the application system includes an application management server and an application server, the application server includes at least two non-uniform memory access NUMA nodes, each NUMA node has a respective storage resource, and at least one path of container is created in each NUMA node, and the apparatus is implemented by the application management server, and includes:

the application starting request receiving module is used for receiving an application starting request aiming at a target application, which is sent by a user terminal, wherein the application starting request comprises an application identifier of the target application;

the target container determining module is used for determining a target container corresponding to the application starting request based on the application identifier;

and the target application starting module is used for sending an application starting request to a target NUMA node to which the target container belongs so that the target NUMA node controls the target container to start the target application based on a target container image file of the target application, wherein the target container image file is read by the target NUMA node from a storage resource corresponding to the target NUMA node, and the target container image file comprises a first image file for running the target application.

According to yet another aspect of the embodiments of the present application, there is provided an electronic device comprising a memory, a processor and a computer program stored on the memory, the processor executing the computer program to carry out the steps of the above method.

According to a further aspect of the embodiments of the present application, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the above method.

According to a further aspect of the embodiments of the present application, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the steps of the method.

The beneficial effects that technical scheme that this application embodiment provided brought are:

embodiments of the present application provide an application starting method, apparatus, electronic device, computer readable storage medium and computer program product. The application starting method is applied to an application system, the application system comprises an application management server and an application server, the application server comprises at least two NUMA nodes, at least one path of container is created in each NUMA node, each NUMA node corresponds to a respective storage resource, so that container image files corresponding to the container created by each NUMA node can be read from the storage resource corresponding to each NUMA node, corresponding container image files do not need to be read from other NUMA nodes, time cost for reading data across the NUMA nodes is reduced, and data reading speed is accelerated.

By receiving an application starting request for a target application sent by a user terminal, a target container corresponding to the application starting request can be determined based on an application identifier of the target application in the application starting request, and the application starting request is sent to a target NUMA node to which the target container belongs, so that the target NUMA node can control the target container to start the target application according to a target container image file corresponding to the target container which is directly read from a corresponding storage resource, the starting speed of the target application can be increased, and the problems that the data reading speed is slow due to data reading across NUMA nodes, the application starting speed is slow and the like are avoided.

Moreover, by setting at least two NUMA nodes, at least two target applications in the application starting request can be independently and parallelly started through each NUMA node, the application starting speed is further increased, the problem that high load is caused by frequently scheduling and acquiring the target container image file when at least two target applications in the application starting request are sequentially started through one NUMA node is avoided, and the waiting time required for starting other target applications when one NUMA node processes any one target application in the at least two target applications is reduced.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that are required to be used in the description of the embodiments of the present application will be briefly described below.

FIG. 1 shows a schematic diagram of a cloud gaming start-up process provided by an embodiment of the present application;

FIG. 2 shows a schematic diagram of a dual C dual NUMA architecture in the related art;

FIG. 3 illustrates a schematic diagram of an application launch system implemented based on the dual C dual NUMA architecture shown in FIG. 2;

FIG. 4 illustrates a schematic diagram of a dual C dual NUMA architecture provided by embodiments of the application;

FIG. 5 illustrates a schematic diagram of an application launch system based on a multi-C multi-NUMA architecture implementation provided by embodiments of the present application;

FIG. 6 is a schematic diagram of a process for initially creating a container provided by an embodiment of the present application;

FIG. 7 illustrates a schematic diagram of a container start-up process provided in accordance with an embodiment of the present application;

FIG. 8 shows a flowchart of an application launch method provided by an embodiment of the present application;

FIG. 9 shows a flowchart of an application start system provided by an embodiment of the present application;

FIG. 10 illustrates a schematic diagram of an application launch process provided by an embodiment of the present application;

FIG. 11 illustrates a schematic diagram of another application launch process provided by embodiments of the present application;

FIG. 12 illustrates a schematic diagram of another application launch system based on a multi-C multi-NUMA architecture implementation provided by embodiments of the present application;

FIG. 13 illustrates a flow diagram of starting a cloud game, as shown in an embodiment of the present application;

FIG. 14 shows a schematic diagram of an application launching device provided by an embodiment of the present application;

fig. 15 shows a schematic structural diagram of an electronic device to which the embodiment of the present application is applied.

Detailed Description

Embodiments of the present application are described below with reference to the drawings in the present application. It should be understood that the embodiments described below with reference to the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application, and the technical solutions of the embodiments of the present application are not limited.

As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and "comprising," when used in this application, specify the presence of stated features, information, data, steps, operations, elements, and/or components, but do not preclude the presence or addition of other features, information, data, steps, operations, elements, components, and/or groups thereof, all of which may be included in the present application. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein indicates that at least one of the items defined by the term, e.g., "a and/or B" may be implemented as "a", or as "B", or as "a and B".

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

The explanation of the terms involved in the embodiments of the present application is as follows:

cloud Gaming (GS): based on the container, technology, system and kernel technology, the technology of audio and video, network optimization, computing resource management and the like are combined, and a great amount of technical force is put into creating a brand new cloud game platform.

SMP (Symmetrical Multi-Processing, symmetric multiprocessor): in an SMP system, there are two or more CPUs (Central Processing Unit, central processing units) and each CPU is connected via a shared interconnect bus (i.e., IO bus) such that each CPU is provided with equal access to system resources (e.g., memory and input/output ports) via the shared interconnect bus.

NUMA (Non Uniform Memory Access Architecture, non-uniform memory Access), each CPU has its own local memory. NUMA is a derivative of SMP designs that can be found in multi-socket systems where the system memory is divided into units or nodes associated with a particular CPU, namely NUMA units or nodes. Memory on other nodes may be requested through the shared interconnect bus, but due to limited bandwidth on the shared interconnect bus, performance of each NUMA node in the NUMA system may be degraded due to contention for resources.

In OpenStack, the SMP CPU is referred to as a core, and the NUMA units or nodes are referred to as sockets. SMT CPUs are referred to as threads. For example, in a four-way eight-core system with hyper-threading functionality, there are four slots, each with eight cores, each consisting of two threads, for a total of 64 CPUs.

NUMA affinity: the CPU and the used memory which are ensured to run as far as possible in the program belong to the same NUMA node.

NVME (Non-Volatile Memory express): one type of AHCI (Advanced Host Controller Interface ) like protocol built on an m.2 interface (M2 interface, an interface specification) is a protocol specifically designed for flash class storage.

Kubernetes (a container orchestration engine, abbreviated as K8S): a portable and expandable open source platform is used for managing containerized workload and services, and can promote declarative configuration and automation, namely, K8S is essentially a resource arrangement tool, and the resources are computing resources, storage resources and network resources.

kubelet: a proxy component on the Kubernetes working node runs on each node.

Containerd: an industry-standard container is responsible for managing the life cycle of the image and container when running. Emphasis is placed on simplicity, robustness and portability. Containerd may manage the complete container declaration period in the host: the transmission and storage of container images, the execution and management of containers, storage and networking, etc. In detail, the con-tainerd is mainly responsible for: management container lifecycle (from creation of container to destruction of container), pull/push container mirroring, storage management (management of mirroring and storage of container data), call runC runtime container (runtime interaction with runC, etc.), management container network interface and network. Containerd is designed to be embedded in a larger system, rather than being used directly by a developer or end user.

gRPC: a high performance, universal open source RPC framework is used in K8S for communication between components. Wherein the RPC (remote procedure call ) identifies a request for service from a remote computer program over a network without requiring knowledge of the protocol of the underlying network technology. A program running on one computer is allowed to call a subroutine of another computer without encoding the interactive process.

CRI (Container Runtime Interface ): interfaces to the services of the container and the mirror are defined to provide computing resources. Several gRPC APIs (APIs, application interface, application program interfaces) and corresponding message types are specifically defined for managing content such as workload operations, e.g. "pull mirror" (imageservice. Pullimage), "create pod" (runimeservice. Runpodsandbox), "create container" (runimeservice. Createcontainer), "start container" (runimeservice. Startcontainer), "stop container" (runimeservice. Stopcontainer), etc.

CRI plug is a built-in plug-in to the containerd, which contains the two most major parts: image service, namely mirror image service; and secondly, a runtime service, i.e. a runtime service. If an item or service is deployed in the con-tainerd, it will schedule to a machine first, and kubinetes on that machine will start working, it will query which images are needed for the current item or service, then let the relevant images run service down, and then start the corresponding pod or related container. Secondly, it interacts with CNI, and it should be noted that the process of interaction may create a container (sandbox container) of the pause in advance, which is a space-occupying process.

CNI (Container Network Interface ): a service interface of the container network is defined to provide network resources.

CSI (Container Storage Interface ): a service interface of the container store is defined to provide storage resources.

pod (pod): kubernetes cluster manages and schedules minimal units, a pod can encapsulate a container or containers, and the pod is scheduled to run on NUMA nodes by default. The pod can be independently operated as a minimum application instance, and is convenient to deploy, horizontally expand and contract, schedule and manage and allocate resources. Containers in one pod share the same data and network address space (network namespace), and uniform resource management and allocation can be performed between different pods. The container is a single process model and the pod is a multi-process model.

Cgroups (control groups): a mechanism provided by the Linux (an operating system) kernel may restrict, record, isolate the physical resources (e.g., CPU, memory, IO, etc.) used by a process group (process groups).

IOPS (Input/Output Operations Per Second, read/write times per second): a measurement mode for testing the performance of a computer storage device (such as Hard Disk Drive (HDD), solid State Disk (SSD) or Storage Area Network (SAN)).

READY event: events that occur typically when the DOM (Document Object Model ) has been loaded and the page (including the image) has been fully rendered.

Cold start: the program is started in the power-off state, so that the running data (such as a memory) of the system needs to be completely cleared, and the program is reloaded.

And (3) hot start: and the program is started in the uninterrupted state, so that the program can be directly operated without clearing the data of the system operation.

The daemon process is a service end program that runs all the time, also called daemon, and typically runs in the background of the system without a control terminal and without interaction with the foreground. The daemon process is typically used as a system service waiting for a client program to communicate with it. daemon is a long running process that typically runs after the system is started and ends when the system is shut down.

The following describes the process of creating Pod by calling CRI interface by kubelet in combination with the above terms, mainly including three steps:

1. creation of PodSandbox: the corresponding CRI interface is RunPodSandbox through which to create the PodSandbox environment itself, i.e., K8S Pod, in which one of the pause containers is run by default to "hold" the network address space for setting up the Pod network. Wherein, when running with different containers, the implementation of the pod sandbox is also different, for example, using kata as run time, the pod sandbox is implemented as a virtual machine; the use of run as run time, the pod sandbox is a stand-alone Namespace and Cgroups.

2. Creating a podcon tainer: the corresponding CRI interface is CreatePodContainer, through which a container podcon tainer, such as an nginx container, to be run by the user can be created. The created podcon pointer will be added to the podbox and the network address space in the pod will be shared with other containers in the podbox.

3. Start podcon tainer: the corresponding CRI interface is startpodcon pointer, through which the container created in the previous step, i.e. podcon pointer, can be started.

In the application starting method provided in the embodiment of the present application, the target application may be any application, for example, the target application may be any local application or any cloud application. When the target application is a cloud application, the target application may specifically include, but is not limited to, cloud games, online office software, cloud storage, email, map navigation, cloud music, cloud video, weight loss and fitness, financial software, electronic commerce, search engines, and the like. The virtualization technology is one of the technical bases of cloud applications, the cloud applications migrate resources originally at the client to the cloud, and the cloud needs to realize physical-to-virtual conversion in a cloud application running environment in order to manage the resources of the applications more flexibly and efficiently, and the container technology is the lightweight virtualization technology and is suitable for the cloud applications.

Cloud technology (Cloud technology) refers to a hosting technology for integrating hardware, software, network and other series resources in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. The application starting method provided by the embodiment of the application can be realized based on cloud computing (closed computing) in cloud technology.

Cloud Computing refers to obtaining required resources through a network in an on-demand and easily-extensible manner, and is a product of traditional computer and network technology development fusion such as Grid Computing (Grid Computing), distributed Computing (Distributed Computing), parallel Computing (Parallel Computing), utility Computing (Utility Computing), network storage (Network Storage Technologies), virtualization (Virtualization), load balancing (Load Balance) and the like.

Cloud gaming (Cloud gaming), which may also be referred to as game on demand, is an online gaming technology based on Cloud computing technology. Cloud gaming technology enables lightweight devices (thin clients) with relatively limited graphics processing and data computing capabilities to run high quality games. In a cloud game scene, game logic does not run in a player game terminal, but runs in a cloud server, the cloud server renders the game scene into video and audio streams, and the video and audio streams are transmitted to the player game terminal through a network. The player game terminal does not need to have strong graphic operation and data processing capability, and only needs to have basic streaming media playing capability and the capability of acquiring player input instructions and sending the player input instructions to the cloud server.

The target application provided in the embodiment of the present application may be an application applied to any operating system, and the embodiment of the present application does not limit a specific system type of the operating system, for example, may be but is not limited to an Android system.

In order to better understand the application starting method provided in the embodiments of the present application, a description is first given below of a process of performing application starting in the related art with reference to a specific application scenario.

As shown in fig. 1, taking a cloud game application (hereinafter referred to as a cloud game) started in an android system as an example, a cloud game starting process may include three steps: android containers (corresponding containers may be referred to as android containers when the operating system of the user terminal to which the game player corresponds is an android system) are started, android system starts, and in-container game pulls (i.e., starts). In this application scenario, the game start time mainly includes three aspects: container pull-up time, operating system pull-up time, and in-container game pull-up time. In the process of starting the cloud game, the disk read-write is needed in the android container starting process, the android system starting process and the in-container game pulling process, namely, corresponding starting operation is carried out based on resources in the disk.

Taking an amp model of a dual C dual NUMA architecture (i.e., two CPUs, two NUMA nodes, one for each NUMA node), as shown in fig. 2, in the dual C dual NUMA architecture, a NUMA0 node corresponds to one pole (i.e., a set of multiple pole), a CPU0, a memory 0, and a disk 0, a NUMA1 node corresponds to one pole, a CPU1, and a memory 1, and each pole includes one or more containers, and one container corresponds to one cloud game. It can be seen that in this architecture there is only one disk, disk 0, and that disk 0 is in the NUMA0 node, the NUMA0 node shares the storage resources in that disk 0 with the NUMA1 node.

The management content for a container generally includes procedures such as create, start, stop, remove, and exec for the pod accommodating the container and the container itself. The management process may in turn classify two categories, namely, first create containers and other operations. When creating a container, a container ID is generated, and after creating a container, the corresponding container can be controlled to execute corresponding operations according to the pod ID or the container ID in operations such as start, stop, remove and exec.

Considering that in the process of managing a container, the downloading, use, deletion, update, etc. of the container image file is managed mainly by a daemon process (container process, corresponding to one NUMA node), which is strongly related to the management process of the container, and considering that the con-tained is performed inside the NUMA node, the operations performed by the con-tained may also be regarded as the operations performed at its corresponding NUMA node hereinafter, which is an activity throughout the life cycle of the container. Each container may manage one or more containers, and in the process of downloading the container image file through the container, the container image file needs to be downloaded into a root directory (i.e., a root directory, which may be simply referred to as an image management directory hereinafter) in a default configuration file (e.g.,/etc/container/config. Toml) corresponding to the container, and a mapping relationship between each container image file and a corresponding container, and a specific application implemented based on each container is stored. When the corresponding container mirror image file is needed to be used in the container, the container mirror image file of the container to be managed is needed to be inquired and obtained from the corresponding mirror image management catalog through the container mirror image, the obtained container mirror image file is loaded into the container mirror image, and the corresponding container is controlled to execute the operation corresponding to the container mirror image file according to the container mirror image file. It should be noted that each container can only configure one disk location as its mirror management directory, and it is impossible for different containers to use the mirror management directories on different disks.

The native kubelet can only communicate with one con-tained, and fig. 3 shows a schematic diagram of an application launch system implemented based on the dual C dual NUMA architecture shown in fig. 2. As shown in fig. 3, a gRPC request such as container creation, deletion, etc. may be sent by CRI to its corresponding container to control the corresponding container to perform the operation corresponding to the gRPC request.

In the system shown in fig. 3, kubelet calls CRI plug in (CRI plug in) through CRI interface, and controls the con unit to operate on the corresponding con unit through function call component (function call). When the contained operates as a container of kubernetes, the complete default configuration of the contained can be directly queried through a containerd config default command.

As shown in fig. 3, taking as an example the creation and initiation of a cloud game in a container A (container A) by kubelet, the container a corresponds to the NUMA0 node in fig. 2, the specific operation is as follows:

1. kubelet calls a runtime services API (runimeservice.runpodsandbox) in CRI plug through the CRI interface, creating pod a (i.e., one of the pods in the NUMA0 node in fig. 2);

2. CRI plug creates and initiates a particular pause container (i.e., the sandboxed container in fig. 3) using the con-tainerd and places the sandboxed container into the Cgroups and network address space of pod a;

3. CRI plug uses ocicini to configure the network address space of pod a through CNI;

4. kubelet calls a mirror image service API (imageservice. Pullimage) and a connected client (containerd client) API in CRI plug through a CRI interface, and pulls an application container mirror image from disk 0;

5. if the application container image is not on this node, CRI plug uses the containerd to pull the application container image;

6. kubelet calls a runtime service API (runimeservice. Createcontainer) and a con-tined client API in CRI plug in through CRI interface, creates a con-tiner a inside pod a using the pulled-down application container image through a con-tined carrier (con-tined shim), and places it into the Cgroups and namespace of pod a;

7. kubelet invokes a runtime services API (run time device) and a con-tainerd client API in CRI plug through the CRI interface, inside pod a, using the pulled application container image, launch con a, and launch the cloud game in that container a.

When the cloud game is started based on the system shown in fig. 3, when disk reading is needed, for a container process running on the CPU0 corresponding to the NUMA0 node, since the NUMA0 node corresponds to the disk 0, the read-write operation can be directly performed from the disk 0. For a container process running on the CPU1 corresponding to the NUMA1 node, since the NUMA1 node does not have a corresponding disk, a read-write operation needs to be performed in the disk 0 corresponding to the NUMA0 node, and by pulling an application program container mirror image from the disk 0, a corresponding container and an application are started, that is, a disk read-write operation needs to be performed across the NUMA node (hereinafter referred to as across the NUMA).

Since Disk 0 corresponds to NUMA0, disk Cache (also referred to as Disk Buffer) is directly related to IO performance, that is, for a container running on CPU1 corresponding to a NUMA1 node, there may be cases where IO reading and writing are performed across NUMA nodes. It can be seen that, in a dual-C dual-NUMA architecture system, although the NUMA affinity of the CPU can be guaranteed by means of Cgroups or a binding core (i.e. the running of the CPU does not need to cross NUMA nodes), the NUMA affinity of the IO cannot be guaranteed.

Since disk access is usually performed through NVME protocol, taking random 4k hybrid read-write disk as an example, taking read average IOPS, read average bandwidth, write average IOPS, and write average bandwidth as evaluation indexes, when evaluating whether to access performance of disk through NVME across NUMA, index values corresponding to the evaluation indexes are as follows:

according to the experimental data, performance is greatly reduced by about 45% when the disk is accessed by the NVME across NUMA compared with when the disk is accessed by the NVME across NUMA.

Considering the time for accessing the disk, when the cloud game is started specifically, especially when the situation that a large amount of game mirror image data on the disk needs to be loaded into the memory is faced, compared with the situation that NVME without NUMA accesses the disk, when running needs to access the disk with NUMA, the situation that READY event delay is long can occur, so that the game starting speed is slow, and the user experience is affected. It can be seen that the time overhead required to access disks across NUMA is an important factor affecting both container start-up speed and game start-up speed, and how to reduce the time overhead required to access disks across NUMA is critical in order to increase container start-up speed and game start-up speed.

In order to solve the above-mentioned technical problem of how to reduce the time overhead required for accessing the disk across NUMA, a possible embodiment is taken as an example, and an application starting method provided by the embodiment of the present application is described below. The method can be realized based on a system realized by a multi-C multi-NUMA architecture, and specifically, the number of CPUs corresponds to the number of NUMA nodes one by one. The application starting method provided in the embodiment of the present application is described below by taking a system implemented based on a dual C dual NUMA architecture as an example.

FIG. 4 illustrates a schematic diagram of a dual C dual NUMA architecture provided by embodiments of the application. As shown in FIG. 4, in the dual-C dual-NUMA architecture, NUMA0 node corresponds to one pole, CPU0, memory 0 and disk 0, NUMA1 node corresponds to one pole, CPU1, memory 1 and disk 1, each pole comprises one or more containers, and one container corresponds to one path of cloud game. When the disk reading is needed, for the container process running on the CPU corresponding to each NUMA node, the reading and writing operation can be directly performed in the disk corresponding to the NUMA node.

FIG. 5 illustrates a schematic diagram of an application launch system based on a multi-C multi-NUMA architecture implementation provided by embodiments of the present application. As shown in fig. 5, the application starting system is implemented based on a K8S platform, in which a connection relationship between one kubelet and a plurality of NUMA nodes can be established through a middleware component nesoi, that is, a connection relationship between one kubelet and a plurality of containerds can be established through the system shown in fig. 5. Each contained corresponds to one NUMA node, and each contained points the mirror management directory to the directory of the disk (namely, the disk with NUMA affinity) bound by the corresponding NUMA node in the corresponding/etc/contained/config.toml default configuration file as the corresponding mirror management directory.

In the following, two connerends (i.e., connerend 0 and connerend 1) set in the K8S platform will be specifically described as an example, the protocol used when accessing the disk 0 bound to the NUMA0 node, i.e., nvme0, the protocol used when accessing the disk 1 bound to the NUMA1 node, i.e., nvme1, the connerend 0 uses/data 0/connerend as its mirror management directory, and the connerend 1 uses/data 1/connerend as its mirror management directory.

As shown in fig. 5, nesoi is essentially a middleware component of CRI, CNI, CSI implementing kubelet, responsible for the routing of gRPC messages (i.e., selecting one of the containers according to demand), forwarding (forwarding the gRPC message stream to the selected container), and congestion control (requesting a current limit per unit time).

The nesoi mainly plays a role of two-layer camouflage, wherein for kubelet, the nesoi is equivalent to camouflaged container of an exposed/var/run/container port, and communication is carried out between the nesoi and kubelet through a container-run-end or image-service-end socket designated by kubelet. For each con-tinend, nesoi corresponds to a camouflaged kubelet issuing CRI request, and each con-tinend (containerd in CPU nvme0 in CPU 0/con-tinend 0. Pack for n-vme 0 and containerd in CPU n-vme 1 (n-vme 1 in CPU 1) communicates with each other through its designated con-tiner-run-end or image-service-end in its con-tinend configuration file, i.e. independent and distinct sockets (e.g. who out of fig. 5/run/con-tinend 0. Pack for con-tind 0 and/run/con-tinend 1. Pack for con-tind 1, respectively).

As shown in fig. 5, since there are a plurality of containerds in the system, one containerd is required as a receiver and manager of the newly created container when the container is initially created by the system. I.e., how to select a container that meets the affinity of the IO NUMA to accommodate and manage the newly created container.

FIG. 6 is a schematic diagram of a process for initially creating containers provided by embodiments of the present application. As shown in fig. 6, the process of kubelet creating a container through nesoi can be divided into three main steps of pod scheduling, pod creation, and container creation. When the pod scheduling is performed, the allocation of resources is realized by adding a GPU (graphics processing Unit) for recording the use of the GPU in an animation item (resource annotation) of the pod. When a pod is created, the GPU is recorded in the animation corresponding to the pod, and the information (such as/sys/devices/pci 0000/00/01.0/numa_node) of the GPU is extracted from a host machine (i.e. an application management server where the system is located), so that the number (i.e. NUMA node identifier, such as 0/numa_node indicates NUMA number of 0) of the NUMA node where the pod is located is obtained based on the obtained GPU information. According to the NUMA number, the first bit of a pod ID (ID) randomly generated by calling a runtime service API through a CRI interface and the first bit of a container ID randomly generated by calling the runtime service API through the CRI interface are modified to be the NUMA number, the modified pod ID is used as the number of the created pod, the modified container ID is used as the ID of the created container, and the corresponding pod and container are conveniently selected through nesoi based on the pod ID and the container ID, so that the management of the pod and the container is realized. Wherein, when modifying the randomly generated container ID, the modification can also be performed according to the identification information of the relevant position in the pod ID.

In the process of modifying the randomly generated pod ID and the container ID based on the NUMA number, only one bit of the 128 bits is modified, the probability of generating repetition with other randomly generated pod ID and container ID is 1/2-128, the probability is extremely small, and the current pod ID can be regarded as not generating repetition with other randomly generated pod ID, and the current container ID can not be generating repetition with other randomly generated container ID.

After the container is created, when operations such as start (start), stop (stop running), remove (remove), exec (call) and the like need to be performed on the created container, the nesoi may receive an operation request from kubelet based on a container-run-end or a socket of an image-service-end specified by kubelet, convert the operation request into an operation instruction according to a pod ID or a first bit in the container ID in the operation request, select a container corresponding to the first bit in the pod ID or the container ID, and stream the operation instruction onto the selected containerd daemon for processing. The specific processing logic is the same as a single containerd.

It should be noted that, when the nesoi receives the image pulling request from kubelet based on the image-service-end socket specified by kubelet, the nesoi will convert the image pulling request into an image pulling instruction, and forward the image pulling instruction to each containing through the gRPC socket of each containing 1 to notify all containing to pull the container image file at the same time, and store the container image file in the image management directory in the disk configured by each containing, after all containing finishes downloading the container image file, it indicates that the image pulling instruction is executed successfully.

Taking container startup as an example, fig. 7 shows a schematic diagram of a container startup process provided in accordance with an embodiment of the present application. As shown in FIG. 7, when the nesoi receives a startup request for container a from kubelet based on the container-run-end specified by kubelet, it is determined that the container a corresponds to NUMA1, namely NUMA node 1, by extracting the first bit of the ID of container a, namely 1, further determining gRPC socket of container "1" to which NUMA1 is bound to run, namely/var/run/container "1/container" sock ", and after converting the startup request into a startup command, forwarding the startup request to container" a "through/var/run/container" 1/container "sock to control loading of container image files corresponding to container" a "into its corresponding image management directory, and starting container" a "according to the loaded container image files, thereby realizing NUMA isolation of IO without need of reading container files in a disk across NUMA. When the cloud game a corresponding to the container a needs to be started, the cloud game a corresponding to the container a can be started in the container a further according to the read container image file after the container a is started.

Therefore, when the application starting method provided by the embodiment of the application starts the container or the game, the container image files can be loaded into the corresponding image management catalogs respectively, so that NUMA isolation of IO is realized, the time overhead of reading the disk caused by NUMA is avoided, and the starting speed of the games in the multipath cloud game container and each cloud game container is accelerated. And based on experiments, compared with the application starting method in the related art, the application starting method provided by the embodiment of the application can improve the application starting time by more than 50%.

The technical solutions of the embodiments of the present application and technical effects produced by the technical solutions of the present application are described below by describing several exemplary embodiments. It should be noted that the following embodiments may be referred to, or combined with each other, and the description will not be repeated for the same terms, similar features, similar implementation steps, and the like in different embodiments.

Fig. 8 shows a flowchart of an application starting method provided in an embodiment of the present application. The method is applied to the application system shown in fig. 9, the application system 10 includes an application management server 110 and an application server 120, the application server includes at least two non-uniform memory access NUMA nodes 121, each NUMA node corresponds to a respective storage resource, at least one path of container is created in each NUMA node, and the method is executed by the application management server.

The application management server or the application server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server or a server cluster for providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, a CDN (Content Delivery Network ), basic cloud computing services such as big data and an artificial intelligence platform.

It will be appreciated by those skilled in the art that the above servers are merely examples, and that other servers now known or hereafter may be available, as applicable, are also included within the scope of the present application and are incorporated herein by reference.

In this embodiment of the present application, a mapping relationship between a node identifier of each NUMA node and a resource identifier of a storage resource corresponding to each NUMA node, and a mapping relationship between a container identifier of each container and an application identifier corresponding to each container may be stored in advance in a storage resource of an application server.

The embodiment of the application does not limit specific forms of node identification, resource identification, container identification and application identification, and can be configured according to actual application requirements. For example, for the application identification, setting may include, but is not limited to, setting based on a name, a number identification, a letter identification, or the like of an application to which the application identification corresponds, or setting based on a combination of two or more thereof.

Note that, in the container identifier, a node identifier of a NUMA node to which the container identifier belongs may be set. For example, when a container identifier is in digital form, one or more of the container identifiers may be set as a node identifier for a NUMA node to which the container belongs, so that the NUMA node to which the container identifier belongs may be determined based on any of the container identifiers.

And running the container image file corresponding to the path container in each path of container, so that the application corresponding to the path container can be started. In the application system, the applications corresponding to each channel of containers may be the same or different, which is not limited in the embodiment of the present application. Of course, even if the applications corresponding to any two containers are the same, the operating systems or application operators corresponding to the containers may be different. For example, corresponding to the same cloud game, when the corresponding game players are different, the cloud game is started at the player game terminal corresponding to each game player through different containers in the application system.

The application system may be an application start system as shown in fig. 5 in the implementation process, where K8S and kubelet correspond to an application management server, and each connected corresponds to an application server. Where the application server includes two NUMA nodes, the corresponding architecture can be as shown in FIG. 4, with each NUMA node corresponding to one disk (i.e., storage resource) as shown in FIG. 4. The operations performed for each NUMA node are primarily performed by the process in that NUMA node that controls the container to perform the operations, such as the containerized process.

It should be understood by those skilled in the art that the description of the containing process as the process for controlling the operations performed by the container in the NUMA node is merely exemplary, and other processes for controlling the operations performed by the container in the NUMA node may be present or may occur in the future, and are also included in the scope of the present application and are incorporated herein by reference.

When the application server includes three or more NUMA nodes, a new NUMA node may be constructed based on the manner of constructing the NUMA0 node or the NUMA1 node shown in fig. 4, so as to form a NUMA architecture. A new containerd can be constructed based on the construction mode of the containerd0 or the containerd1 shown in fig. 5, and a connection relationship is established with the nesoi, so as to form a new application system. The embodiment of the application does not limit the number of NUMA nodes and the number of containerds in a specific NUMA architecture and a new application system, and can be selected according to actual conditions.

As shown in fig. 8, the method includes the following steps S810 to S830:

step S810: and receiving an application starting request aiming at the target application, wherein the application starting request comprises an application identifier of the target application, and the application starting request is sent by the user terminal.

The user terminal may be a terminal corresponding to the target application. The terminal (which may also be referred to as a User terminal or User Equipment (UE)) may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart voice interaction device (e.g., a smart speaker), a wearable electronic device (e.g., a smart watch), a vehicle-mounted terminal, a smart home appliance (e.g., a smart television), an AR (Augmented Reality), an augmented Reality)/VR (Virtual Reality) device, or the like.

Those skilled in the art will appreciate that the above-described terminals are merely examples, and that other terminals that may be present in the present application or in the future are intended to be included within the scope of the present application and are incorporated herein by reference.

Step S820: and determining a target container corresponding to the application starting request based on the application identifier.

Specifically, a container corresponding to the application identifier of the target application may be determined based on a mapping relationship between each container identifier stored in the storage resource of the application server and the application identifier corresponding to the container identifier, and the determined container is determined as a target container corresponding to the application start request.

Of course, in the case that it is determined that the storage resource of the application server does not include the target container corresponding to the application identifier, the target container corresponding to the application identifier may also be created based on the above manner of creating the container, so as to further execute the subsequent starting process of the target application.

Step S830: and sending an application starting request to a target NUMA node to which the target container belongs so that the target NUMA node controls the target container to start the target application based on a target container image file of the target application, wherein the target container image file is read by the target NUMA node from a storage resource corresponding to the target NUMA node, and the target container image file comprises a first image file for running the target application.

In this example, the target NUMA node to which the target container belongs can be determined by a preset location in the identity of the target container (i.e., one or more bits in the container identity). The target NUMA node to which the target container belongs can also be determined according to the mapping relation between the node identification of each NUMA node and the container identification of each container created by the node identification.

For any container image file, the image file for starting the container corresponding to the container image file and the image file for running the corresponding application in the container corresponding to the container image file can be included.

Corresponding to the target container image file, a first image file for running the target application and a second image file for launching the target container may be included. And after the target NUMA node receives the application starting request, if the target container is in a starting state, directly controlling the target container to start the target application based on the first image file. If the target container is in a non-starting state, the target container can be controlled to start the target container based on the second image file and then start the target application based on the first image file.

In the application starting method, each NUMA node is provided with the respective storage resource, so that when the storage resource needs to be read, the container mirror image file corresponding to the container created by the NUMA node can be read from the storage resource corresponding to each NUMA node, the corresponding container mirror image file does not need to be read from other NUMA nodes, the time cost for reading data across the NUMA nodes is reduced, and the data reading speed is accelerated.

After receiving the application starting request, the target container corresponding to the application starting request can be determined based on the application identifier of the target application in the application starting request, and the application starting request is sent to the target NUMA node to which the target container belongs, so that the target NUMA node can directly read the target container image file corresponding to the target container created by the target NUMA node from the storage resource corresponding to the target NUMA node, and according to the first image file used for running the target application in the target container image file, the target container is controlled to start the target application, the starting speed of the target application can be accelerated, and the problems that the data reading speed is slow due to the fact that data are read across the NUMA node, the starting speed of the subsequent application is slow and the like are avoided.

Moreover, by setting at least two NUMA nodes, at least two target applications in the application starting request can be independently and parallelly started by each NUMA node, the application starting speed is further increased, the problem that when at least two target applications in the application starting request are sequentially started by one NUMA node, the target container image file is frequently scheduled and acquired, so that the high load of the node is caused when only one NUMA node exists is avoided, and the waiting time required for starting other target applications when one NUMA node processes any one target application in the at least two target applications is reduced.

When the target application is a cloud game application, a specific application starting method may be:

in one possible implementation manner, the target application is a cloud game application, the application system is a cloud game scheduling background, the application server is a cloud game server, and the target container is a container of an operating system corresponding to the cloud game application;

the receiving the application starting request for the target application sent by the user terminal includes:

receiving an application starting request for the cloud game application sent by a user terminal, wherein the application starting request comprises an application identifier of the cloud game application and a target operating system corresponding to the user terminal;

The sending an application starting request to a target NUMA node to which a target container belongs, so that the target NUMA node controls the target container to start a target application based on a target container image file of the target application, including:

and sending an application starting request of the cloud game application to a target NUMA node to which the target container belongs, so that the target NUMA node starts the target operating system and the cloud game application based on the container image file of the cloud game application.

When the target application is a cloud game application, the user terminal may be a terminal held by any game player of the cloud game application, and of course, the user terminal may also be a terminal held by a tester for testing the cloud game application. The embodiments of the present application are not limited in this regard.

The cloud game application starting method comprises the steps of determining a target container corresponding to the cloud game application based on the application identifier of the cloud game, starting an operating system corresponding to a user terminal in the target container, and further starting the cloud game application in the container, so that the starting speed of the cloud game application is increased.

In one possible implementation, before receiving the application start request, the method may further include:

determining the storage resources corresponding to each NUMA node in the storage resources of the application server, and storing the binding relation between each NUMA node and the corresponding storage resources;

Creating at least one path container in each NUMA node;

and sending an image file acquisition instruction to each NUMA node so that each NUMA node acquires a container image file corresponding to at least one application according to the image file acquisition instruction, and storing the acquired container image file into a storage resource corresponding to each NUMA node, wherein at least one application corresponding to the at least one NUMA node comprises a target application, and the container image file corresponding to the at least one application comprises the target container image file.

By determining the storage resource corresponding to each NUMA node in the storage resources of the application server and storing the binding relation between each NUMA node and the corresponding storage resource, the application management server can conveniently construct a corresponding pod or container through each NUMA node when creating each NUMA node, and the corresponding storage resource and pod are created to the NUMA node.

It should be noted that after each NUMA node receives the image file obtaining instruction, all container image files corresponding to the application may be obtained, and based on the container creating instruction, a corresponding container image file is selected based on each container identifier in the container creating instruction, and a container corresponding to each container identifier is created. It is also possible to acquire only a part of the application-corresponding container image file and create a corresponding container based on the acquired container image file. The embodiments of the present application are not limited in this regard.

After each NUMA node creates a corresponding container, the node identifier of each NUMA node, the container identifier of each container created by each NUMA node, and the mapping relationship between the application identifiers corresponding to each container can be fed back to the application management server, so that the application management server can determine the container corresponding to the application representation based on the operation request corresponding to the application identifier of any application, and thus the operation request is sent to the NUMA node corresponding to the determined container, so as to control the NUMA node to execute the operation corresponding to the operation request on the corresponding container.

As described above, the target NUMA node may restart the target container after receiving the application start request, or may start the target container before receiving the application start request. The following describes an application start procedure by taking a restart target container after receiving an application start request as an example. Since the target container is started after receiving the application start request, the target container may also be referred to as cold start, and a specific application start procedure is as follows:

in one possible implementation manner, the sending an application start request to the target NUMA node to which the target container belongs, so that the target NUMA node controls the target container to start the target application based on the application start request, so that the target NUMA node controls the target container to start the target application based on the target container image file of the target application may include:

Sending a container starting instruction to a target NUMA node based on an application starting request so that the target NUMA node responds to the container starting instruction and reads a target container image file corresponding to a target application from storage resources corresponding to the target NUMA node, wherein the target container image file can also comprise a second image file for starting a target container;

and sending the image file of the target container to the target container so that the target container starts the target container based on the first image file and starts the target application based on the second image file.

Fig. 10 is a schematic diagram of an application starting procedure according to an embodiment of the present application. As shown in fig. 10, after receiving a cloud game initiated by a game player in an android system, a container image file corresponding to the cloud game is pulled through a cloud game scheduling background (i.e., the application system), and an android container is sequentially initiated according to the container image file, the android system is initiated in the android container, and the cloud game is initiated in the android container.

After receiving the application starting request, the target NUMA node is controlled to read a target container image file corresponding to the target application from storage resources corresponding to the target NUMA node, start the target container based on a first image file in the target container image file, and start the target application based on a second image file, so that the starting speed of the target application is increased.

The following describes an application start procedure by taking a restart target container after receiving an application start request as an example. Since the target container is started before receiving the application start request, the target container may also be called a hot start, and a specific application start process is as follows:

in one possible implementation, after each NUMA node stores the obtained container image file in a storage resource corresponding to each NUMA node, the method may further include:

sending a container start instruction of the target application to at least one NUMA node so that the NUMA node receiving the container start instruction performs the following operations:

reading a target container image file from a storage resource corresponding to the NUMA node;

transmitting the read target container image file to at least one container of the NUMA node so that the container receiving the target container image file starts the container based on the second image file;

the determining, based on the application identifier, the target container corresponding to the application start request may include:

determining a target container from the started containers corresponding to the target application based on the application identification;

the target NUMA node controlling the target container to start the target application based on the target container image file of the target application may include: the target NUMA node sends an application launch indication to the target container to cause the target container to launch the target application based on the first image file.

After each NUMA node stores the acquired container image file into the storage resource corresponding to each NUMA node, a container starting command of a target application is sent to at least one NUMA node, each NUMA node is controlled to read the target container image file and start the corresponding container, after the acquired application starting request is based on the application identifier in the application starting request, the target container is determined from the started containers, the pre-starting of the target container before the application starting request is acquired is realized, the time for starting the target container can be reduced after the acquired application starting request, the target application is directly started in the target container, the time for waiting for starting the target application of a user terminal is reduced, and the user experience is improved.

Fig. 11 is a schematic diagram of another application starting procedure according to an embodiment of the present application. As shown in fig. 11, a target container image file corresponding to 100 paths of containers can be obtained through pre-pulling a scheduling background, any NUMA node is selected to start 100 paths of containers in batches, after a game player starts a cloud game in an android system, the target container corresponding to the cloud game is selected from the started 100 paths of containers through the cloud game scheduling background, and the android system and the cloud game are started in the android container in sequence according to the target container image file, so that the time for the game player to wait for the cloud game to start is reduced.

Of course, the target container image files corresponding to 100 paths of containers can be obtained through pre-pulling a scheduling background, two NUMA nodes are selected to respectively start 50 paths of containers in batches, after a game player starts a cloud game in an android system, the target container corresponding to the cloud game is selected from the 100 paths of containers started through the cloud game scheduling background, the NUMA nodes corresponding to the target container are controlled, and the android system and the cloud game are started in the android container in sequence according to the target container image files.

When the application system is a system implemented based on K8S, considering that kubelet can only establish a communication relationship with one container, namely one NUMA node, in order to simultaneously control different NUMA nodes to perform corresponding operations on corresponding containers, the embodiments of the present application further provide the following alternative implementation manners, which may be specifically implemented by using the system shown in fig. 5:

in one possible implementation, the application management server communicates with at least two NUMA nodes through a middleware component, where the sending an application start request to a target NUMA node to which the target container belongs may include:

based on the container identification of the target container, the following operations are performed by the middleware component:

Determining a target NUMA node to which the target container belongs based on the container identification of the target container;

and sending an application starting request to the target NUMA node through a communication interface corresponding to the target NUMA node.

In this implementation, the application management server may be one of the kubelet, the middleware component may be the nesoi, the communication interface corresponding to the target NUMA node may be a socket specified in the config.toml configuration file of the con-inerd corresponding to the target NUMA node, for example, the above-mentioned container-run-end or image-service-end specifically includes/run/container 0.Sock and/run/container 1.Sock.

The middleware component determines the target NUMA node to which the target container belongs based on the container identification of the target container, and sends an application starting request to the target NUMA node through a communication interface corresponding to the target NUMA node, so that the application system can control a plurality of NUMA nodes to execute corresponding starting requests simultaneously.

In one possible implementation manner, the application management server communicates with the at least two NUMA nodes through proxy components corresponding to each NUMA node, and the sending an application starting request to the target NUMA node to which the target container belongs may include:

Determining a target NUMA node corresponding to a container identifier based on the container identifier of the target container;

and sending an application starting request to the target NUMA node through the proxy component corresponding to the target NUMA node.

In this implementation, the application management server may be one of the above-mentioned K8S, and each proxy component may be one kubelet. I.e. a NUMA node is controlled by a kubelet of the native, so as to realize the control operation on each container corresponding to the NUMA node.

The application management server determines the target NUMA node corresponding to the container identifier based on the container identifier of the target container, and sends an application starting request to each NUMA node through the proxy component corresponding to each NUMA node, so that the control operation of each container corresponding to the NUMA node can be realized, and further, the application system can control a plurality of NUMA nodes to execute corresponding starting requests simultaneously.

FIG. 12 illustrates a schematic diagram of another application launch system based on a multi-C multi-NUMA architecture implementation provided by embodiments of the present application. As shown in fig. 12, the containerd0 can be directly controlled by kubelet0, and the containerd1 can be directly controlled by kubelet 1.

In one possible implementation, the method may further include:

Acquiring a container operation instruction, wherein the container operation instruction comprises a container identifier of a container to be processed;

and sending a container operation instruction to the NUMA node to which the container to be processed belongs based on the container identification of the container to be processed, so that the NUMA node which receives the container operation instruction executes the operation corresponding to the container operation instruction to the container to be processed.

A container operation instruction may include an operation instruction for operations such as pod accommodating the container and create, start, stop, remove and exec of the container itself, which may be determined according to practical situations without limitation in the embodiment of the present application.

By sending the container operation instruction to the NUMA node to which the container to be processed belongs based on the container identification of the container to be processed in the container operation instruction, the NUMA node which receives the container operation instruction can execute the operation corresponding to the container operation instruction to the container to be processed, and control of each container is realized.

Fig. 13 shows a schematic flow chart of starting a cloud game according to an embodiment of the present application. As shown in fig. 13, the start-up procedure may include the steps of:

step 1: binding a disk on each NUMA node in a kubelet;

step 2: pulling up a con-tainerd on each NUMA node, and respectively configuring and using the disk bound on the corresponding NUMA node as a root directory;

Step 3: a nesoi is pulled up in a kubelet and is used as a communication route forwarding module for all the containers and kubelet.

Step 4: and pulling the mirror image through each containerd, and storing the downloaded mirror image on a corresponding disk.

Step 5: and reading the container image file on the corresponding disk through each container, starting the corresponding container based on the read container image file, and starting the cloud game in the corresponding container.

Based on the same principle as the application starting method provided by the embodiment of the application, the embodiment of the application also provides an application starting device. Fig. 14 shows a schematic diagram of an application starting apparatus provided in an embodiment of the present application. As shown in fig. 14, the apparatus 400 is applied to an application system, where the application system includes an application management server and an application server, and the application server includes at least two non-uniform memory access NUMA nodes, each NUMA node has a respective storage resource, and at least one path of container is created in each NUMA node, and the apparatus is implemented by the application management server, and includes:

an application start request receiving module 410, configured to receive an application start request for a target application sent by a user terminal, where the application start request includes an application identifier of the target application;

A target container determining module 420, configured to determine, based on the application identifier, a target container corresponding to the application start request;

and the target application starting module 430 is configured to send an application starting request to a target NUMA node to which the target container belongs, so that the target NUMA node controls the target container to start the target application based on a target container image file of the target application, where the target container image file is read by the target NUMA node from a storage resource corresponding to the target NUMA node, and the target container image file includes a first image file for running the target application.

In one possible implementation, the apparatus further includes a resource management module, a container creation module, and an image file acquisition module,

the resource management module is used for determining the storage resource corresponding to each NUMA node in the storage resources of the application server before receiving the application starting request and storing the binding relation between each NUMA node and the corresponding storage resource;

a container creation module for creating at least one path container in each NUMA node;

the image file acquisition module is used for sending an image file acquisition instruction to each NUMA node so that each NUMA node acquires a container image file corresponding to at least one application according to the image file acquisition instruction, and storing the acquired container image file into a storage resource corresponding to each NUMA node, wherein at least one application corresponding to the at least one NUMA node comprises a target application, and at least one container image file corresponding to the application comprises a target container image file.

In a possible implementation manner, the above-mentioned target application starting module 430 sends an application starting request to a target NUMA node to which the target container belongs, so that the target NUMA node controls the target container to start the target application based on the application starting request, so that the target NUMA node controls the target container to start the target application based on the target container image file of the target application, which may be specifically used:

sending a container starting instruction to a target NUMA node based on an application starting request so that the target NUMA node responds to the container starting instruction and reads a target container image file corresponding to a target application from a storage resource corresponding to the target NUMA node, wherein the target container image file can also comprise a second image file for starting a target container;

In one possible implementation manner, after each NUMA node stores the obtained container image file in a storage resource corresponding to each NUMA node, the apparatus further includes a container pre-starting module, where the container pre-starting module is configured to:

the above-mentioned target container determining module 420 may be specifically configured to, when determining, based on the application identifier, a target container corresponding to the application start request:

In one possible implementation, the application management server communicates with at least two NUMA nodes through a middleware component, and the target application starting module 430 may be specifically configured to, when sending an application starting request to a target NUMA node to which the target container belongs:

In one possible implementation manner, the application management server communicates with the at least two NUMA nodes through proxy components corresponding to each NUMA node, the target application starting module 430 sends an application starting request to a target NUMA node to which the target container belongs, and the target application starting module 430 may be configured in:

In one possible implementation manner, the apparatus may further include a container management module, configured to:

the application start request receiving module 410 may be specifically configured to, when receiving an application start request for a target application sent by a user terminal:

the above-mentioned target application starting module 430 may be specifically configured to, when sending an application starting request to a target NUMA node to which the target container belongs, cause the target NUMA node to control the target container to start the target application based on the target container image file of the target application:

and sending an application starting request of the cloud game application to a target NUMA node to which the target container belongs, so that the target NUMA node starts a target operating system and the cloud game application based on the container image file of the cloud game application.

The apparatus of the embodiments of the present application may perform the method provided by the embodiments of the present application, and implementation principles of the method are similar, and actions performed by each module in the apparatus of each embodiment of the present application correspond to steps in the method of each embodiment of the present application, and detailed functional descriptions of each module of the apparatus may be referred to in the corresponding method shown in the foregoing, which is not repeated herein.

Based on the same principle as the application starting method and apparatus provided in the embodiments of the present application, an electronic device (such as a server) is also provided in the embodiments of the present application, where the electronic device may include a memory, a processor, and a computer program stored on the memory, where the processor executes the computer program to implement the steps of the method provided in any of the alternative embodiments of the present application.

Alternatively, fig. 15 shows a schematic structural diagram of an electronic device to which the embodiment of the present application is applied, as shown in fig. 15, an electronic device 4000 shown in fig. 15 includes: a processor 4001 and a memory 4003. Wherein the processor 4001 is coupled to the memory 4003, such as via a bus 4002. Optionally, the electronic device 4000 may further comprise a transceiver 4004, the transceiver 4004 may be used for data interaction between the electronic device and other electronic devices, such as transmission of data and/or reception of data, etc. It should be noted that, in practical applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 is not limited to the embodiment of the present application.

The processor 4001 may be a CPU (Central Processing Unit ), general purpose processor, DSP (Digital Signal Processor, data signal processor), ASIC (Application Specific Integrated Circuit ), FPGA (Field Programmable Gate Array, field programmable gate array) or other programmable logic device, transistor logic device, hardware components, or any combination thereof. Which may implement or perform the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. The processor 4001 may also be a combination that implements computing functionality, e.g., comprising one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

Bus 4002 may include a path to transfer information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus or an EISA (Extended Industry Standard Architecture ) bus, or the like. The bus 4002 can be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in fig. 15, but not only one bus or one type of bus.

Memory 4003 may be, but is not limited to, ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, RAM (Random Access Memory ) or other type of dynamic storage device that can store information and instructions, EEPROM (Electrically Erasable Programmable Read Only Memory ), CD-ROM (Compact Disc Read Only Memory, compact disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store a computer program and that can be Read by a computer.

The memory 4003 is used for storing a computer program that executes an embodiment of the present application, and is controlled to be executed by the processor 4001. The processor 4001 is configured to execute a computer program stored in the memory 4003 to realize the steps shown in the foregoing method embodiment.

Embodiments of the present application provide a computer readable storage medium having a computer program stored thereon, where the computer program, when executed by a processor, may implement the steps and corresponding content of the foregoing method embodiments.

The embodiments of the present application also provide a computer program product, which includes a computer program, where the computer program can implement the steps of the foregoing method embodiments and corresponding content when executed by a processor.

The terms "first," "second," "third," "fourth," "1," "2," and the like in the description and in the claims of this application and in the above-described figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the present application described herein may be implemented in other sequences than those illustrated or otherwise described.

It should be understood that, although the flowcharts of the embodiments of the present application indicate the respective operation steps by arrows, the order of implementation of these steps is not limited to the order indicated by the arrows. In some implementations of embodiments of the present application, the implementation steps in the flowcharts may be performed in other orders as desired, unless explicitly stated herein. Furthermore, some or all of the steps in the flowcharts may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or phases may be performed at the same time, or each of these sub-steps or phases may be performed at different times, respectively. In the case of different execution time, the execution sequence of the sub-steps or stages may be flexibly configured according to the requirement, which is not limited in the embodiment of the present application.

The foregoing is merely an optional implementation manner of the implementation scenario of the application, and it should be noted that, for those skilled in the art, other similar implementation manners based on the technical ideas of the application are adopted without departing from the technical ideas of the application, and also belong to the protection scope of the embodiments of the application.

Claims

1. The application starting method is characterized in that the method is applied to an application system, the application system comprises an application management server and an application server, the application server comprises at least two non-uniform memory access (NUMA) nodes, each NUMA node corresponds to a respective storage resource, and at least one path of container is created in each NUMA node; the method is performed by the application management server, the method comprising:

and sending the application starting request to a target NUMA node to which the target container belongs so that the target NUMA node controls the target container to start the target application based on a target container image file of the target application, wherein the target container image file is read from a storage resource corresponding to the target NUMA node by the target NUMA node, and the target container image file comprises a first image file for running the target application.

2. The method of claim 1, wherein prior to receiving the application launch request, the method further comprises:

creating the at least one path container in each NUMA node;

and sending an image file acquisition instruction to each NUMA node, so that each NUMA node acquires a container image file corresponding to at least one application according to the image file acquisition instruction, and storing the acquired container image file into a storage resource corresponding to each NUMA node, wherein the at least one application corresponding to the at least one NUMA node comprises the target application, and the container image file corresponding to the at least one application comprises the target container image file.

3. The method of claim 2, wherein the sending the application launch request to the target NUMA node to which the target container belongs to cause the target NUMA node to control the target container to launch the target application based on a target container image of the target application comprises:

Sending a container starting instruction to the target NUMA node based on the application starting request so that the target NUMA node responds to the container starting instruction and reads the target container image file corresponding to the target application from a storage resource corresponding to the target NUMA node, wherein the target container image file further comprises a second image file for starting the target container;

and sending the target container image file to the target container so that the target container starts the target container based on the first image file and starts the target application based on the second image file.

4. The method of claim 2, wherein after each of the NUMA nodes stores the obtained container image file in a respective storage resource of each of the NUMA nodes, the method further comprises:

sending a container start instruction of the target application to the at least one NUMA node to enable the NUMA node that receives the container start instruction to:

reading the target container image file from the storage resource corresponding to the NUMA node;

The determining, based on the application identifier, a target container corresponding to the application start request includes:

the target NUMA node controls the target container to start the target application based on a target container image file of the target application, and the target NUMA node comprises the following steps: and the target NUMA node sends an application starting instruction to the target container so that the target container starts the target application based on the first image file.

5. The method of claim 1, wherein the application management server communicates with the at least two NUMA nodes through a middleware component, the sending the application launch request to the target NUMA node to which the target container belongs comprising:

based on the container identification of the target container, performing, by the middleware component:

and sending the application starting request to the target NUMA node through a communication interface corresponding to the target NUMA node.

6. The method of claim 1, wherein the application management server communicates with the at least two NUMA nodes through a proxy component corresponding to each of the NUMA nodes, the sending the application launch request to the target NUMA node to which the target container belongs comprising:

Determining a target NUMA node corresponding to the container identifier based on the container identifier of the target container;

and sending the application starting request to the target NUMA node through a proxy component corresponding to the target NUMA node.

7. The method according to claim 2, wherein the method further comprises:

and sending the container operation instruction to a NUMA node to which the container to be processed belongs based on the container identification of the container to be processed, so that the NUMA node which receives the container operation instruction executes the operation corresponding to the container operation instruction on the container to be processed.

8. The method of claim 1, wherein the target application is a cloud gaming application, the application system is a cloud gaming scheduling background, the application server is a cloud gaming server, and the target container is a container of an operating system corresponding to the cloud gaming application;

the receiving the application starting request for the target application sent by the user terminal comprises the following steps:

receiving an application starting request for the cloud game application sent by the user terminal, wherein the application starting request comprises an application identifier of the cloud game application and a target operating system corresponding to the user terminal;

The sending the application starting request to the target NUMA node to which the target container belongs so that the target NUMA node controls the target container to start the target application based on the target container image file of the target application comprises the following steps:

and sending an application starting request of the cloud game application to a target NUMA node to which the target container belongs, so that the target NUMA node starts the target operating system and the cloud game application based on a container image file of the cloud game application.

9. An application starting device is characterized in that the device is applied to an application system, the application system comprises an application management server and an application server, the application server comprises at least two non-uniform memory access (NUMA) nodes, each NUMA node corresponds to a respective storage resource, and at least one path of container is created in each NUMA node; the device is realized by the application management server and comprises:

an application starting request receiving module, configured to receive an application starting request for a target application sent by a user terminal, where the application starting request includes an application identifier of the target application;

and the target application starting module is used for sending the application starting request to a target NUMA node to which the target container belongs so that the target NUMA node controls the target container to start the target application based on a target container image file of the target application, wherein the target container image file is read by the target NUMA node from a storage resource corresponding to the target NUMA node, and the target container image file comprises a first image file for running the target application.

10. An electronic device comprising a memory, a processor and a computer program stored on the memory, characterized in that the processor executes the computer program to carry out the steps of the method according to any one of claims 1-8.

11. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any of claims 1-8.

12. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any of claims 1-8.