WO2017113074A1 - Procédé, dispositif et système d'allocation de ressource - Google Patents

Procédé, dispositif et système d'allocation de ressource Download PDF

Info

Publication number
WO2017113074A1
WO2017113074A1 PCT/CN2015/099258 CN2015099258W WO2017113074A1 WO 2017113074 A1 WO2017113074 A1 WO 2017113074A1 CN 2015099258 W CN2015099258 W CN 2015099258W WO 2017113074 A1 WO2017113074 A1 WO 2017113074A1
Authority
WO
WIPO (PCT)
Prior art keywords
resource
container
application controller
manager
pool
Prior art date
Application number
PCT/CN2015/099258
Other languages
English (en)
Chinese (zh)
Inventor
梁殿鹏
赵彦荣
刘佳
党李飞
彭磊
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to CN201580084802.8A priority Critical patent/CN108293041B/zh
Priority to PCT/CN2015/099258 priority patent/WO2017113074A1/fr
Publication of WO2017113074A1 publication Critical patent/WO2017113074A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/40—Network security protocols

Definitions

  • the embodiments of the present invention relate to the field of computers, and in particular, to a resource allocation method.
  • Yarn Yet Another Resource Negotiator (yarn) is a new Hadoop resource manager. It is a universal resource management system that provides unified resource management and scheduling for upper-layer applications. The benefits of utilization, unified resource management, and data sharing have brought tremendous benefits. Yarn was originally designed to fix the apparent lack of MRv1 and to improve scalability, reliability, and cluster utilization. Yarn implements these requirements by dividing Job Tracker's two main functions (resource management and job scheduling/monitoring) into two separate service programs, one for the Global Resource Manager (RM) and for each Job Master (AM) for each job.
  • RM Global Resource Manager
  • AM Job Master
  • Each job in Yarn enables a separate AM, which solves the single point of failure and expansion bottleneck in MRv1. But this approach will introduce a new problem: the job delay is large.
  • Each job first needs to start an AM with the RM application resource. After the AM requests the resource from the RM and starts the resource container, the job can be officially started. So the Yarn job will have a long run delay, which is not conducive to running small jobs, and because you need to apply for an AM for each job, you need more computing resources.
  • an embodiment of the present invention provides a resource allocation method, device, and system, Reduce the waiting time for jobs.
  • the present application provides a method for allocating resource containers in a distributed system.
  • the distributed system includes a resource manager and a node manager.
  • the resource manager manages node resources of the distributed system, and the node manager starts based on the node resources.
  • the resource container is used to execute the task of the application.
  • the method includes: the resource manager starts the application controller when the trigger timing is met, and configures an initial specification of the resource pool managed by the application controller, where the initial specification of the resource pool is used.
  • the resource manager After instructing the application controller to apply for the number and specification of the resource container for the first time to the resource manager, the resource manager receives the first resource request of the resource pool sent by the application controller according to the initial specification of the resource pool, according to the resource pool a resource request, allocate an initial resource container for the resource pool, and send a first resource allocation message of the resource pool to the application controller, where the first resource allocation message includes a resource pool allocation managed by the resource manager for the application controller.
  • the resource manager actively starts the application controller and configures the initial specification of the resource pool managed by the application controller, so that the application controller applies a certain number of resource containers to the resource manager in advance according to the initial specification of the resource pool, thereby starting the application in advance.
  • the container reduces the time that subsequent application jobs wait for the resource container to start.
  • the resource manager starting the application controller when the triggering timing is met includes: the resource manager receiving the request for pre-launching the application controller or pre-configuring the resource pool The request is initiated, or when the system is initialized, the application controller is started.
  • the startup time of the application controller can be started in a variety of ways. It can be started by the administrator according to the user's requirements, or can be started by the resource manager through system configuration.
  • the initial specification of the resource pool managed by the resource manager configuration application controller includes: the resource manager according to The expected resource requirement information of the preset application, the initial specification of the resource pool managed by the application controller is configured; or the resource manager configures the resource pool managed by the application controller according to the collected usage information of the node resource of the distributed system.
  • the initial specifications For example, when there are sufficient unused node resources, the resource manager can configure the initial specification of the larger resource pool; when there are fewer unused node resources, the resource manager can configure the initial of the smaller resource pool. specification.
  • the initial specification of the resource pool can be carried by the administrator in the startup command of the application controller, and the resource manager starts the response.
  • configure the application controller When using the controller, configure the application controller.
  • the present application provides a computer readable medium, comprising computer executed instructions, when the processor of the computer executes the computer to execute an instruction, the computer executes the first aspect or any of the possible implementations of the first aspect Methods.
  • the present application provides a computing device, including: a processor, a memory, a bus, and a communication interface; the memory is configured to store an execution instruction, the processor is connected to the memory through the bus, when the computing device is running The processor executes the execution instructions stored by the memory to cause the computing device to perform the method of any of the first aspect or the first aspect.
  • the present application provides a method for allocating resource containers in a distributed system, where the distributed system includes a resource manager and a node manager, where the resource manager is used to manage node resources of the distributed system, and the node manager uses Starting a resource container based on the node resource, where the resource container is used to execute an application task, and the resource manager starts an application controller when the trigger timing is met, and configures an initial specification of the resource pool managed by the application controller, and according to the resource The initial specification of the pool allocates an initial resource container for the resource pool managed by the application controller, and the initial resource container of the resource pool is started;
  • the method includes: receiving, by the application controller, a resource allocation request from a client, where the resource allocation request is used to request a resource container for an application running on the client, where the resource allocation request carries resource requirement information of the application, and according to the application Resource requirement information, from which an idle resource container is selected and allocated to the client.
  • the method before the application controller receives the resource allocation request from the client, the method further includes: the application controller according to an initial specification of the resource pool
  • the resource manager sends a first resource request, where the first resource request carries a quantity of resources determined according to an initial specification of the resource pool, and acquires a first resource allocation message of the resource pool sent by the resource manager, where the first resource is
  • the allocation message includes information about the node where the resource manager allocates the initial resource container for the resource pool, and sends a startup request to the node manager to request the node manager to start the initial resource container of the resource pool.
  • the resource container in the resource pool is applied to the resource manager in advance by the application controller, when the application controller receives the resource allocation request from the client, the resource container can be allocated to the client in time, thereby avoiding In the prior art, the waiting time of the AM and the resource container is started after receiving the application job.
  • the resource allocation request includes an identifier of the client, where the application controller is in the resource pool After the idle resource container is allocated to the client, the method further includes: the application controller sending an indication message to each resource container in the resource container allocated to the client, where the indication message carries the identifier of the client, the indication A message is used to indicate that each resource container is assigned to the client.
  • the method further includes: the application controller determining the remaining idle resource containers in the resource pool The number, if the number of remaining free resource containers is less than a preset first threshold, sending a second resource request to the resource manager, and receiving a second resource allocation message of the resource pool of the resource manager, according to the The second resource allocation message of the resource pool sends a startup request to the node manager, requesting the node manager to start a new resource container allocated for the resource pool.
  • the number of idle resource containers remaining in the resource pool may be determined, or the application controller periodically determines the resource pool.
  • the number of remaining idle resource containers When the number of remaining free resource containers is less than a preset first threshold, indicating that the resource container of the resource pool may not satisfy the subsequent allocation operation, the resource is re-applied to the resource manager. A container to supplement the resource container in the resource pool.
  • the method further includes: determining, by the application controller, the remaining idle resource containers in the resource pool And a quantity release message, if the number of remaining idle resource containers is greater than a preset second threshold, sending a resource release message to the at least one resource container in the idle resource container, where the resource release message is used to release the at least one resource container occupation resource of.
  • the number of idle resource containers remaining in the resource pool may be determined, or the application controller periodically determines remaining free resources in the resource pool.
  • the number of containers when the number of remaining free resource containers is greater than a preset second threshold, sending a resource release message to at least one resource container in the idle resource container. Thereby releasing the resources occupied by the resource pool and ensuring the rational use of the global resources.
  • the application controller maintains state information of each resource container in the resource pool, the state information Indicates whether the corresponding resource container is idle; the application controller selects an idle resource container from the resource pool and allocates the resource to the client. The application controller selects the resource pool according to status information of each resource container in the resource pool. An idle resource container, and the selected resource container in the selected resource pool is allocated to the client.
  • the application controller may maintain a state information table, where the state information table maintains state information of each resource container in the resource pool, so that the application controller can quickly determine the current idle resource container, and according to the idle resources.
  • Container and resource allocation requests allocate containers for clients.
  • the sixth possible implementation manner of the fourth aspect after the application controller selects an idle resource container from the resource pool and allocates the The method further includes the application controller setting a state of each of the resource containers allocated to the client to be not idle.
  • the application controller allocates a status of each resource container in the resource container of the client After being set to not idle, the method further includes: the application controller receiving a status update message from each of the resource containers allocated to the client, the status update message being used to indicate that the task assigned by the client is completed; The application controller sets the state of each resource container in the resource container assigned to the client to idle.
  • the application controller dynamically updates the state information maintained by itself according to the change of the state of the resource container, thereby ensuring that the idle resource container in the resource pool can be accurately determined when there is a resource allocation request.
  • the resource allocation request further includes user rights information; the method further includes: the application controller The user rights information is verified according to a preset user permission library, and the user rights library contains the user rights information.
  • the user permission library contains user rights information of different users. If the user permission information of the client is not included in the user permission library, the resource allocation request of the client is rejected or not responded; If the user rights library contains the user rights information of the client 102, the subsequent resource allocation step is performed.
  • the present application provides a computer readable medium, comprising computer executed instructions, in a possible implementation manner of the fourth aspect or the fourth aspect, when the processor of the computer executes the computer to execute the instruction Methods.
  • the present application provides a computing device, including: a processor, a memory, a bus, and a communication interface; the memory is configured to store an execution instruction, the processor is connected to the memory through the bus, when the computing device is running The processor executes the execution instructions stored by the memory to cause the computing device to perform the method of any of the possible implementations of the fourth aspect or the fourth aspect.
  • the present application provides a resource container allocation apparatus in a distributed system, where the distributed system includes the apparatus and a node manager, where the apparatus is configured to manage node resources of the distributed system, and the node manager is configured to be based on The node resource starts a resource container, where the resource container is used to perform an application task, and the device includes: a startup unit, configured to start an application controller when the trigger timing is met, and configure an initial specification of the resource pool managed by the application controller; a receiving unit, configured to receive a first resource request of the resource pool that is sent by the application controller according to an initial specification of the resource pool; and an allocating unit, configured to allocate an initial resource container to the resource pool according to the first resource request; And a sending unit, configured to send, to the application controller, a first resource allocation message of the resource pool, where the first resource allocation message includes information about a node where the initial resource container allocated by the allocation unit is allocated to the resource pool.
  • a startup unit configured to start an application controller when the trigger timing is met, and configure an initial
  • the device actively starts the application controller and configures an initial specification of the resource pool managed by the application controller, so that the application controller applies for a certain number of resource containers in advance according to the initial specification of the resource pool, thereby starting the application container in advance. , reducing the time that subsequent application jobs wait for the resource container to start.
  • the starting unit is configured to start the application controller when the trigger timing is met, including: the startup unit is configured to receive a request for pre-launching the application controller or The resource pool request is pre-configured, or the application controller is started when the system is initialized.
  • the startup time of the application controller can be started in a variety of ways. It can be started by the administrator according to the user's requirements, or can be started by the startup unit when the system is initialized.
  • the starting unit is configured to configure a resource pool managed by the application controller
  • the initial specification includes: the startup unit is configured to configure an initial specification of a resource pool managed by the application controller according to the expected resource requirement information of the preset application; or configure according to the collected usage information of the node resource of the distributed system.
  • the initial specification of the resource pool managed by the application controller For example, when there are sufficient unused node resources, the startup unit can configure an initial specification of a larger resource pool; when there are fewer unused node resources, the startup unit can configure an initial specification of a smaller resource pool.
  • the initial specification of the resource pool may be carried by the administrator in the startup command of the application controller, and the startup controller activates the application controller to the application controller. Configure it.
  • the application provides a resource container allocation apparatus in a distributed system, where the distributed system includes a resource manager and a node manager, and the resource manager is configured to manage node resources of the distributed system, and the node manager uses Starting a resource container based on the node resource, where the resource container is used to execute an application task, the resource manager starts the device when the trigger timing is met, configures an initial specification of the resource pool managed by the device, and according to the initial of the resource pool The specification allocates an initial resource container for the resource pool managed by the device, and the initial resource container is started;
  • the device includes: a receiving unit, configured to receive a resource allocation request from a client, where the resource allocation request is used to request a resource container for an application running on the client, where the resource allocation request carries resource requirement information of the application; And selecting, according to the resource requirement information of the application, an idle resource container from the resource pool to be allocated to the client.
  • the apparatus further includes: a sending unit, the receiving unit is configured to use, according to the initial of the resource pool, a resource allocation request from the client The first resource request is sent to the resource manager, where the first resource request carries the quantity of resources determined according to the initial specification of the resource pool; the receiving unit is further configured to receive the first resource allocation message sent by the resource manager, The first resource allocation message includes information about a node where the initial resource container allocated by the resource manager is allocated to the resource pool; the sending unit is further configured to send a startup request to the node manager, requesting the node manager to start the The initial resource container for the resource pool.
  • the allocating unit can allocate the activated resource container to the client in time, thereby avoiding In the prior art, the waiting time of the AM and the resource container is started after receiving the application job.
  • the device further includes a sending unit, where the resource allocation request includes an identifier of the client, and the sending unit selects an idle resource container from the resource pool and allocates the resource to the client, where the sending unit is used for An indication message is sent to each resource container in the resource container allocated to the client, where the indication message carries an identifier of the client, and the indication message is used to indicate that each resource container is allocated to the client.
  • the apparatus further includes a determining unit, and a sending unit, the determining unit is configured to determine the resource pool The number of the remaining free resource containers, if the number of remaining free resource containers is less than a preset first threshold, the sending unit is configured to send a second resource request to the resource manager, and the receiving unit is further configured to receive the resource. a second resource allocation message of the resource pool sent by the manager, the sending unit is further configured to send a startup request to the node manager according to the second resource allocation message of the resource pool, requesting the node manager to start a new allocation for the resource pool Increased resource container.
  • the determining unit determines the number of free resource containers remaining in the resource pool, or periodically determines the resource pool by the determining unit.
  • the number of remaining idle resource containers when the number of remaining free resource containers is less than a preset first threshold, indicating that the resource container of the resource pool may not satisfy the subsequent allocation operation, and then re-send to the resource manager through the sending unit. Apply for a resource container to supplement the resource container in the resource pool.
  • the device further includes a determining unit, and a sending unit, the determining unit is configured to determine the resource pool The number of remaining idle resource containers, if the number of remaining free resource containers is greater than a preset second threshold, the sending unit is configured to send a resource release message to at least one resource container in the idle resource container, where The resource release message is used to release resources occupied by the at least one resource container.
  • the determining unit determines the number of free resource containers remaining in the resource pool, or periodically determines, by the determining unit, the remaining idle in the resource pool.
  • the number of resource containers when the remaining free resource containers If the number is greater than the preset second threshold, the resource release message is sent by the sending unit to the at least one resource container in the idle resource container. Thereby releasing the resources occupied by the resource pool and ensuring the rational use of the global resources.
  • the allocating unit is further configured to maintain state information of each resource container in the resource pool, where The status information indicates whether the corresponding resource container is idle; the allocation unit is configured to select an idle resource container from the resource pool, and the allocation unit is configured to: according to the status information of each resource container in the resource pool, Select an idle resource container in the resource pool and assign the selected resource container in the resource pool to the client.
  • the allocating unit may maintain a state information table, where the state information table maintains state information of each resource container in the resource pool, so that the allocating unit can quickly determine the current idle resource container, and according to the idle resource container and A resource allocation request allocates a container for the client.
  • the allocation unit selects an idle resource container from the resource pool and allocates the virtual resource container to the client, It is also used to set the state of each resource container in the resource container assigned to the client to not idle.
  • the allocating unit sets a state of each resource container in the resource container allocated to the client After not being idle, the receiving unit is further configured to receive a status update message from each of the resource containers allocated to the client, the status update message is used to indicate that the task assigned by the client is completed; Used to set the state of each resource container in the resource container assigned to the client to idle based on the status update information.
  • the allocation unit dynamically updates the state information maintained by itself according to the change of the state of the resource container, thereby ensuring that the idle resource container in the resource pool can be accurately determined when there is a resource allocation request.
  • the resource allocation request further includes user rights information; the receiving unit is further configured to be configured according to the preset The user permission library verifies the user rights information, and the user permission library contains the user rights information.
  • the user permission library contains user rights information of different users. If the user permissions are not in the library If the user rights information of the client is included, the resource allocation request of the client is rejected or not; if the user rights information of the client 102 is included in the user rights library, the subsequent resource allocation step is performed.
  • the present application provides a resource container allocation system in a distributed system, where the distributed system includes a resource manager and a node manager, and the resource manager is configured to start an application controller when a trigger timing is met.
  • Configuring an initial specification of the resource pool managed by the application controller, and receiving a first resource request of the resource pool sent by the application controller according to the initial specification of the resource pool, and initializing the resource pool according to the first resource request The resource container sends a first resource allocation message of the resource pool to the application controller; the application controller is configured to acquire a first resource allocation message of the resource pool sent by the resource manager, according to the resource pool Information indicating a node where the initial resource container allocated for the resource pool is indicated in a resource allocation message, requesting the node manager to start an initial resource container of the resource pool; the node manager is configured to use according to the application The controller's request to start the initial resource container.
  • the resource manager is configured to start the application controller when the trigger timing is met, and the resource manager is configured to receive the request for starting the application controller in advance The application controller is started when the request for the resource pool is pre-configured.
  • the initial specification of the resource pool used by the resource manager to configure the application controller includes: The resource manager is configured to configure an initial specification of the resource pool managed by the application controller according to the preset resource requirement information of the application set in advance or according to the collected usage information of the node resource of the distributed system.
  • the application controller is further configured to receive a resource allocation request from a client, where the resource allocation request is And configured to request a resource container for an application running on the client, and select an idle resource container from the resource pool to allocate to the client according to the resource requirement information of the application in the resource allocation request.
  • the resource allocation request includes an identifier of the client, where the application controller is in the resource pool After the idle resource container is allocated to the client, the method further includes: sending an indication message to each resource container in the resource container allocated to the client, where the indication message is carried The identifier of the client, the indication message is used to indicate that each resource container is allocated to the client.
  • the application controller is further configured to: determine the remaining idle resource containers in the resource pool a number, if the number of remaining free resource containers is less than a preset first threshold, sending a second resource request to the resource manager, and receiving a second resource allocation message of the resource pool of the resource manager, according to the resource The second resource allocation message of the pool sends a start request to the node manager, requesting the node manager to start a new resource container allocated for the resource pool.
  • the application controller is further configured to: determine the remaining idle resource containers in the resource pool a number, if the number of remaining free resource containers is greater than a preset second threshold, sending a resource release message to at least one resource container in the idle resource container, the resource release message being used to release the at least one resource container Resources.
  • the application controller is further configured to maintain state information of each resource container in the resource pool, in a seventh possible implementation manner of the ninth aspect,
  • the status information is used to indicate whether the corresponding resource container is idle.
  • the application controller selects an idle resource container from the resource pool to be allocated to the client, and is further configured to: according to status information of each resource container in the resource pool. To determine the free resource container in the resource pool.
  • the application controller selects an idle resource container from the resource pool and allocates the Also used to: set the state of each resource container in the resource container assigned to the client to not idle.
  • the application controller allocates a status of each resource container in the resource container of the client After being set to not idle, the method is further configured to: receive a status update message from each of the resource containers allocated to the client, the status update message is used to indicate that the task assigned by the client is completed, and is assigned to the The status of each resource container in the client's resource container is set to idle.
  • the resource allocation request further includes user rights information;
  • the system further includes: the application controller
  • the user rights information is verified according to a preset user permission library, and the user rights library contains the user rights information.
  • the ninth aspect is the system implementation corresponding to the first aspect and the fourth aspect, the first aspect or any one of the possible implementation manners of the first aspect, and the feature description in any one of the possible implementation manners of the fourth aspect or the fourth aspect Applicable to any possible implementation of the ninth aspect or the ninth aspect, and details are not described herein again.
  • the resource container is applied to the resource manager in advance by the application controller, and the requested resource container is started in advance.
  • the resource container resource can be allocated in time to avoid The waiting time of the resource container is started, and the resource container in the resource pool is reused, thereby avoiding resource consumption of multiple opening and closing of the resource container, and directly managing the resource container in the resource pool through the application controller, thereby realizing more Flexible management.
  • FIG. 1 is a block diagram of an exemplary networked environment of a resource allocation system in accordance with an embodiment of the present invention
  • FIG. 2 is a schematic structural diagram of a hardware of a computing device according to an embodiment of the invention.
  • FIG. 3 is a signaling diagram of a resource allocation method according to an embodiment of the present invention.
  • FIG. 4 is a signaling diagram of a resource allocation method according to an embodiment of the present invention.
  • FIG. 5 is a schematic diagram showing the logical structure of a resource allocation apparatus according to an embodiment of the present invention.
  • FIG. 6 is a schematic diagram showing the logical structure of a resource allocation apparatus according to an embodiment of the present invention.
  • FIG. 7 is a schematic diagram showing the logical structure of a resource allocation apparatus according to an embodiment of the present invention.
  • Resource Manager is a global resource manager responsible for resource management and allocation of the entire yarn system.
  • a resource container is a resource abstraction in yarn that encapsulates multiple types of resources on a node. Such as memory, CPU, disk, network, etc.
  • the resource container is used to perform the tasks of the application.
  • the client is a device running an application to be applied for, and the type of the application may be MapReduce, Giraph, Storm, Spark, Tez/Impala, or Message Passing Interface (MPI).
  • MapReduce MapReduce, Giraph, Storm, Spark, Tez/Impala, or Message Passing Interface (MPI).
  • the Node Manager is a resource and task manager on each node. On the one hand, it periodically reports the resource usage on the node and the running status of each resource container to the RM; on the other hand, It receives and processes various requests such as start/stop of resource containers from the application controller.
  • the application controller is used to apply for and manage the resource pool to the RM.
  • the resource pool runs the started resource container, and allocates the resource container to the client according to the resource allocation request of the client.
  • FIG. 1 shows an exemplary networked environment block diagram of a resource allocation system 100, as shown in FIG. 1, a system 100 including a client 102, an application controller 104, a resource manager 112, and a plurality of nodes 106, each of which The node 106 includes a node manager 108 and at least one resource container 110, and the resource pool 114 contains at least one resource container 110.
  • the client 102 runs an application to be applied for, and the type of the application may be MapReduce, Giraph, Storm, Spark, Tez/Impala, or MPI.
  • the client 102 can be any type of computing device, which is not limited by the embodiment of the present invention.
  • the resource container 110 is a resource abstraction in a node, which can encapsulate multiple types of resources on a node, such as memory, CPU, disk, network, and the like.
  • the resource container 110 may also encapsulate only a part of the resources on a certain node, for example, only the memory and the CPU are encapsulated, which is not limited by the embodiment of the present invention.
  • the resource container 110 can run any type of task.
  • a MapReduce application can request a resource container 110 to initiate a map or reduce task
  • a Giraph application can request a resource container 110 to run a Giraph task.
  • Users can also implement a custom application type that runs a specific task through the resource container 110 to implement a completely new application framework.
  • the application controller 104 is configured to manage the resource container 110 in the resource pool 114.
  • the resource container in the resource pool 114 is a resource container that has been started, and is applied and started by the application controller 104 to the resource manager 112 in advance. At the same time, the task of the job can be performed as soon as possible, thereby saving the startup time of the resource container 114.
  • the application controller 104 requests resources from the resource manager 112, the resources returned by the resource manager 112 for the application controller 104 are represented by the resource container 110.
  • the resource pool 114 shown in FIG. 1 includes multiple resource containers 110 among the plurality of nodes 106, but does not limit the plurality of nodes 106 or the resource containers 110 of the plurality of nodes 106 all belong to the resource pool. 114.
  • the resource pool 114 is composed of a resource container 110.
  • the application controller 104 is configured to receive a resource allocation request from an application on the client 102, and allocate a resource container required for the application of the application to the application of the application.
  • a resource container required for the application of the application to the application of the application.
  • the client 102 is in the application's job.
  • Each task is assigned a resource container, and the task can only use the resources described in the resource container.
  • the resource manager 112 is a global resource manager responsible for resource management and allocation of the entire system. When receiving a resource request from the application controller 104, the resource controller can be allocated to the application controller 104 according to the load of the entire system. .
  • the client 102, the application controller 104, the resource manager 112, and the node manager 108 of each node 106 can communicate through a network, where the network can be the Internet, an intranet, or a local area network (LAN). , Wireless Local Area Networks (WLANs), Storage Area Networks (SANs), etc., or a combination of the above.
  • LAN local area network
  • WLANs Wireless Local Area Networks
  • SANs Storage Area Networks
  • FIG. 1 is merely exemplary participants of the system 100 and their interrelationships. Therefore, the depicted system 100 is greatly simplified, and the embodiments of the present invention are merely described in general terms, and the implementation thereof is not limited in any way.
  • the client 102, the application controller 104, and the node 106 in FIG. 1 may be of any architecture, which is not limited by the embodiment of the present invention.
  • the application controller 104 and/or resource manager 112 shown in FIG. 1 can be implemented by the computing device 200 shown in FIG. 2.
  • computing device 200 includes a processor 202, a memory unit 204, an input/output interface 206, a communication interface 208, a bus 210, and a storage device 212.
  • processor 202, the memory unit 204, and the input/output connection implement a communication connection with each other via the bus 210.
  • the processor 202 is a control center of the computing device 200 for executing related programs to implement the technical solutions provided by the embodiments of the present invention.
  • the processor 202 includes one or more central processing units (CPUs), such as the central processing unit 1 and the central processing unit 2 shown in FIG.
  • the computing device 200 can also include multiple processors 202, each of which can be a single core processor (including one CPU) or a multi-core processor (including multiple CPUs).
  • a component for performing a specific function for example, the processor 202 or the memory unit 204, may be implemented by configuring a general-purpose component to perform a corresponding function, or may be specifically performed by a specific one.
  • the processor 202 can be a general-purpose central processing unit, a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits for executing related programs to implement the technology provided by the present application. Program.
  • Processor 202 can be coupled to one or more storage schemes via bus 210.
  • the storage scheme can include a memory unit 204 and a storage device 212.
  • the storage device 212 can be a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM).
  • Memory unit 204 can be a random access memory.
  • the memory unit 204 can be integrated with or integrated with the processor 202, or it can be one or more memory units independent of the processor 202.
  • Program code for execution by the processor 202 or a CPU internal to the processor 202 may be stored in the storage device 212 or the memory unit 204.
  • program code eg, an operating system, an application, a resource allocation module, or a communication module, etc.
  • stored internal to storage device 212 is copied to memory unit 204 for execution by processor 202.
  • the storage device 212 can be a physical hard disk or a partition thereof (including a small computer system interface storage or a global network block device volume), a network storage protocol (including a network file system NFS or the like network or a cluster file system), a file-based virtual storage device (virtual Disk mirroring), logical volume-based storage devices. It may include high speed random access memory (RAM), and may also include non-volatile memory, such as one or more disk memories, flash memories, or other non-volatile memory.
  • RAM random access memory
  • the storage device may further include the one or more processors 202 separate remote storage, such as a network disk accessed through a communication interface 208 with a communication network, which may be the Internet, an intranet, a local area network (LANs), a wide area network (WLANs), a storage area network (SANs), etc., or A combination of the above networks.
  • a communication network which may be the Internet, an intranet, a local area network (LANs), a wide area network (WLANs), a storage area network (SANs), etc., or A combination of the above networks.
  • Operating systems include controls and management of general system tasks (such as memory management, storage device control, power management, etc.) And various software components and/or drivers that facilitate communication between various hardware and software components.
  • the input/output interface 206 is for receiving input data and information, and outputting data such as operation results.
  • Communication interface 208 enables communication between computing device 200 and other devices or communication networks using transceivers such as, but not limited to, transceivers.
  • Bus 210 may include a path for communicating information between various components of computing device 200, such as processor 202, memory unit 204, input/output interface 206, communication interface 208, and storage device 212.
  • the bus 210 can use a wired connection or a wireless communication mode, which is not limited in this application.
  • computing device 200 shown in FIG. 2 only shows the processor 202, the memory unit 204, the input/output interface 206, the communication interface 208, the bus 210, and the storage device 212, in a specific implementation process, the field Those skilled in the art will appreciate that computing device 200 also includes other devices necessary to achieve proper operation.
  • the computing device 200 can be a general purpose computer or a special purpose computing device, including but not limited to a portable computer, a personal desktop computer, a network server, a tablet computer, a mobile phone, a personal digital assistant (PDA), or the like. Or a combination of two or more of the above, the present application does not limit the specific implementation of the computing device 200.
  • a portable computer including but not limited to a portable computer, a personal desktop computer, a network server, a tablet computer, a mobile phone, a personal digital assistant (PDA), or the like.
  • PDA personal digital assistant
  • computing device 200 of FIG. 2 is merely an example of one computing device 200, which may include more or fewer components than those shown in FIG. 2, or have different component configurations.
  • computing device 200 may also include hardware devices that implement other additional functions, depending on the particular needs.
  • computing device 200 may also only include the components necessary to implement embodiments of the present invention, and does not necessarily include all of the devices shown in FIG.
  • the various components shown in Figure 2 can be implemented in hardware, software, or a combination of hardware and software.
  • FIG. 2 and the foregoing description are applicable to various computing devices provided by the embodiments of the present invention, and are applicable to performing various resource allocation methods provided by the embodiments of the present invention.
  • the memory unit 204 of the computing device 200 includes a resource allocation module, and the processor 202 executes the resource allocation module program code to implement resource management and allocation.
  • the resource allocation module can be comprised of one or more operational instructions to cause computing device 200 to perform one or more method steps in accordance with the above description. The specific method steps are described in detail in the following sections of this application.
  • the distributed system includes a resource manager 112, a node manager 108, and at least two nodes 106.
  • the resource manager 112 is configured to manage the distributed system.
  • the node resource is used by the node manager 108 to start the resource container 110 based on the node resource.
  • the resource container 110 encapsulates the resource of the node 106 for performing the task of the application.
  • the resource allocation process includes:
  • the resource manager 112 launches the application controller 104.
  • the resource manager 112 is configured to manage resources of the at least two nodes 106, and launch the application controller 104 when the trigger timing is met, and configure an initial specification of the resource pool 114 managed by the application controller 104.
  • the resource manager 112 may start the application controller 104 when the system is initialized; or may be dynamically started by the user according to the requirements according to the requirements, for example, the resource manager 112 may receive the pre-launched application controller.
  • the request of 112, or the request to pre-configure the resource pool 114 initiates the application controller 104; or is initiated by the resource manager 112 according to its own resource status, for example, when each application is frequently received
  • the application controller 104 of the corresponding application type can be started. It should be understood that embodiments of the present invention do not limit the startup form of the application controller 104.
  • the resource manager 112 configures an initial specification of the resource pool 114 managed by the application controller 104, so that when the application controller 104 subsequently applies for resources to the resource manager 112, This initial specification determines the number and specifications of resource containers that are applied to the resource manager 112.
  • the initial specification of the resource pool 114 may be the number of resource containers initially included in the resource pool 114, and the specifications of each resource container, where the specification of the resource container is the resource type included in the resource container, and the number of each type of resource; Resource container specifications are set in advance, then resource pool
  • the initial specification of 114 may be the number of resource containers initially included in the resource pool 114; if the specifications of the resource container are configured in advance, and the resource container has multiple specifications, the resource types in each specification and the number of resources of each type are If set in advance, the initial specification of the resource pool 114 may be the resource container specification identifier initially included in the resource pool 114, and the number of resource containers for each specification.
  • the initial specification of the resource pool 114 may also be the type of various resources initially included by the resource pool 114, as well as the number of each resource.
  • the embodiment of the present invention does not limit the form of the initial specification of the resource pool 114, and the representation of the number of resources may be different in different scenarios.
  • the resource manager 112 may configure an initial specification of the resource pool 114 managed by the application controller 104 according to the expected resource requirement information of the preset application; the resource manager 112 may also be based on the collected node resources of the distributed system.
  • the usage information configures the initial specifications of the resource pool 114 managed by the application controller 104.
  • resource manager 112 may configure an initial specification of a larger resource pool 114 when there are sufficient unused node resources; resource manager 112 may configure smaller resources when there are fewer unused node resources The initial specification of pool 114.
  • the initial specification of the resource pool 114 can be carried by the user in the startup command of the application controller 104, and the resource manager 112 configures the application controller 104 when the application controller 104 is launched.
  • the application controller 104 may have a one-to-one correspondence with the application, because different applications have different requirements for resource containers, and the types of resources and the number of each resource in the resource containers corresponding to different applications may be different, and resource pools corresponding to different applications.
  • the initial specifications of the 114 may also be different, which is not limited by the embodiment of the present invention.
  • the application controller 104 sends a first resource request to the resource manager 112.
  • the application controller 104 may also register with the resource manager 112 before sending the first resource request to the resource manager 112 to facilitate subsequent resource manager 112 to manage the application controller 104.
  • the application controller 104 can also register with a registration server, and the address information of the registration server is distributed to the client 102 for the subsequent client 102 to query the address information of the application controller 104 through the registration server.
  • the registration server can store the mapping between the type of the application and the application controller 104.
  • the type of the application can be MapReduce, Giraph, Storm, Spark, Tez/Impala, or MPI.
  • the registration server stores the Spark application and the corresponding Spark.
  • Applied application controller 104 Corresponding relationship, when the application running on the client 102 is Spark, the client 102 can find the address information of the application controller 104 corresponding to the Spark application according to the correspondence. It is also possible to set up a separate registration server for a certain type of application. For example, a corresponding registration server can be set specifically for a Spark application.
  • the embodiment of the present invention does not limit the specific implementation manner of the registration server.
  • each type of application may correspond to multiple application controllers 104, and the multiple application controllers 104 may be used by different users, thereby facilitating the expansion of the distributed system scale.
  • the correspondence between the client 102 and the application controller 104 may be maintained on the registration server, or multiple registration servers may be set, and each registration server corresponds to a different user. It should be understood that the embodiment of the present invention does not limit the number of application servers corresponding to each type of application and the implementation manner of the registration server.
  • the application controller 104 carries the quantity of resources determined according to the initial specification of the resource pool in the first resource request sent to the resource manager 112.
  • the number of resources may be the number of resource containers requested by the application controller 104 to the resource manager 112 and the specifications of each resource container, wherein the specification of the resource container refers to the kind of resources included in the resource container and the number of each resource.
  • the resources included in each resource container may include one or more of a processor resource, a memory resource, a network, a disk, and the like. It should be understood that, according to the type of application, the resource type in the corresponding requested resource container and the number of each resource may be different, which is not limited by the embodiment of the present invention.
  • the request that is carried in the first resource request sent by the application controller 104 to the resource manager 112 is The number of resources can be the number of resource containers. If the specification of the resource container is configured in advance, and the resource container has multiple specifications, the resource type in each specification and the resource quantity of each type are set in advance, the resource quantity may be the application controller 104 to the resource manager.
  • the amount of resources may also be the kind of various resources that the application controller 104 needs, as well as the number of each resource.
  • the embodiment of the present invention does not limit the form of the quantity of resources requested in the first resource request, and the representation of the quantity of resources may be different in different scenarios.
  • the first resource request may further carry node information and/or rack information in which the resource container requested by the application controller 104 is located.
  • the application controller 104 may preferably host a shorter link with the application controller 104 as a node of the resource container running the request, thereby enabling more efficient control of the resource pool 114.
  • the resource manager 112 determines a node to allocate a resource container to the application controller 104 on the determined node.
  • the resource manager 112 is responsible for global resource management and allocation. After the resource manager 112 receives the first resource request of the resource pool 114 sent by the application controller 104 according to the initial specification of the resource pool 114, according to the first resource request, An initial resource container is allocated for the resource pool. Specifically, after receiving the first resource request, the resource manager 112 first determines an optional node, and according to the quantity of resources requested in the first resource request of the resource pool 114, the resource pool 114 is on the optional node. Assign the initial resource container.
  • the resource manager 112 preferentially obtains from the host node and/or the rack.
  • the application controller 112 allocates a resource container. If the host node and/or the rack load requested in the first resource request cannot be satisfied currently, for example, the requested host cannot satisfy the request due to a load problem, it may be in the rack where the node is located.
  • a resource container is allocated for the application controller 112. If the rack cannot satisfy the request due to load balancing, the resource controller can be allocated to the application controller 112 in the rack adjacent to the rack.
  • the resource manager 112 may allocate the resource container to the application controller 104 according to the load balancing of the node, or The application controller 104 is preferably allocated resources on a host or rack that has a short link to the application controller 104.
  • embodiments of the present invention do not define a policy for resource manager 112 to allocate resource containers to application controllers 104.
  • the resource manager 112 sends a first resource allocation message to the application controller 104.
  • the resource manager 112 sends a first resource allocation message of the resource pool 114 to the application controller 104, where the first resource allocation message includes an initial resource allocated by the resource manager 112 to the resource pool 114 managed by the application controller 104. Information about the node where the container is located.
  • the first resource allocation message may carry the resource manager 112 as an application controller.
  • the specification of a resource container is the kind of resource that the resource container contains, and the number of each resource.
  • the information of the node where each resource container is located may be carried in the first resource allocation message. If the resource manager 112 allocates multiple resource containers to the application controller 104 on the same node, the first resource allocation message may also carry the number of resource containers allocated in each node.
  • the first resource allocation message may carry the allocation.
  • the resource manager 112 may not immediately return the resource that meets the requirements for the application controller 104, but requires the application controller 104 to continuously and resource management.
  • the device 112 communicates, detects the allocated resources, and pulls them past.
  • the application controller 104 launches a resource container.
  • the application controller 104 sends a start request to the node manager requesting the node manager to initiate the initial resource container of the resource pool 114.
  • the application controller 104 After the application controller 104 obtains the resource container allocated to the resource manager 112, it sends a startup request to each node where the resource container is located, and more specifically, to the node of the node where each resource container is located.
  • the manager sends the start request to start the resource container for which the resource manager 112 is assigned.
  • the startup request also carries the specification of the resource container, and the specification of the resource container is the type of the resource container including the resource, and the quantity of each resource.
  • the resource type in each specification and the number of resources in each type are set in advance, the resource container that requests the startup is also carried in the startup request. Specification of the logo.
  • the resource manager 112 allocates multiple resource containers on the same node for the application controller 104, the multiple resource containers may be initiated by carrying a request in one startup request.
  • the node manager of the node where the resource container is located After receiving the startup request, the node manager of the node where the resource container is located first performs resource localization, that is, creates a working directory of the resource container, and downloads each required for running the resource container from the distributed file system (Hadoop Distributed File System, HDFS). Resources (jar packages, executable files, etc.), etc., and then start the resource container.
  • resource localization that is, creates a working directory of the resource container, and downloads each required for running the resource container from the distributed file system (Hadoop Distributed File System, HDFS). Resources (jar packages, executable files, etc.), etc., and then start the resource container.
  • HDFS Hadoop Distributed File System
  • the activated resource container constitutes the resource pool 114.
  • the resource containers in the resource pool 114 are already in the startup state, when the task is received, the execution can be performed quickly, and the startup time before the task is avoided, thereby speeding up the execution of the application task.
  • the application controller 104 receives a resource allocation request from the client 102.
  • the client 102 runs an application, and the type of the application may be MapReduce, Giraph, Storm, Spark, Tez/Impala, or MPI.
  • the client 102 When the application running on the client 102 requires resources for task processing, the client 102 sends a resource allocation request to the application controller.
  • the client 102 can query the address information of the application controller 104 corresponding to the application through a registration server, and send the resource allocation request to the application controller 104 indicated by the corresponding address information.
  • the registration server stores the corresponding information of the application controller 104 and the application, and different applications may have different application controllers 104.
  • resource containers of different applications have various specifications, and an application controller 104 and a resource pool 114 are configured for each type of application, so that rational utilization of resource containers can be realized.
  • the Spark application is used as an example.
  • the specific implementation process can be as follows: The user submits a spark application through the spark-submit.
  • the user program automatically creates a SparkContext.
  • the SparkContext is used by the Spark to provide the external interface for the Spark.
  • the schedulerBackend queries the registration server, searches for the address information of the application controller 104 corresponding to the Spark application, and sends the resource allocation request to the corresponding application controller 104.
  • the resource allocation request carries resource requirement information of the application requested by the client 102 to the application controller 104.
  • the resource requirement information of the application in the resource allocation request may be the number of resource containers requested, and the specifications of each resource container.
  • the specification of a resource container is the kind of resource that the resource container contains, and the number of each resource.
  • the resource requirement information of the application may be the requested resource container number.
  • the resource requirement information of the application may be the requested resource.
  • the resource requirement information of the application in the resource allocation request may also be the kind of various resources requested by the client 102 to the application controller 104, and the number of each resource.
  • the resource allocation request further includes user rights information of the client 102
  • the application controller 104 verifies the user rights information of the client 102 according to the preset user permission library, where the user rights The library contains user permission information for different users. If the user rights information of the client 102 is not included in the user rights library, the resource allocation request of the client 102 is rejected or not; if the user rights library contains the user rights information of the client 102, the following steps are performed.
  • the application controller 104 allocates a resource container to the client 102.
  • the application controller 104 After receiving the resource allocation request from the client 102, the application controller 104 selects an idle resource container from the resource pool 114 and allocates it to the client 102 according to the resource requirement information of the application in the resource allocation request.
  • the application controller 104 first determines an idle resource container in the resource pool 114, and according to the resource requirement information of the application carried in the resource allocation request, from the idle resource container.
  • a resource container is allocated for the client 102.
  • the idle resource container refers to a resource container that is not currently assigned to perform an application task.
  • the resource containers in the resource pool 114 controlled by the application controller 104 can be simultaneously allocated for use by the plurality of clients 102, one resource container can only perform one client 102 allocation at the same time. The task, so the application controller 104 needs to first determine the currently idle resource container before allocating the resource container to the client 102, thereby allocating the requested resource from the idle resource container to the client 102.
  • the application controller 104 can determine whether the current resource container is currently in an idle state by sending a query message to each resource container in the resource pool 114.
  • the application controller 104 maintains state information of each resource container in the resource pool 114, the state information indicating whether the resource container is in an idle state, and if the resource container is currently allocated and other tasks are running, the resource container The status of the resource container is idle. If the resource container is not currently allocated and no other tasks are running, the status of the resource container is idle.
  • the application controller 104 may determine the resource container in the resource pool 114 according to the state information of each resource container in the resource pool 114, and then request the resource requirement of the application according to the resource allocation request. Information, the resource container is allocated to the client 102 from the currently idle resource container.
  • the application controller 104 selects an idle resource container from the resource pool 114 and allocates it to the client 102.
  • the application controller 104 selects an idle resource container in the resource pool 114 according to the state information of each resource container in the resource pool 114.
  • the resource containers that are free in the selected resource pool 114 are allocated to the client 102.
  • the resource container allocated by the application controller 104 to the client 102 is referred to as a first resource container group, and the first resource container group includes at least one resource container.
  • the application controller 104 If the application controller 104 maintains state information for each resource container in the resource pool 114, the application controller 104 allocates the resource containers in the first resource container group to the client 102, and also each of the first resource container groups. The status of the resource container is set to not idle.
  • the application controller 104 further determines the number of idle resource containers remaining in the resource pool 114, and sends the second resource to the resource manager 112 if the number of remaining idle resource containers is less than a preset first threshold. And requesting, and receiving the second resource allocation message of the resource pool 114 sent by the resource manager 112, sending a startup request to the node manager according to the second resource allocation message of the resource pool 114, requesting the node manager to start as the resource pool 114 The newly added resource container.
  • the second resource request may carry the requested resource quantity, and the requested resource quantity may be a difference between the first threshold and the current idle resource container number, or may be positively correlated with the difference.
  • the form of the second resource request is similar to the form of the first resource request, and details are not described herein again.
  • the second resource allocation message is similar to the first resource allocation message, and details are not described herein again.
  • the application server 104 may determine the number of idle resource containers remaining in the resource pool 114 after allocating the resource containers in the first resource container group to the client 102 or according to a preset period. It should be appreciated that the time and manner in which the application server 104 determines the number of free resource containers remaining in the resource pool 114 is not limited by the inventive embodiment.
  • the application controller 104 sends an indication message to each resource container in the first resource container group, where the indication message carries the identifier of the client 102.
  • the resource allocation request sent by the client 102 to the application controller 104 further carries the identifier of the client 102, and the indication message sent by the application controller to each resource container in the first resource container group is used to indicate that the first resource is to be used.
  • Each resource container in the container group is assigned to the client 102.
  • the identifier of the client 102 may be address information of the client 102 and a port number that communicates with the resource container in the first resource container group.
  • Each resource container in the first resource container group sends a registration message to the client 102.
  • the registration message carries the identification information of the resource container, and the identification information of the resource container is the identification information for the client to uniquely identify the resource container.
  • the identifier information is address information of a node where the resource container is located and information of the resource container at the node.
  • the registration message can also carry the specification of the resource container, that is, the kind of resources in the resource container and the number of each resource.
  • the registration message may carry the specification identifier of the resource container.
  • the client 102 assigns the application's job to the resource container that the application controller 114 allocates for the client 102.
  • the client 102 After receiving the registration message of the resource container in the first resource container group, the client 102 divides the application job into at least one task, and sends each task to one resource container in the first resource container group.
  • the resource container allocated by the application controller 114 to the client 102 sends the execution result of the task to the client 102.
  • the task After the resource container in the first resource container group is assigned to its task execution, the task will be The execution result is sent to the client 102. It should be understood that the execution result here may be an actual task execution result. For a task that does not need to return a result, the execution result here may be a task completion indication message.
  • the client 102 ends the connection with the resource container in the first resource container group.
  • the client 102 After receiving the task execution result returned by the resource container in the first resource container group, the client 102 sends an end connection message to the resource container of the first resource container group, ending the connection with the resource container in the first resource container group, and releasing Resource container resource.
  • the resource container in the first resource container group sends a status update message to the application controller 104.
  • the resource container in the first resource container group After the client 102 releases the resource container resource in the first resource container group, the resource container in the first resource container group sends a status update message to the application controller 104 indicating that a new task can be received.
  • the application controller 104 If the application controller 104 maintains state information of each resource container in the resource pool 114, the application controller 104 sets the state of the corresponding resource container to idle after receiving the status update message of the resource container in the first resource container group. When there is a new resource allocation request, it can be allocated to the client 102 as an alternative resource container.
  • the application controller 104 further determines the number of idle resource containers remaining in the resource pool 114, and if the number of remaining free resource containers is greater than a preset second threshold, at least one of the idle resource containers The resource container sends a resource release message, and the resource release message is used to release resources occupied by the at least one resource container.
  • the number of resource containers released may be the difference between the number of currently idle resource containers and the second threshold, or may be positively correlated with the difference.
  • the application server 104 further maintains the idle time of each resource container in the resource pool 114.
  • the application server 104 provides the idle time to be greater than the preset time.
  • the third threshold of the resource container sends a resource release message.
  • the application server 104 when the number of remaining idle resource containers is greater than a preset second threshold, the application server 104 preferentially sends a resource release message to the resource container whose idle time is greater than a preset third threshold. Or when the number of remaining idle resource containers is greater than a preset second threshold, and the idle time of the resource container in the resource pool 114 is greater than a preset third threshold, the application server 104 is idle longer than the preset third.
  • the threshold resource container sends a resource release message.
  • the application server 104 may receive the resources in the first resource container group.
  • the status update message of the source container after re-setting the resource container in the first resource container group to the idle state, or determining the number of free resource containers remaining in the resource pool 114 according to a preset period. It should be appreciated that the time and manner in which the application server 104 determines the number of free resource containers remaining in the resource pool 114 is not limited by the inventive embodiment.
  • the application controller 104 and the resource pool 114 are separately configured for each type of application, so that the rational use of resources can be realized.
  • the embodiment of the present invention does not limit the application, and the application controller 104 and the resource pool 114 can be shared by multiple types of applications.
  • the resource controller is applied to the resource manager in advance by the application controller, and the requested resource container is started in advance.
  • the resource container resource can be allocated in time.
  • FIG. 4 is a signaling diagram of a resource allocation process according to an embodiment of the present invention. As shown in FIG. 4, the resource allocation process includes:
  • Steps 402-410 refer to steps 302-310 of the embodiment of FIG. 3, and details are not described herein again.
  • the client 102 sends an application job to the application controller 104.
  • the client runs an application of the user, and the type of the application may be MapReduce, Giraph, Storm, Spark, Tez/Impala, or MPI.
  • the application job is sent to the application controller through the client 102.
  • the client 102 can query the address information of the application controller 104 corresponding to the application through a registration server, and send the application job to the application controller 104 indicated by the corresponding address information.
  • the registration server stores the corresponding information of the application controller 104 and the application, and different applications may have different application controllers 104.
  • an application controller 104 and a resource pool are configured for each type of application, so that the resource container can be rationalized and utilized.
  • the application controller 104 allocates a resource container for the application job of the client 102.
  • the application controller 104 after receiving the application job sent by the client 102, the application controller 104 first determines an idle resource container in the resource pool 114, and divides the application job into at least one task according to the scale of the application job. And allocate a resource container for the application job of the client 102 from the idle resource container according to the number of tasks.
  • the application controller 104 also allocates different specifications of resources for different tasks according to the types of tasks. container. For example, if the application type is MapReduce, the task may be a Map type or a Reduce type. The Map task and the Reduce task may have different resource container resources. The application controller 104 may allocate different types of resource containers for different types of resource containers.
  • the application controller 104 is the client. Before allocating the resource container 102, it is necessary to first determine the currently idle resource container, thereby allocating resources for the application job of the client 102 from the idle resource container.
  • the application controller 104 can determine whether the current resource container is currently in an idle state by sending a query message to each resource container in the resource pool 114.
  • the application controller 104 maintains state information of each resource container in the resource pool 114, the state information is used to indicate whether the resource container is in an idle state, and if the resource container is currently allocated and other tasks are running, The state of the resource container is not idle. If the resource container is not currently allocated and no other tasks are running, the state of the resource container is idle.
  • the application controller 104 can determine the resource containers that are free in the resource pool 114 based on the state information of each resource container in the resource pool 114.
  • the resource container allocated by the application controller 104 to the application task of the client 102 is referred to as a second resource container group, and the second resource container group includes at least one resource container.
  • the application controller 104 If the application controller 104 maintains state information of each resource container in the resource pool 114, the application controller 104 allocates the resource container in the second resource container group to the task of the application job of the client 102, and also the second resource. The status of each resource container in the container group is set to not idle.
  • the application controller 104 further determines the number of idle resource containers remaining in the resource pool 114, and sends the second resource to the resource manager 112 if the number of remaining idle resource containers is less than a preset first threshold. Requesting, and receiving a second resource allocation message of the resource pool 114 sent by the resource manager 112, according to the second resource allocation message of the resource pool 114, the application controller 104 sends a startup request to the node manager, requesting the node manager to start A new resource container that is allocated for resource pool 114.
  • the number of resources of the second resource request request may be a difference between the first threshold and the current number of idle resource containers, or may be positively correlated with the difference.
  • the application server 104 may determine the number of idle resource containers remaining in the resource pool 114 after the resource container in the second resource container group is allocated to the application job of the client 102, or according to a preset period. . It should be appreciated that the time and manner in which the application server 104 determines the number of free resource containers remaining in the resource pool 114 is not limited by the inventive embodiment.
  • the application controller 104 delivers a task to each resource container in the second resource container group.
  • the application controller 104 divides the job of the client 102 into at least one task, and selects a resource container for each task, and then delivers the corresponding task to the resource container in the second resource container group.
  • the application controller 104 receives the task execution result returned by each resource container in the second resource container group.
  • the execution result of the task is returned to the application controller 104.
  • the execution result here may be an actual task execution result.
  • the execution result here may be a task completion indication message.
  • the application controller 104 If the application controller 104 maintains the state information of each resource container in the resource pool 114, the application controller 104 receives the task execution result returned by the resource container in the second resource container group, and sets the state of the corresponding resource container to Idle, when there is a new application job, it can be assigned to the new application job as an alternate resource container.
  • the application controller 104 further determines the number of idle resource containers remaining in the resource pool 114, and if the number of remaining free resource containers is greater than a preset second threshold, at least one of the idle resource containers The resource container sends a resource release message, and the resource release message is used to release resources occupied by the at least one resource container.
  • the number of resource containers released can be current The difference between the number of idle resource containers and the second threshold, or a positive correlation with the difference.
  • the application server 104 further maintains the idle time of each resource container in the resource pool 114.
  • the application server 104 provides the idle time to be greater than the preset time.
  • the third threshold of the resource container sends a resource release message.
  • the application server 104 when the number of remaining idle resource containers is greater than a preset second threshold, the application server 104 preferentially sends a resource release message to the resource container whose idle time is greater than a preset third threshold. Or when the number of remaining idle resource containers is greater than a preset second threshold, and the idle time of the resource container in the resource pool 114 is greater than a preset third threshold, the application server 104 is idle longer than the preset third.
  • the threshold resource container sends a resource release message.
  • the application server 104 may receive the execution result returned by each resource container in the second resource container group, reset the resource container of the second resource container group to the idle state, or follow the preset.
  • the period determines the number of free resource containers remaining in resource pool 114. It should be appreciated that the time and manner in which the application server 104 determines the number of free resource containers remaining in the resource pool 114 is not limited by the inventive embodiment.
  • the application controller 104 returns the execution result of the application job of the client 102 to the client 102.
  • the application controller 104 may merge the execution results returned by each resource container in the second resource container group, and then send the merged execution result to the client 102.
  • the application controller 104 and the resource pool 114 are separately configured for each type of application, so that the rational use of resources can be realized.
  • the embodiment of the present invention does not limit the application, and the application controller 104 and the resource pool 114 can be shared by multiple types of applications.
  • the resource controller is applied to the resource manager in advance by the application controller, and the requested resource container is started in advance.
  • the resource container resource can be allocated in time.
  • the waiting time of the resource container startup is avoided, and the resource consumption of the resource container in the resource pool is avoided, and the resource consumption of the resource container being opened and closed multiple times and the resource consumption of multiple application managers being turned on and off are avoided.
  • the controller directly manages the resource containers in the resource pool, enabling more flexible management.
  • FIG. 5 is a schematic diagram of a logical structure of a resource container allocation apparatus 500 in a distributed system according to an embodiment of the present invention.
  • the distributed system includes a device 500 and a node manager, where the apparatus 500 is used.
  • the node resource of the distributed system is managed, the node manager is configured to start the resource container based on the node resource, and the resource container is used to perform the task of the application.
  • the device 500 includes a starting unit 502, a receiving unit 504, an allocating unit 506, and processing.
  • Unit 508 wherein
  • the startup unit 502 is configured to start the application controller when the trigger timing is met, and configure an initial specification of the resource pool managed by the application controller.
  • the activation unit 502 can start the application controller upon receiving a request to start the application controller in advance or receiving a request for pre-configuring the resource pool.
  • the startup unit 502 can be started when the system is initialized, or can be dynamically started by the user according to the requirements according to the requirements, or can be started by the resource manager according to the resource status of the user, and the embodiment of the present invention does not Define the startup form of the application controller.
  • the initiating unit 502 may configure an initial specification of the resource pool managed by the application controller according to the expected resource requirement information of the preset application; the initiating unit 502 may further use the node resource of the distributed system collected by the device 500. Information, configure the initial specifications of the resource pool managed by the application controller. For example, when there are sufficient unused node resources, the startup unit 502 can configure an initial specification of a larger resource pool; when there are fewer unused node resources, the startup unit 502 can configure an initial of a smaller resource pool. specification.
  • the initial specification of the resource pool may be carried by the user in the startup command of the application controller, and the startup controller 502 configures the application controller when the application controller is started.
  • the receiving unit 502 can be implemented by the processor 202, the memory unit 204, and the communication interface 208 shown in FIG. More specifically, the communication module and resource allocation module in memory unit 204 can be executed by processor 202 to cause communication interface 208 to launch the application server by instruction.
  • the receiving unit 504 is configured to receive a first resource request of the resource pool that is sent by the application controller according to an initial specification of the resource pool.
  • the receiving unit 504 can be implemented by the processor 202, the memory unit 204, and the communication interface 208 shown in FIG. More specifically, the communication module in memory unit 204 can be executed by processor 202 to cause communication interface 208 to receive a resource request from a resource pool that the application controller sent according to the initial specification of the resource pool.
  • the allocating unit 506 is configured to allocate an initial resource container to the resource pool according to the first resource request.
  • the allocating unit 506 can be implemented by the processor 202 and the memory unit shown in FIG. 2. More specifically, resource allocation in memory unit 204 can be performed by processor 202. A module that allocates node resources for a resource pool to an application controller.
  • the sending unit 508 is configured to send, to the application controller, a first resource allocation message of the resource pool, where the first resource allocation message includes information about a node where the initial resource container allocated by the allocation unit 506 is allocated to the resource pool.
  • the sending unit 508 can be implemented by the processor 202, the memory unit 204, and the communication interface 208 shown in FIG. 2. More specifically, the communication module in memory unit 204 can be executed by processor 202 to cause communication interface 208 to send a first resource allocation message for the resource pool to the application controller.
  • the embodiment of the present invention is an apparatus embodiment of the resource manager in the embodiment of FIG. 3 and FIG. 4, and the feature description of the embodiment of FIG. 3 and FIG. 4 is applicable to the embodiment of the present invention, and details are not described herein again.
  • FIG. 6 is a schematic diagram showing the logical structure of a resource container allocation apparatus 600 in a distributed system according to an embodiment of the present invention.
  • the distributed system includes a resource manager and a node manager, and the resource manager is used to manage node resources and nodes of the distributed system.
  • the manager is configured to start a resource container based on the node resource, where the resource container is used to execute the task of the application, and the resource manager starts the device 600 when the triggering time is met, and configures the initial specification of the resource pool managed by the device 600, and according to the initial specification of the resource pool,
  • the resource pool allocates an initial resource container, and the initial resource container in the resource pool is started;
  • Apparatus 600 includes:
  • the receiving unit 602 is configured to receive a resource allocation request from the client, where the resource allocation request is used to request a resource container for the application running on the client, where the resource allocation request carries resource requirement information of the application.
  • the receiving unit 602 can be implemented by the processor 202, the memory unit 204, and the communication interface 208 shown in FIG. More specifically, the communication module in memory unit 204 can be executed by processor 202 to cause communication interface 208 to receive a resource allocation request from the client.
  • the allocating unit 604 is configured to select, from the resource pool, an idle resource container to be allocated to the client according to the resource requirement information of the application.
  • the allocating unit 604 can be implemented by the processor 202 and the memory unit shown in FIG. 2. More specifically, the resource allocation module in the memory unit 204 can be executed by the processor 202 to allocate the node resources of the resource pool to the application controller.
  • the apparatus 600 further includes a sending unit 606.
  • the sending unit 606 is configured to send the first resource request to the resource manager according to the initial specification of the resource pool, where The resource request carries the initial specification according to the resource pool.
  • the receiving unit 602 is further configured to receive a first resource allocation message of the resource pool sent by the resource manager, where the first resource allocation message includes an initial resource container allocated by the resource manager for the resource pool.
  • the information of the node where the node is located; the sending unit 606 is further configured to send a startup request to the node manager, requesting the node manager to start the initial resource container of the resource pool.
  • the resource allocation request includes the identifier of the client
  • the allocating unit 604 selects the idle resource container from the resource pool and allocates the data to the client
  • the sending unit 606 is configured to send an indication message to each resource container in the resource container allocated to the client.
  • the indication message carries the identifier of the client, and the indication message is used to indicate that each resource container is allocated to the client.
  • the apparatus 600 further includes a determining unit 608, configured to determine the number of free resource containers remaining in the resource pool, and if the number of remaining free resource containers is less than a preset first threshold, send The unit 606 is configured to send a second resource request to the resource manager, and the receiving unit 602 is further configured to receive the second resource allocation message of the resource pool sent by the resource manager, where the sending unit 606 is further configured to use the second resource according to the resource pool. Assign a message, send a start request to the node manager, and request the node manager to start the new resource container allocated for the resource pool.
  • the sending unit 606 is configured to send a resource release message to the at least one resource container in the idle resource container, where the resource release message is used to release at least one resource container occupation resource of.
  • the allocating unit 604 is further configured to maintain state information of each resource container in the resource pool, where the state information indicates whether the corresponding resource container is idle; and the allocating unit 604 is configured to select an idle resource container from the resource pool.
  • the allocation to the client includes: an allocating unit 604, configured to select an idle resource container in the resource pool according to status information of each resource container in the resource pool, and allocate the selected resource container in the selected resource pool to The client.
  • the allocation unit 604 selects an idle resource container from the resource pool and allocates it to the client, it also sets the state of each resource container in the resource container allocated to the client to be not idle.
  • the receiving unit 602 is further configured to receive a status update message from each of the resource containers allocated to the client, the status update.
  • the message is used to indicate that the task of the client allocation is completed; the allocating unit 604 is further configured to set the state of each resource container in the resource container allocated to the client to be idle according to the status update information.
  • the resource allocation request further includes user rights information; the receiving unit 602 is further configured to verify user rights information according to the preset user rights library, where the user rights library includes user rights information.
  • the embodiment of the present invention is an apparatus embodiment of the application controller in the embodiment of FIG. 3, and the feature description of the embodiment of FIG. 3 is applicable to the embodiment of the present invention, and details are not described herein again.
  • the disclosed systems, devices, and methods may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the modules is only a logical function division, and may be implemented in another manner, for example, multiple modules or components may be combined or may be Integrate into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or module, and may be electrical, mechanical or otherwise.
  • the modules described as separate components may or may not be physically separated.
  • the components displayed as modules may or may not be physical modules, that is, may be located in one place, or may be distributed to multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.
  • the above integrated modules can be implemented in the form of hardware or in the form of hardware plus software function modules.
  • the above-described integrated modules implemented in the form of software function modules can be stored in a computer readable storage medium.
  • the software functional modules described above are stored in a storage medium and include instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform some of the steps of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes: a mobile hard disk, a read-only memory (English: Read-Only Memory, ROM for short), a random access memory (English: Random Access Memory, RAM for short), a magnetic disk or an optical disk, and the like. The medium of the code.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

Conformément à des modes de réalisation, la présente invention concerne un procédé, un dispositif et un système d'allocation de ressource, utilisés pour allouer une ressource à une application. Le procédé comprend les opérations suivantes : un dispositif de commande d'application reçoit une requête d'allocation de ressource à partir d'une extrémité de client, la requête d'allocation de ressource transportant des informations de demande de ressource d'une application s'exécutant sur l'extrémité de client, le dispositif de commande d'application étant configuré pour gérer un groupe de ressources, le groupe de ressources comprenant au moins un contenant lancé, et le contenant dans le groupe de ressources étant négocié à partir d'un gestionnaire de ressources par le dispositif de commande d'application et lancé à l'avance ; et le dispositif de commande d'application sélectionne, selon les informations de demande de ressource de l'application, un contenant de ressource au repos à partir du groupe de ressources, et alloue le contenant de ressource au repos à l'extrémité de client. Selon le procédé, un dispositif de commande d'application peut allouer rapidement une ressource d'un contenant de ressource lors de la réception d'une requête d'allocation de ressource d'une application, de telle sorte qu'un temps d'attente pour lancer un contenant de ressource est éliminé. En outre, par réutilisation d'un contenant de ressource dans un groupe de ressources, un gaspillage de ressources résultant de la création et de la fin de manière répétée d'un contenant de ressource est évité.
PCT/CN2015/099258 2015-12-28 2015-12-28 Procédé, dispositif et système d'allocation de ressource WO2017113074A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201580084802.8A CN108293041B (zh) 2015-12-28 2015-12-28 分布式系统、资源容器的分配方法、资源管理器及应用控制器
PCT/CN2015/099258 WO2017113074A1 (fr) 2015-12-28 2015-12-28 Procédé, dispositif et système d'allocation de ressource

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2015/099258 WO2017113074A1 (fr) 2015-12-28 2015-12-28 Procédé, dispositif et système d'allocation de ressource

Publications (1)

Publication Number Publication Date
WO2017113074A1 true WO2017113074A1 (fr) 2017-07-06

Family

ID=59224029

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/099258 WO2017113074A1 (fr) 2015-12-28 2015-12-28 Procédé, dispositif et système d'allocation de ressource

Country Status (2)

Country Link
CN (1) CN108293041B (fr)
WO (1) WO2017113074A1 (fr)

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110109649A (zh) * 2018-02-01 2019-08-09 中国电信股份有限公司 针对Web服务的容器控制方法、装置和容器系统
CN111274022A (zh) * 2018-12-05 2020-06-12 北京华胜天成科技股份有限公司 服务器资源分配方法和系统
CN111427675A (zh) * 2020-03-20 2020-07-17 腾讯科技（深圳）有限公司 一种数据处理方法、装置以及计算机可读存储介质
CN111445101A (zh) * 2020-05-15 2020-07-24 广联达科技股份有限公司 一种云计算资源调度的方法、系统及介质
CN111949407A (zh) * 2020-08-13 2020-11-17 北京字节跳动网络技术有限公司 一种资源分配方法及装置
CN112015542A (zh) * 2019-05-29 2020-12-01 潘仲光 资源收集方法、设备及存储介质
CN112052084A (zh) * 2019-06-05 2020-12-08 杭州海康威视数字技术股份有限公司 一种资源分配方法和计算机设备
CN112306640A (zh) * 2020-11-12 2021-02-02 广州方硅信息技术有限公司 容器分配方法及其装置、设备、介质
CN112653571A (zh) * 2020-08-20 2021-04-13 国家电网公司华中分部 一种基于虚拟机与容器的混合调度方法
CN113064714A (zh) * 2021-04-28 2021-07-02 中国工商银行股份有限公司 利用研发环境中的空闲资源处理任务的方法及装置
CN113419839A (zh) * 2021-07-20 2021-09-21 北京字节跳动网络技术有限公司 多类型作业的资源调度方法、装置、电子设备及存储介质
CN113507441A (zh) * 2021-06-08 2021-10-15 中国联合网络通信集团有限公司 安全资源扩充方法及安全防护管理平台、数据节点
CN115454391A (zh) * 2022-11-11 2022-12-09 零氪科技（北京）有限公司 客户端、客户端构建方法、装置、电子设备及存储介质
CN117873738A (zh) * 2024-03-12 2024-04-12 苏州元脑智能科技有限公司 资源分配方法、装置、电子设备以及存储介质
CN114840125B (zh) * 2022-03-30 2024-04-26 曙光信息产业(北京)有限公司 设备资源配置、管理方法、装置、设备、介质和程序产品

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111966442A (zh) * 2019-05-20 2020-11-20 北京白山耘科技有限公司 一种容器端口管理的方法，装置，介质及设备
CN112445602A (zh) * 2019-08-27 2021-03-05 阿里巴巴集团控股有限公司 资源调度方法、装置、系统及电子设备
CN112835996A (zh) * 2019-11-22 2021-05-25 北京初速度科技有限公司 一种地图生产系统及其方法
CN111429091A (zh) * 2020-03-19 2020-07-17 北京字节跳动网络技术有限公司 一种资源分配方法、装置、电子设备及存储介质
CN111694649B (zh) * 2020-06-12 2023-07-18 北京火山引擎科技有限公司 资源调度方法、装置、计算机设备和存储介质
CN112083932B (zh) * 2020-08-18 2022-02-25 上海交通大学 虚拟化网络设备上的函数预热系统及方法
CN113326988A (zh) * 2021-06-10 2021-08-31 国网浙江省电力有限公司兰溪市供电公司 一种电力专线资源分配优化系统及方法
CN113391906B (zh) * 2021-06-25 2024-03-01 北京字节跳动网络技术有限公司 作业更新方法、装置、计算机设备及资源管理系统
CN113395291B (zh) * 2021-06-30 2023-03-17 北京爱奇艺科技有限公司 流量控制方法、装置、电子设备及存储介质
CN115827255B (zh) * 2023-02-16 2023-04-21 中国电力科学研究院有限公司 一种用于集中器的应用资源自适应分配管理方法及系统
CN116389172B (zh) * 2023-06-05 2023-09-19 国网四川省电力公司信息通信公司 基于多租户的容器云平台资源安全管理方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104731595A (zh) * 2015-03-26 2015-06-24 江苏物联网研究发展中心 面向大数据分析的混合计算系统
CN104765870A (zh) * 2015-04-26 2015-07-08 成都创行信息科技有限公司 关于网络数据的延迟调度方法
CN104780146A (zh) * 2014-01-13 2015-07-15 华为技术有限公司 资源管理方法及装置
CN105045656A (zh) * 2015-06-30 2015-11-11 深圳清华大学研究院 基于虚拟容器的大数据存储与管理方法

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9602423B2 (en) * 2013-06-28 2017-03-21 Pepperdata, Inc. Systems, methods, and devices for dynamic resource monitoring and allocation in a cluster system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104780146A (zh) * 2014-01-13 2015-07-15 华为技术有限公司 资源管理方法及装置
CN104731595A (zh) * 2015-03-26 2015-06-24 江苏物联网研究发展中心 面向大数据分析的混合计算系统
CN104765870A (zh) * 2015-04-26 2015-07-08 成都创行信息科技有限公司 关于网络数据的延迟调度方法
CN105045656A (zh) * 2015-06-30 2015-11-11 深圳清华大学研究院 基于虚拟容器的大数据存储与管理方法

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110109649A (zh) * 2018-02-01 2019-08-09 中国电信股份有限公司 针对Web服务的容器控制方法、装置和容器系统
CN110109649B (zh) * 2018-02-01 2023-08-08 中国电信股份有限公司 针对Web服务的容器控制方法、装置和容器系统
CN111274022A (zh) * 2018-12-05 2020-06-12 北京华胜天成科技股份有限公司 服务器资源分配方法和系统
CN111274022B (zh) * 2018-12-05 2024-05-14 北京华胜天成科技股份有限公司 服务器资源分配方法和系统
CN112015542A (zh) * 2019-05-29 2020-12-01 潘仲光 资源收集方法、设备及存储介质
CN112052084A (zh) * 2019-06-05 2020-12-08 杭州海康威视数字技术股份有限公司 一种资源分配方法和计算机设备
CN112052084B (zh) * 2019-06-05 2024-06-28 杭州海康威视数字技术股份有限公司 一种资源分配方法和计算机设备
CN111427675A (zh) * 2020-03-20 2020-07-17 腾讯科技（深圳）有限公司 一种数据处理方法、装置以及计算机可读存储介质
CN111427675B (zh) * 2020-03-20 2023-03-14 腾讯科技（深圳）有限公司 一种数据处理方法、装置以及计算机可读存储介质
CN111445101A (zh) * 2020-05-15 2020-07-24 广联达科技股份有限公司 一种云计算资源调度的方法、系统及介质
CN111949407A (zh) * 2020-08-13 2020-11-17 北京字节跳动网络技术有限公司 一种资源分配方法及装置
CN111949407B (zh) * 2020-08-13 2024-04-12 抖音视界有限公司 一种资源分配方法及装置
CN112653571A (zh) * 2020-08-20 2021-04-13 国家电网公司华中分部 一种基于虚拟机与容器的混合调度方法
CN112653571B (zh) * 2020-08-20 2024-03-22 国家电网公司华中分部 一种基于虚拟机与容器的混合调度方法
CN112306640A (zh) * 2020-11-12 2021-02-02 广州方硅信息技术有限公司 容器分配方法及其装置、设备、介质
CN113064714A (zh) * 2021-04-28 2021-07-02 中国工商银行股份有限公司 利用研发环境中的空闲资源处理任务的方法及装置
CN113507441A (zh) * 2021-06-08 2021-10-15 中国联合网络通信集团有限公司 安全资源扩充方法及安全防护管理平台、数据节点
CN113419839A (zh) * 2021-07-20 2021-09-21 北京字节跳动网络技术有限公司 多类型作业的资源调度方法、装置、电子设备及存储介质
CN114840125B (zh) * 2022-03-30 2024-04-26 曙光信息产业(北京)有限公司 设备资源配置、管理方法、装置、设备、介质和程序产品
CN115454391B (zh) * 2022-11-11 2023-06-16 零氪科技（北京）有限公司 客户端、客户端构建方法、装置、电子设备及存储介质
CN115454391A (zh) * 2022-11-11 2022-12-09 零氪科技（北京）有限公司 客户端、客户端构建方法、装置、电子设备及存储介质
CN117873738A (zh) * 2024-03-12 2024-04-12 苏州元脑智能科技有限公司 资源分配方法、装置、电子设备以及存储介质
CN117873738B (zh) * 2024-03-12 2024-05-24 苏州元脑智能科技有限公司 资源分配方法、装置、电子设备以及存储介质

Also Published As

Publication number Publication date
CN108293041B (zh) 2020-10-09
CN108293041A (zh) 2018-07-17

Similar Documents

Publication Publication Date Title
WO2017113074A1 (fr) Procédé, dispositif et système d'allocation de ressource
US10275851B1 (en) Checkpointing for GPU-as-a-service in cloud computing environment
US9934073B2 (en) Extension of resource constraints for service-defined containers
US11429408B2 (en) System and method for network function virtualization resource management
CN108737468B (zh) 云平台服务集群、构建方法及装置
CN107690622B9 (zh) 实现硬件加速处理的方法、设备和系统
CN109313564B (zh) 用于支持多个不同租户的高度可用虚拟桌面的服务器计算机管理系统
US8756599B2 (en) Task prioritization management in a virtualized environment
US20190334765A1 (en) Apparatuses and methods for site configuration management
JP5837206B2 (ja) 分離された仮想空間を提供するための方法および装置
WO2015176636A1 (fr) Système de gestion de service de base de données distribué
JP2021504795A (ja) クラウドサービス移行のための方法、装置、及び電子デバイス
US11169846B2 (en) System and method for managing tasks and task workload items between address spaces and logical partitions
CN108089913B (zh) 一种超融合系统的虚拟机部署方法
WO2016035003A1 (fr) Sensibilité à l'accès mémoire non uniforme (numa) transparent
US20210200573A1 (en) Virtual Machine Live Migration Method And Communications Device
US10728169B1 (en) Instance upgrade migration
US9092272B2 (en) Preparing parallel tasks to use a synchronization register
US20190356541A1 (en) Apparatuses and methods for zero touch computing node initialization
US20210250234A1 (en) Methods and apparatus to migrate physical server hosts between virtual standard switches and virtual distributed switches in a network
WO2015083255A1 (fr) Systeme d'ordinateurs et procede de commande de machines virtuelles
US10289306B1 (en) Data storage system with core-affined thread processing of data movement requests
JP5655612B2 (ja) 情報処理システム、情報処理方法、及び制御プログラム
US10747567B2 (en) Cluster check services for computing clusters
WO2015090195A1 (fr) Procédé et dispositif pour créer un exemple de système d'exploitation

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15911705

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15911705

Country of ref document: EP

Kind code of ref document: A1