CN110532087B

CN110532087B - Resource allocation method and resource manager

Info

Publication number: CN110532087B
Application number: CN201810506713.3A
Authority: CN
Inventors: 张杨; 冯亦挥; 欧阳晋
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2018-05-24
Filing date: 2018-05-24
Publication date: 2023-04-07
Anticipated expiration: 2038-05-24
Also published as: CN110532087A

Abstract

The application provides a resource allocation method and a resource manager, wherein the method comprises the following steps: acquiring a resource scheduling request; responding to the scheduling request, and determining the resource quota of each user group according to a preset strategy, wherein the preset strategy is as follows: distributing the total cluster resources among all user groups in equal quantity, and distributing the resources exceeding the demand of the user groups in equal quantity in the user groups with insufficient demand after equal quantity distribution; and distributing resources for each user group according to the determined resource quota of each user group. By utilizing the technical scheme provided by the embodiment of the application, the technical problem that resources cannot be distributed fairly among different user groups in the existing mode of sequentially distributing the queuing requests in the queue by traversing the queuing queue is solved, and the technical effect of simply and efficiently realizing fair distribution of the resources among different user groups is achieved.

Description

Resource allocation method and resource manager

Technical Field

The present application belongs to the field of computer technologies, and in particular, to a resource allocation method and a resource manager.

Background

In a distributed system, there are generally three roles: resource scheduler, job (including job manager and subordinate job node), machine node. As shown in FIG. 1, when a user needs to submit a job, the job manager will apply a certain amount of machine resources (e.g., CPU, memory, etc.) to the resource scheduler for the job node to use. After receiving the job resource application, the resource scheduler calculates an available resource list (for example, 1-core CPU and 1GB memory can be used on machine a, and 2-core CPU and 2GB memory can be used on machine B) that can be allocated to the job manager according to the remaining available resources in the distributed system; the resource scheduler then sends the resource list to the job manager and the corresponding machine node. After receiving the resource list, the job manager allocates the job node to the corresponding machine node to execute the job program.

As shown in fig. 2, when the job node finishes executing, the job manager informs the resource manager to return the resource of the job node, and after receiving the return application, the resource manager informs the machine node to release the resource of the job node, and the released resource can be allocated to other jobs again.

In a distributed system, the situation that resource application queuing is caused by the exhaustion of the whole cluster resources often occurs, and in the face of the situation, a general processing mode is to queue the resource applications, and when a machine has vacant resources, the queuing queue is traversed, and queuing requests in the queue are sequentially distributed.

However, considering that there are multiple users submitting jobs simultaneously in a distributed system, and that there is a possibility that each user may submit multiple independent jobs, when allocating resources, it is desirable that the total amount of resources taken by different users is fair and equal as much as possible, because if resources favor one user, the jobs submitted by other users will not get the resources. By adopting a mode of traversing the queuing queue and sequentially distributing the queuing requests in the queue, if the jobs arranged in front of the queue all belong to one user group, the remaining resources of the machine are occupied by the user group arranged in the front, and the fair distribution of the resources among different user groups cannot be achieved.

An effective solution for how to keep resources distributed fairly among different user groups has not been proposed at present.

Disclosure of Invention

The application aims to provide a resource allocation method and a resource manager, which can achieve the aim of keeping resources allocated fairly among different user groups.

The application provides a resource allocation method and a resource manager, which are realized as follows:

a method of resource allocation, the method comprising:

acquiring a resource scheduling request;

responding to the scheduling request, and determining the resource quota of each user group according to a preset strategy, wherein the preset strategy is as follows: distributing the total cluster resources among all user groups in equal quantity, and distributing the resources exceeding the demand of the user groups in equal quantity in the user groups with insufficient demand after equal quantity distribution;

and distributing resources for each user group according to the determined resource quota of each user group.

A resource manager comprising a processor and a memory for storing processor-executable instructions that, when executed by the processor, implement:

acquiring a resource scheduling request;

responding to the scheduling request, and determining the resource quota of each user group according to a preset strategy, wherein the preset strategy is as follows: the total cluster resources are distributed among all the user groups in an equal amount, and the resources exceeding the demand of the user groups are distributed in the user groups with insufficient demand in an equal amount after being distributed in an equal amount;

A computer readable storage medium having stored thereon computer instructions which, when executed, implement the steps of the above-described method.

According to the resource allocation method and the resource manager, the total cluster resources are equally allocated among the user groups, and the resources exceeding the user group demand after equal allocation are equally allocated among the user groups with insufficient demand, so that the resources are dynamically and fairly distributed as much as possible after each scheduling, the technical problem that the resources cannot be fairly allocated among different user groups in the existing mode of traversing the queuing queue and sequentially allocating the queuing requests in the queue is solved, and the technical effect of simply and efficiently realizing fair allocation of the resources among different user groups is achieved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without any creative effort.

FIG. 1 is a schematic diagram of an example resource allocation;

FIG. 2 is a schematic diagram of an example resource recovery;

FIG. 3 is a schematic diagram of queuing based sequential allocation;

FIG. 4 is a schematic diagram of a resource allocation method according to an embodiment of the present application;

FIG. 5 is a schematic diagram illustrating resource allocation compared to water filling into a bucket according to an embodiment of the present application;

FIG. 6 is a detailed schematic diagram of water filling into a water bucket according to an embodiment of the application;

FIG. 7 is a schematic diagram of determining boundaries according to an embodiment of the present application;

FIG. 8 is a schematic diagram of a binary tree according to an embodiment of the present application;

FIG. 9 is a diagram of a secondary queue according to an embodiment of the present application;

FIG. 10 is a flow diagram of a resource allocation method according to an embodiment of the application;

FIG. 11 is an architectural diagram of a server according to an embodiment of the present application;

fig. 12 is a block diagram of a resource configuration apparatus according to an embodiment of the present application.

Detailed Description

In order to make those skilled in the art better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Considering that a plurality of users can submit jobs simultaneously in a distributed system, each user can submit a plurality of independent jobs, and different jobs submitted by the same user can be attributed to the same user group (userGroup). When resource allocation is performed, it is desirable that the total amount of resources (CPU, memory, etc.) taken up among different user groups is fair or as equal as possible. Because if the resource is inclined to one of the user groups, the jobs submitted by the other user groups cannot reach the resource.

For example, as shown in FIG. 3, if all jobs queued at the front belong to the same user group, then if the remaining resources of the machine are all allocated to jobs owned by user1, the resources are already split when it is time for jobs in the next user group to be allocated. Specifically, the resource allocation process may be: the remaining resources 10 are all allocated to jobs owned by user1, and it is actually desirable that the resources allocated to jobs owned by user1 are 5, and the resources allocated to jobs owned by user2 are also 5. Of course, this allocation is based on the assumption that the total number of resources taken before user1 and user2 is 0, and if user2 has taken 2 resources before allocation and user1 has taken 0 resource, then in the case of 10 remaining resources, the desired allocation result is: the job owned by user1 allocates 6 resources and the job owned by user2 allocates 4 resources, so that after the allocation is finished, user1 and user2 are allocated 6 resources, respectively, thereby achieving fairness.

However, in the actual implementation, there may also be cases where: the sum of the resource requests for jobs owned by user1 is 5, and the sum of the resource requests for jobs owned by user2, which are also 5 resources allocated, is 10, but only 0 is allocated. If the remaining resources are 10, then the desired allocation results in 10 resources being allocated by user2, rather than 5 resources being allocated by use 2. Although it appears that the allocation of user2 to 10 resources and user1 to 5 resources is unfair. However, all resource requests of user1 have been satisfied.

That is, in the allocation process, in addition to the resources that each user group has been allocated to, the sum of the resource requests of the user groups needs to be considered. That is, it can be understood that if the total number of resource shares of a cluster is 15, then user1 and user2 should be divided equally by 7.5, but since the allocation of user1 to 5 is already satisfied, the extra 2.5 can be taken out again, i.e. the fair allocation continues, and thus user2 is allocated 2.5 again.

That is, if it is desired to maintain resource allocation among different user groups, the purpose of fairly scheduling queuing queues can be obtained, and the following aspects need to be achieved:

1) When the jobs of a plurality of user groups exist in the queuing queue, the resources integrally distributed by different user groups can be ensured to be as fair as possible under the condition that the number of the jobs of the user groups, the size of the resource application request and the arrangement positions of the jobs of the users are different;

2) While fairness is considered, not only the allocated resources of the user group but also the total resource request amount of the user group needs to be considered. Further, it is desirable to achieve efficient distribution.

The method aims to solve the problem that in the existing queuing queue model, when the number of jobs owned by a user group is larger, the application request of single-user job resources is larger, and the position of a user job queue is more advanced (the submission time is earlier), the more resources possibly distributed by the user group are, so that the resource fairness among the user groups cannot be ensured when the resources are distributed. In this example, a queuing queue scheduling method is proposed to keep resources fairly distributed among different user groups, so as to ensure that the result of resources distributed by the different user groups as a whole is as fair as possible even if there are jobs of multiple user groups in the queuing queue, the number of jobs of user groups, the size of resource application requests, and the positions of the jobs in the queue are all different.

As shown in fig. 4, in this example, a resource allocation method is provided, including:

s1: acquiring total cluster resources;

for example, as shown in FIG. 4, the total cluster resources are 50. The user groups to be allocated include user group 1 (demand is 3), user group 2 (demand is 20), user group 3 (demand is 30), user group 4 (demand is 11), and user group 5 (demand is 2).

S2: the total cluster resource amount is divided into user groups;

considering that there are 5 user groups in this example, the bisection is 10 resources per user group.

S3: and determining whether the resource quantity acquired by each user group after the equal division exceeds the requested resource quantity, taking the resource exceeding the requested resource quantity as a re-allocable resource, iteratively dividing the resource exceeding the requested resource quantity into the user groups of which the allocated resource quantity does not reach the requested resource quantity until no re-allocable resource exists, so as to ensure that the total cluster resources are equally allocated among the user groups, and equally allocating the resource exceeding the demand quantity of the user groups in the user groups with insufficient demand quantity after equal allocation.

Since the user group 1 demand is 3 and the user group 5 demand is 2, if 10 is allocated, then user group 1, user group 4 and user group 5 are all out of demand, which corresponds to user group 1 exceeding 7 and user group 5 exceeding 8, i.e. a total of 15. The 15 can be further divided into the user group 2, the user group 3 and the user group 4, so that after the division, the user group 2, the user group 3 and the user group 4 all have 15, and after the further judgment, the requirement of the user group 4 is 11, so that the user group 4 has 4 more, the 4 can be taken out and divided into the user group 2 and the user group 3, and after the division, the user group 2 and the user group 3 are divided into 17 in total.

That is, the final division result is that user group 1: demand is 3, resources are 3, user group 2: demand is 20, resources are 17, user group 3: demand is 30, resources are 17, user group 4: demand 11, resources 11, user group 5: demand is 2 and the assigned resources are 2.

In one embodiment, the resource quota for each user group may be determined as follows:

1) And distributing the total cluster resources according to a fair proportion to obtain the user static balance quota resources.

2) Users (user groups) are divided into two categories, a category: the user resource requirement is less than the user static balance quota resource; b type: the user resource demand is greater than the user statically balanced quota resource.

3) For a type A user, the difference between the static balance quota resource and the user resource requirement can be allocated to a type B user for use; the differences are accumulated and summed to be recorded as the total amount of resources which can be redistributed in the system

4) Will be provided with

And re-allocating the resource quota to the class B user according to a fair proportion, wherein the new available resource quota of the class B user = static balance resource quota + newly obtained resource quota, and is marked as omega.

5) If the omega of the B-type User is larger than the resource requirement of the User, the difference between the omega and the resource requirement of the User can be also distributed to the User of which the omega is still smaller than the resource requirement of the User in the B-type users, namely 3) is skipped to for iteration until the total amount of the resources which can be redistributed

＝0。

As shown in FIG. 5, each user is considered to be a bucket containing water, the bottom edge of the bucket reflects the proportion of resources divided among different users, and the width of the bottom edge of each user can be considered to be the same (i.e., the ratio is equal) because of fair scheduling. The area is the sum of the resource demands of the jobs owned by the User, and the process of allocating resource quotas refers to the process of filling water (total cluster resources) into each bucket, and the resource quotas refer to the amount of resources allocated to each User group.

As shown in fig. 6, in order to uniformly spread the total cluster resources to each user group (pouring water into the bucket) according to a fair proportion (Resource Allocation Ratio), it is possible to ensure that the water level in each bucket rises all at once. If the water level in one bucket is at the top and the cluster still has the remaining allocable resources, the water level in the bucket does not rise any more, and the water is continuously filled into the buckets with the other buckets with the water level which is not at the top until the remaining allocable resources of the cluster are 0 or the water levels of all the buckets are at the top.

When the remaining allocable resources of the cluster are 0, it needs to be determined that the user resource demand is met and the bucket boundary where the user resource demand is not completely met, and the available resource quota of the bucket on the left side of the boundary is the user resource demand, however, the available resource quota of the bucket on the right side of the boundary is a Ratio of H × user, where H is shown in fig. 7 below. Thus, if the boundary sum H can be calculated, the available resource quota per User can be determined. As shown in FIG. 7, below the horizontal line is the total resources of the cluster, above the horizontal line is the sum of the resources that still need to be requested.

Therefore, if the amount of resources that should be allocated to each user group is determined in real time, the amount of resources allocated to each user group can be finally determined by determining the boundary and H.

For example, whether a boundary is located between E and F can be determined by the following equation:

that is, it is determined whether the total cluster resources exceed the requirement of E but are insufficient to satisfy the requirement of F in the above allocation manner, and if it is determined that the total cluster resources exceed the requirement of E but are insufficient to satisfy the requirement of F, the boundary is proved to be before E and F.

For simple and efficient determination of the boundary, the boundary may be determined by a dichotomy, that is, the boundary may be determined in a tree manner, so that the boundary may be found by an algorithm with a time complexity of O (logn).

As shown in fig. 8, for the established binary tree diagram, it can be assumed that the right side of D is the boundary, and the above formula is substituted to obtain:

thus, it can be inferred that the boundary is to the right of D, and therefore, can transition to F.

Assuming that the right side of F is the boundary, substituting the above equation, one can get:

thus, it can be inferred that the boundary is to the left of F, and therefore, moves to node E.

Assuming that the right side of E is the boundary, substituting the above equation, one can get:

thereby determining that the boundary is to the right of E.

After the boundary is determined, H can be solved back by the following equation:

after determining the boundary position and H, the available resource quota per user can be determined: the resource quota of the user on the left side of the boundary is the resource amount requested by the user, and the available resource quota of the user on the right side of the boundary is the ratio H of the user.

Then, after determining the resource quota, the total amount of resources that each user should newly allocate is: resource quota-used resource = x, if x is greater than zero, then it is like the user sorting x amount of resources, and if x is less than zero, then it subtracts | x | amount of resources from the resources already allocated for the user. After the resources are allocated to the user, each of the plurality of user jobs owned by the user may be allocated in sequence.

Considering that it is desirable to allocate resources to users who have acquired less resources first, after determining the available resource quota of each user, resource allocation may be performed in the order of the amount of allocated resources from small to large.

However, it should be noted that the resource allocation in the order of the allocated resource amount from less to more is only an illustrative description, and in practical implementation, the resource allocation may be in other orders, for example, the resource allocation may be in the order of the allocated resource amount from less to less, or the demand amount from less to more, or the demand amount from more to less, and this application is not limited thereto.

In one embodiment, the resource allocation may be performed by constructing a queuing queue. As shown in fig. 9, a 2-stage queuing queue may be constructed, where the first-stage queue is in units of users, and each user structure records the total used resource used of user jobs owned by a user and the total resource request Req of the owned user jobs. The sorting rule of the first-level queue may be that the less the sum of UsedCpu owned by the user group and obtained by the job, the more advanced the user group is, wherein the cpu resource and the mem resource are applied proportionally, so that the sorting can be performed by taking the cpu as a reference. Through the first-level queue, when residual resources appear, resources can be preferentially distributed to jobs owned by a user group with few UsedCpu. The second level queue represents all jobs under the same user group, which may be ordered by time of submission. That is, there is one first level queue and one second level queue per user group.

Fig. 10 is a flowchart of a method of an embodiment of a resource allocation method described herein. Although the present application provides method operational steps or apparatus configurations as illustrated in the following examples or figures, more or fewer operational steps or modular units may be included in the methods or apparatus based on conventional or non-inventive efforts. In the step or structure in which the necessary cause and effect relationship does not logically exist, the execution sequence of the steps or the module structure of the apparatus is not limited to the execution sequence or the module structure described in the embodiment of the present application and shown in the drawings. When the described methods or modular structures are applied in a practical device or end product, they can be executed sequentially or in parallel according to the embodiments or the methods or modular structures shown in the figures (for example, in the environment of parallel processors or multi-thread processing, or even in the environment of distributed processing).

Specifically, as shown in fig. 10, a resource allocation method provided in an embodiment of the present application may include:

s1001: acquiring a resource scheduling request;

s1002: responding to the scheduling request, and determining the resource quota of each user group according to a preset strategy, wherein the preset strategy is as follows: distributing the total cluster resources among all user groups in equal quantity, and distributing the resources exceeding the demand of the user groups in equal quantity in the user groups with insufficient demand after equal quantity distribution;

specifically, determining the resource quota of each user group according to a preset policy may include:

s1: arranging all user groups in the order of less to more demand;

s2: determining a division boundary based on the sorting result;

s3: determining the resource quota of the user group positioned on the left side of the dividing boundary as the resource quantity requested by each user group;

s4: and determining the resource quota of the user group positioned on the right side of the dividing boundary as the resource quantity obtained by averaging the group number of the user group positioned on the right side of the dividing boundary after the total cluster resources are removed from the sum of the requested resource quantities of the user groups positioned on the left side of the dividing boundary.

That is, the user groups may be sequentially divided into a first group and a second group according to the total cluster resource amount based on the sorting result, where the resource quota of the user group in the first group is the resource amount requested by each user group, and the resource quota of the user group in the second group is the resource amount obtained by averaging the group numbers of the user groups located on the right side of the dividing boundary after the sum of the requested resource amounts of the user groups located on the left side of the dividing boundary is removed from the total cluster resource.

In order to determine the boundary efficiently and simply, a binary tree can be established according to the sorting result; and determining the boundary according to the sequence from the root node to the leaf node from the root node of the binary tree as an assumed boundary.

When determining the resource quota, the resource quota may also be determined in an iterative manner, for example, by the following steps:

s1: distributing the cluster total resources evenly among all user groups;

s2: after allocation, repeating the following steps until the amount of allocable resources is 0:

determining a first class user group and a second class user group, wherein the first class user group is a user group of which the distributed resource amount reaches the request amount, and the second class user group is a user group of which the distributed resource amount does not reach the request amount;

accumulating the resource amount of the excessive part of the first user group to obtain the reallocated resource amount;

and distributing the amount of the reallocated resources evenly among the second type user groups.

S1003: and distributing resources for each user group according to the determined resource quota of each user group.

In consideration of rationality in the resource allocation process, resources can be allocated to each user group according to the determined resource quota of each user group and the sequence of the allocated resources from less to more.

In order to implement dynamic and fair resource allocation, allocating resources for each user group according to the determined resource quota of each user group may include:

1) For the user group with the determined resource quota exceeding the allocated resource amount, determining the exceeded resource amount, and extracting the resource with the exceeded resource amount from the resources allocated to the user group;

2) And determining the lacking resource amount for the user group of which the determined resource quota is less than the allocated resource amount, and allocating the resources of the lacking resource amount for the user group.

After the above step S1003, resources may be allocated to the user jobs owned by each user group.

The method embodiments provided in the present application may be executed in a mobile terminal, a computer terminal or a similar computing device. Taking the operation on the server side as an example, fig. 11 is a hardware structure block diagram of the server side of the resource allocation method according to the embodiment of the present invention. As shown in fig. 11, the server 10 may include one or more processors 102 (only one is shown in the figure) (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. It will be understood by those skilled in the art that the structure shown in fig. 11 is only an illustration and is not intended to limit the structure of the electronic device. For example, the server side 10 may also include more or fewer components than shown in FIG. 11, or have a different configuration than shown in FIG. 11.

The memory 104 may be used to store software programs and modules of application software, such as program instructions/modules corresponding to the resource allocation method in the embodiment of the present invention, and the processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, so as to implement the above-mentioned resource allocation method. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission module 106 is used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the transmission module 106 may be a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.

In the software aspect, the resource allocation apparatus may be as shown in fig. 12, and includes: an acquisition unit 1201, a determination unit 1202, and an assignment unit 1203. Wherein:

an obtaining unit 1201, configured to obtain a resource scheduling request;

a determining unit 1202, configured to determine, in response to the scheduling request, a resource quota of each user group according to a preset policy, where the preset policy is: distributing the total cluster resources among all user groups in equal quantity, and distributing the resources exceeding the demand of the user groups in equal quantity in the user groups with insufficient demand after equal quantity distribution;

an allocating unit 1203 is configured to allocate resources to each user group according to the determined resource quota of each user group.

In one embodiment, the determining unit 1202 may specifically arrange the user groups in order of the demanded quantity from less to more; and based on the sequencing result, sequentially dividing the user groups into a first group and a second group according to the total cluster resource amount, wherein the resource quota of the user group in the first group is the resource amount requested by each user group, and the resource quota of the user group in the second group is the resource amount obtained by averaging the number of the user groups positioned on the right side of the dividing boundary after the sum of the requested resource amounts of the user groups positioned on the left side of the dividing boundary is removed from the total cluster resource.

In an embodiment, the determining unit 1202 may specifically establish a binary tree according to the sorting result; and determining the boundary according to the sequence from the root node to the leaf node from the root node of the binary tree as an assumed boundary.

In an embodiment, the determining unit 1202 may specifically distribute the total cluster resources evenly among the user groups; after allocation, repeating the following steps until the amount of allocable resources is 0: determining a first class user group and a second class user group, wherein the first class user group is a user group of which the distributed resource amount reaches the request amount, and the second class user group is a user group of which the distributed resource amount does not reach the request amount; accumulating the resource amount of the excessive part of the first user group to obtain the reallocated resource amount; and distributing the amount of the reallocated resources evenly among the second type user groups.

In an embodiment, the allocating unit 1203 may specifically allocate resources to the user groups in an order from a few to a large number of allocated resources according to the determined resource quotas of the user groups.

In an embodiment, the allocating unit 1203 may specifically determine, for a user group whose determined resource quota exceeds an allocated resource amount, the exceeded resource amount, and extract a resource of the exceeded resource amount from resources allocated to the user group; and determining the lacking resource amount for the user group of which the determined resource quota is less than the allocated resource amount, and allocating the resources of the lacking resource amount for the user group.

According to the resource distribution method and the resource manager, the total cluster resources are distributed among the user groups in an equal amount, and the resources exceeding the user group demand amount after being distributed in an equal amount are distributed in the user groups with insufficient demand amount, so that the resources are dynamically and fairly distributed as much as possible after each scheduling, the technical problem that the resources cannot be fairly distributed among different user groups in the existing mode of traversing the queuing queue and sequentially distributing the queuing requests in the queue is solved, and the technical effect of simply and efficiently realizing fair distribution of the resources among different user groups is achieved.

Although the present application provides method steps as described in an embodiment or flowchart, additional or fewer steps may be included based on routine or non-inventive labor. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of sequences, and does not represent a unique order of performance. When implemented in practice, the apparatus or client products may be executed sequentially or in parallel (e.g., in the context of parallel processors or multi-threaded processing) according to the methods shown in the embodiments or figures.

The apparatuses or modules illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. For convenience of description, the above devices are described as being divided into various modules by functions, and are described separately. The functionality of the modules may be implemented in the same one or more software and/or hardware implementations of the present application. Of course, a module that implements a certain function may be implemented by a plurality of sub-modules or sub-units in combination.

The methods, apparatus or modules described herein may be implemented in computer readable program code to a controller implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, application Specific Integrated Circuits (ASICs), programmable logic controllers and embedded microcontrollers, examples of which include, but are not limited to, the following microcontrollers: ARC 625D, atmel AT91SAM, microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic for the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller in purely computer readable program code means, the same functionality can be implemented by logically programming method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Such a controller may therefore be considered as a hardware component, and the means included therein for performing the various functions may also be considered as a structure within the hardware component. Or even means for performing the functions may be regarded as being both a software module for performing the method and a structure within a hardware component.

Some of the modules in the apparatus described herein may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

From the above description of the embodiments, it is clear to those skilled in the art that the present application can be implemented by software plus necessary hardware. Based on such understanding, the technical solutions of the present application may be embodied in the form of software products or in the implementation process of data migration, which essentially or partially contributes to the prior art. The computer software product may be stored in a storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to perform the methods described in the various embodiments or portions of the embodiments of the present application.

The embodiments in the present specification are described in a progressive manner, and the same or similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. All or portions of the present application are operational with numerous general purpose or special purpose computing system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, mobile communication terminals, multiprocessor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.

While the present application has been described with examples, those of ordinary skill in the art will appreciate that there are numerous variations and permutations of the present application without departing from the spirit of the application, and it is intended that the appended claims encompass such variations and permutations without departing from the spirit of the application.

Claims

1. A method for resource allocation, the method comprising:

acquiring a resource scheduling request;

responding to the scheduling request, and determining the resource quota of each user group according to a preset strategy, wherein the preset strategy is as follows: distributing the total cluster resources among all user groups in equal quantity, and distributing the resources exceeding the user group demand in equal quantity in the user groups with insufficient demand after equal quantity distribution;

distributing resources for each user group according to the determined resource quota of each user group;

the method for determining the resource quota of each user group according to the preset strategy comprises the following steps:

arranging all user groups according to the order of the demand from less to more;

and based on the sequencing result, sequentially dividing the user group into a first grouping and a second grouping according to the total cluster resource amount, wherein the resource quota of the user group in the first grouping is the resource amount requested by each user group, and the resource quota of the user group in the second grouping is the resource amount obtained by averaging the total cluster resources after the requested resource amount of the user group in the first grouping is removed.

2. The method of claim 1, wherein determining the resource quota for each user group according to a preset policy in response to the scheduling request comprises:

distributing the cluster total resources evenly among all user groups;

after allocation, repeating the following steps until the amount of allocable resources is 0:

3. The method of claim 1, wherein allocating resources to each user group according to the determined resource quota of each user group comprises:

and according to the determined resource quota of each user group, allocating resources to each user group in the sequence of less allocated resources to more allocated resources.

4. The method of claim 1, wherein allocating resources to each user group according to the determined resource quota of each user group comprises:

for the user group with the determined resource quota exceeding the allocated resource amount, determining the exceeded resource amount, and extracting the resource with the exceeded resource amount from the resources allocated to the user group;

and determining the lacking resource amount for the user group of which the determined resource quota is less than the allocated resource amount, and allocating the resources of the lacking resource amount for the user group.

5. The method according to any one of claims 1 to 4, wherein after allocating resources for each user group according to the determined resource quota of each user group, the method further comprises:

resources are allocated for user jobs owned by each user group.

6. A resource manager comprising a processor and a memory for storing processor-executable instructions that, when executed by the processor, implement:

acquiring a resource scheduling request;

the method for determining the resource quota of each user group by the processor according to the preset strategy comprises the following steps:

arranging all user groups in the order of less to more demand;

and based on the sequencing result, sequentially dividing the user group into a first group and a second group according to the total cluster resource amount, wherein the resource quota of the user group in the first group is the resource amount requested by each user group, and the resource quota of the user group in the second group is the resource amount obtained by averaging the cluster total resources after the resource amount requested by the user group in the first group is removed.

7. The resource manager of claim 6, wherein the processor determines the resource quota for each user group according to a preset policy in response to the scheduling request, comprising:

distributing the cluster total resources evenly among all user groups;

accumulating the resource amount of the exceeding part of the first user group to obtain the reallocated resource amount;

8. The resource manager of claim 6, wherein the processor allocates resources for each user group according to the determined resource quota of each user group, comprising:

9. The resource manager of claim 6, wherein the processor allocates resources for each user group according to the determined resource quota of each user group, comprising:

10. A computer readable storage medium having stored thereon computer instructions which, when executed, implement the steps of the method of any one of claims 1 to 5.