CN114911618B - Heterogeneous resource allocation method and device, electronic equipment and storage medium - Google Patents

Heterogeneous resource allocation method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN114911618B
CN114911618B CN202210516665.2A CN202210516665A CN114911618B CN 114911618 B CN114911618 B CN 114911618B CN 202210516665 A CN202210516665 A CN 202210516665A CN 114911618 B CN114911618 B CN 114911618B
Authority
CN
China
Prior art keywords
resource
heterogeneous
target user
resources
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210516665.2A
Other languages
Chinese (zh)
Other versions
CN114911618A (en
Inventor
王学敏
孟维业
王涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Telecom Corp Ltd
Original Assignee
China Telecom Corp Ltd
Filing date
Publication date
Application filed by China Telecom Corp Ltd filed Critical China Telecom Corp Ltd
Priority to CN202210516665.2A priority Critical patent/CN114911618B/en
Publication of CN114911618A publication Critical patent/CN114911618A/en
Application granted granted Critical
Publication of CN114911618B publication Critical patent/CN114911618B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

The disclosure provides a heterogeneous resource allocation method, a heterogeneous resource allocation device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: receiving a resource allocation request sent by a target user to a heterogeneous cluster, wherein the target user is a user with the lowest current resource occupancy rate in the heterogeneous cluster, and the current resource occupancy rate is obtained based on preset performance weight calculation; responding to a resource allocation request, and acquiring a target resource demand of a target user, wherein the target resource demand is the resource demand of a task to be executed by the target user; if the target resource demand is smaller than or equal to the total amount of the idle resources of the corresponding type in the heterogeneous cluster, the idle resources are randomly selected and distributed to the target user, so that the target user obtains the resources meeting the target resource demand. The method and the device can allocate the resources in the cluster according to the actual running performance of the equipment, and ensure the fairness of resource allocation.

Description

Heterogeneous resource allocation method and device, electronic equipment and storage medium
Technical Field
The disclosure relates to the technical field of computers, and in particular relates to a heterogeneous resource allocation method, a heterogeneous resource allocation device, electronic equipment and a storage medium.
Background
Fairness for multiple resource allocation in a big data cluster environment is one of the most important indexes of a resource scheduling subsystem, and only if fairness of a resource scheduling method is guaranteed, basic benefits of users are not damaged, and a foundation is laid for realizing sub-user weight resource scheduling by adding user weights.
At present, a common mode is to fairly allocate various resources in a cluster according to a dominant resource fairness (DRF, dominant Resource Fairness) allocation algorithm, but the DRF allocation algorithm only focuses on the allocation quantity of the resources, such as the core number of a CPU, the size of a memory and the like, and does not consider the performance gap among the resources, so that the allocation is unfair when the cluster has heterogeneous resources. Specifically, in an actual large data cluster environment, devices may not be purchased at the same time, and differences exist between brands and models of the devices, such as main frequencies of CPUs, reading rates of memories, and rotational speeds of magnetic disks. In the actual resource allocation process, even though the same number of resources are allocated, the allocation scheme may deviate from the expected effect due to the performance difference between devices, which is contrary to the fairness principle.
It should be noted that the information disclosed in the above background section is only for enhancing understanding of the background of the present disclosure and thus may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
The disclosure provides a heterogeneous resource allocation method, a heterogeneous resource allocation device, electronic equipment and a storage medium, which at least overcome the problem that allocation is unfair due to the fact that only the number of resources is considered when the heterogeneous resources are allocated in the related art to a certain extent.
Other features and advantages of the present disclosure will be apparent from the following detailed description, or may be learned in part by the practice of the disclosure.
According to one aspect of the present disclosure, there is provided a heterogeneous resource allocation method, including:
receiving a resource allocation request sent by a target user to a heterogeneous cluster, wherein the target user is a user with the lowest current resource occupancy rate in the heterogeneous cluster, and the current resource occupancy rate is obtained based on preset performance weight calculation;
responding to the resource allocation request, and acquiring a target resource demand of the target user, wherein the target resource demand is the resource demand of a task to be executed by the target user;
And if the target resource demand is smaller than or equal to the total amount of the idle resources of the corresponding type in the heterogeneous cluster, randomly selecting the idle resources and distributing the idle resources to the target user so that the target user can acquire the resources meeting the target resource demand.
In one embodiment of the present disclosure, after randomly selecting free resources and assigning to a target user to cause the target user to acquire resources that meet the target resource demand, the method includes:
And updating the current resource occupancy rate of the target user according to the preset performance weight.
In one embodiment of the present disclosure, the heterogeneous clusters include multiple types of resources therein;
The updating the current resource occupancy rate of the target user according to the preset performance weight specifically comprises:
According to the preset performance weight, calculating the heterogeneous resource occupancy rate corresponding to each type of resources occupied by the target user;
And taking the maximum value of the heterogeneous resource occupancy rate of the target user for each type of resource as the current resource occupancy rate of the target user.
In one embodiment of the present disclosure, calculating, according to a preset performance weight, heterogeneous resource occupancy rates corresponding to each type of resources occupied by the target user, specifically includes:
according to a preset performance weight, carrying out weighted summation on the heterogeneous resources of the same type occupied by the target user to obtain a first weighted summation; according to the preset performance weight, carrying out weighted summation on the corresponding type of heterogeneous resources in the heterogeneous cluster to obtain a second weighted sum;
and taking the ratio of the first weighted sum and the second weighted sum of each type of resource as the heterogeneous resource occupancy rate respectively corresponding to each type of resource.
In one embodiment of the present disclosure, the preset performance weight is generated based on a resource performance, and the magnitude of the preset performance weight is proportional to the resource performance.
In one embodiment of the present disclosure, when there are multiple users with the lowest resource occupancy in the heterogeneous cluster, a user with the lowest resource occupancy is randomly selected as the target user.
In one embodiment of the present disclosure, after updating the current resource occupancy of the target user according to a preset performance weight, the method further comprises:
Repeatedly receiving a resource allocation request sent by a target user to the heterogeneous cluster, and updating the current resource occupancy rate of the target user after the resource allocation is carried out until the target resource demand of the target user is greater than the total idle resource in the heterogeneous cluster, and ending the resource allocation.
According to another aspect of the present disclosure, there is provided a heterogeneous resource allocation apparatus including:
the request receiving module is used for receiving a resource allocation request sent by a target user to the heterogeneous cluster, wherein the target user is a user with the lowest current resource occupancy rate in the heterogeneous cluster, and the current resource occupancy rate is obtained based on preset performance weight calculation;
The demand acquisition module is used for responding to the resource allocation request and acquiring the target resource demand of the target user, wherein the target resource demand is the resource demand of the task to be executed by the target user;
And the resource allocation module is used for randomly selecting idle resources and allocating the idle resources to a target user when the target resource demand is smaller than or equal to the total idle resources of the corresponding type in the heterogeneous cluster, so that the target user can acquire the resources meeting the target resource demand.
According to still another aspect of the present disclosure, there is provided an electronic apparatus including:
A processor; and
A memory for storing executable instructions of the processor; wherein the processor is configured to perform the heterogeneous resource allocation method described above via execution of the executable instructions.
According to yet another aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the heterogeneous resource allocation method described above.
According to the heterogeneous resource allocation method, the heterogeneous resource allocation device, the electronic equipment and the storage medium, on the basis of the maximum and minimum fairness (Max-MIN FAIRNESS) allocation principle, the performance management of heterogeneous resources in the cluster is refined by introducing the preset performance weight, and the resource occupancy rate of the user is calculated correspondingly based on the preset performance weight, so that resources in the cluster are allocated according to the actual performance of equipment operation, and the fairness of resource allocation is guaranteed.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure. It will be apparent to those of ordinary skill in the art that the drawings in the following description are merely examples of the disclosure and that other drawings may be derived from them without undue effort.
FIG. 1 illustrates a schematic diagram of a heterogeneous cluster system architecture in an embodiment of the present disclosure;
FIG. 2 illustrates a flow chart of a heterogeneous resource allocation method in an embodiment of the present disclosure;
FIG. 3 is a flowchart of a method for acquiring a current resource occupancy rate in an embodiment of the disclosure;
FIG. 4 is a flowchart of a method for calculating occupancy of heterogeneous resources in an embodiment of the disclosure;
Fig. 5 illustrates an application scenario diagram of heterogeneous resource allocation in an embodiment of the present disclosure;
FIG. 6 is a schematic diagram of a heterogeneous resource allocation device according to an embodiment of the present disclosure; and
Fig. 7 shows a block diagram of an electronic device in an embodiment of the disclosure.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus a repetitive description thereof will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software or in one or more hardware modules or integrated circuits or in different networks and/or processor devices and/or microcontroller devices.
For ease of understanding, the following first describes the background art to which the present disclosure relates one by one, and the inventive concepts of the present disclosure are directed.
Heterogeneous clusters, which refer to high performance computing systems composed of multiple different types of computing nodes, each of which may be any device with computing capabilities, thereby providing computing power for user tasks. The computing nodes in the cluster are open, so that users can not only add or delete the nodes at will, but also submit tasks with different demands to any one of the computing nodes freely, and the computing nodes of the whole cluster are shared by all users. Heterogeneous clusters can be divided into type-heterogeneous clusters and resource-heterogeneous clusters. Type heterogeneity refers to the different Instruction Set Architecture (ISA) that each computing node has, and different Operating System (OS) types; resource isomerism refers to the difference of the ownership or performance of resources (mainly CPU, memory and read-write resources) of each computing node, and heterogeneous clusters in this disclosure mainly refer to clusters of resource isomerism. It should be appreciated that, based on the same inventive concept, the method provided by the present disclosure may also be applicable to type-heterogeneous clusters, provided that preset performance weights can be determined.
The DRF allocation algorithm is a general maximum and minimum fair allocation strategy of multiple resources, and the core idea is that the allocation of resources of a user should be determined by the dominant share of resources of the user in multiple environments. The dominant share of resources is one of the largest share of all the various resources that have been allocated to the user. In short, the DRF allocation algorithm attempts to maximize the smallest dominant share among all users. It is worth noting that, when the DRF allocation algorithm performs resource allocation, since only the occupation share of the user to the resource, that is, the occupation ratio of the user on the number of the resources, in the cluster with heterogeneous resources, the fairness of allocation measured by the number of the resources still causes a large difference in calculation power obtained between different users, so that the final allocation result cannot reach the expectations.
In order to solve the technical problems, the scheme provided by the present disclosure includes that firstly, a heterogeneous cluster receives a resource allocation request sent by a target user to the heterogeneous cluster; then, the heterogeneous cluster responds to the resource allocation request to acquire the target resource demand of the target user; if the target resource demand is smaller than or equal to the total amount of the idle resources of the corresponding type in the heterogeneous cluster, the idle resources are randomly selected and distributed to the target user, so that the target user obtains the resources meeting the target resource demand. The target users in the present disclosure are users with the lowest current resource occupancy rate in the heterogeneous cluster, the current resource occupancy rate is obtained based on the preset performance weight calculation of the resources in the heterogeneous cluster, and the preset performance weight is introduced to distinguish the performance of the resources, so that the present disclosure can realize fairness allocation in terms of calculation.
Fig. 1 shows a schematic diagram of an exemplary system architecture to which a heterogeneous resource allocation method or a heterogeneous resource allocation apparatus of an embodiment of the present disclosure may be applied.
As shown in fig. 1, heterogeneous cluster architecture 100 includes multiple types of computing nodes 102 (three different configurations of computing nodes are shown in the figure) interconnected in network 101, with which a user's local computing device (e.g., personal Digital Assistant (PDA) or cellular telephone 103, desktop computer 104, laptop computer 105, and/or automobile computer system 106) may communicate. Computing nodes 102 may also communicate with each other. They may be grouped (not shown) physically or virtually in one or more networks to form different types of clusters, such as high availability clusters, load balancing clusters, high performance computing clusters, or a combination thereof, depending on traffic requirements.
It should be appreciated that the types of local computing devices 103, 104, 105, 106 used by the users shown in fig. 1 are intended to be illustrative only, and that the computing nodes 102 and heterogeneous cluster architecture 100 may communicate with any type of computerized device via any type of network and/or network-addressable connection (e.g., using a web browser).
Those skilled in the art will appreciate that the number of computing nodes, networks, and local computing devices in FIG. 1 are merely illustrative, and that any number of computing nodes, networks, and local computing devices may be provided as desired. The embodiments of the present disclosure are not limited in this regard.
The present exemplary embodiment will be described in detail below with reference to the accompanying drawings and examples.
First, in the embodiments of the present disclosure, a heterogeneous resource allocation method is provided, where the method may be performed by any electronic device with computing processing capability.
Fig. 2 shows a flowchart of a heterogeneous resource allocation method in an embodiment of the present disclosure, and as shown in fig. 2, the heterogeneous resource allocation method provided in the embodiment of the present disclosure includes the following steps:
s202, receiving a resource allocation request sent by a target user to the heterogeneous cluster, wherein the target user is the user with the lowest current resource occupancy rate in the heterogeneous cluster, and the current resource occupancy rate is obtained based on preset performance weight calculation.
It should be noted that, the resource allocation request includes at least one task to be executed of the target user; the task to be executed may be a task that the target user has not started to execute but is about to execute, or may be a task that the target user is executing but does not have enough currently held resources to complete, and it should be understood that, in the present disclosure, the task that enables the target user to generate a resource requirement may be understood as the task to be executed.
It should be noted that, the resource requirement information carries the requirement of the target user for at least one type of resource, and may specifically include type information and number information of the required resource.
In some embodiments, the target users are obtained by cluster screening, and the users with the lowest current resource occupancy rate are selected as the target users by comparing the current resource occupancy rates of the users in the current cluster.
Specifically, the current resource occupancy of each user in the heterogeneous cluster may be calculated and updated after each allocation of resources to the target user by the cluster, where the resource occupancy of the users not allocated resources may be initially 0.
In some embodiments, when there are multiple users with the lowest resource occupancy in the heterogeneous cluster, a user with the lowest resource occupancy is randomly selected as the target user.
It should be noted that, since the resources occupied by the users in the cluster are usually of multiple types, in order to balance the influence of different types of resources on the allocation fairness, for each user in the cluster, the maximum value of the heterogeneous resource occupancy rates of the user on different types of resources may be selected as the current resource occupancy rate of the user.
Specifically, the calculation mode of the heterogeneous resource occupancy rate is as follows: according to the preset performance weight, carrying out weighted summation on heterogeneous resources of the same type occupied by the target user to obtain a first weighted sum; according to the preset performance weight, carrying out weighted summation on the corresponding type of heterogeneous resources in the heterogeneous cluster to obtain a second weighted sum; and taking the ratio of the first weighted sum and the second weighted sum of each type of resource as the heterogeneous resource occupancy rate respectively corresponding to each type of resource.
It should be appreciated that when a user occupies a single type of resource, the same calculation mode may be utilized to calculate the heterogeneous resource occupancy of that type of resource, and the heterogeneous resource occupancy of that type of resource is directly used as the current resource occupancy of that user.
It should be noted that the preset performance weight of the heterogeneous resource is positively related to the performance of the unit heterogeneous resource, and is independent of the size or number of heterogeneous resources.
In some embodiments, the preset performance weights of the heterogeneous resources may be set according to the performance ratio of the heterogeneous resources of the same type, for example, when CPUs whose main frequencies are 1.6GHz, 2.0GHz and 2.4GHz exist in the cluster, the preset performance weights of the three may be set to 1.6, 2.0 and 2.4, respectively, and may also be set to 4, 5 and 6, respectively, for the convenience of calculation and statistics.
It should be noted that, the measurement manner of the performance of the heterogeneous resources is not limited to the operating frequency, and different instruction sets, processes, architectures, etc. among the heterogeneous resources can be used as consideration factors of the preset performance weights, which are not limited in the embodiments of the present disclosure.
S204, responding to the resource allocation request, and acquiring the target resource demand of the target user, wherein the target resource demand is the resource demand of the task to be executed by the target user.
It should be noted that, after receiving the resource allocation request of the target user, the heterogeneous cluster may obtain the target resource demand of the target user by analyzing the resource allocation request of the target user.
In some embodiments, the target resource demand may be the amount of resources necessary for the target user to perform the next task.
In some embodiments, the resources required in the target resource demand may be one type of resource (e.g., CPU) or may be multiple types of resources (e.g., CPU, memory, hard disk), which is not limited by the embodiments of the present disclosure.
S206, if the target resource demand is smaller than or equal to the total amount of the idle resources of the corresponding type in the heterogeneous cluster, randomly selecting the idle resources and distributing the idle resources to the target user so that the target user can acquire the resources meeting the target resource demand.
It should be noted that, when the target resource demand includes multiple types of resources, the multiple types of resources should be compared respectively. And if the required quantity of each type of resource is smaller than or equal to the total quantity of the idle resources of the corresponding type in the heterogeneous cluster, randomly selecting the idle resources on any equipment in the heterogeneous cluster and distributing the idle resources to the target user.
After each time of resource allocation, the current resource occupancy rate of the target user needs to be updated because the resources occupied by the target user change.
In some embodiments, by repeatedly performing the above steps S202 to S206, the resource allocation may be performed multiple times until the target resource demand of the target user is greater than the total amount of free resources in the heterogeneous cluster, and the resource allocation ends.
It should be noted that, in the process of repeating execution, since the current resource occupancy rate of the users in the cluster will change after each resource allocation, the target user is the user with the lowest current resource occupancy rate, so that the target user may also change when starting a new round of allocation.
Fig. 3 shows a flowchart of a current resource occupancy rate obtaining method in an embodiment of the present disclosure, where the method provided in the embodiment is applied to a heterogeneous cluster where multiple types of resources exist, as shown in fig. 3, the current resource occupancy rate obtaining method provided in the embodiment of the present disclosure includes the following steps:
S302, according to the preset performance weight, the heterogeneous resource occupancy rate corresponding to each type of resource occupied by the target user is calculated.
S304, taking the maximum value of the heterogeneous resource occupancy rate of the target user for each type of resource as the current resource occupancy rate of the target user.
Since the implementation manner and technical effects of the embodiment of the present disclosure are similar to those of the heterogeneous resource allocation method shown in fig. 2, the repetition is not repeated.
Fig. 4 further shows a flowchart of a method for calculating the occupancy rate of heterogeneous resources in an embodiment of the present disclosure, as shown in fig. 4, where the method for calculating the occupancy rate of heterogeneous resources provided in the embodiment of the present disclosure includes the following steps:
S402, carrying out weighted summation on heterogeneous resources of the same type occupied by a target user according to preset performance weights to obtain a first weighted summation; and carrying out weighted summation on the heterogeneous resources of the corresponding type in the heterogeneous cluster according to the preset performance weight to obtain a second weighted sum.
S404, the ratio of the first weighted sum and the second weighted sum of each type of resource is used as the heterogeneous resource occupancy rate corresponding to each type of resource.
Since the implementation manner and technical effects of the embodiment of the present disclosure are similar to those of the heterogeneous resource allocation method shown in fig. 2, the repetition is not repeated.
For easy understanding, an embodiment will be provided below, which is a description of the heterogeneous resource allocation method provided in the present disclosure and a calculation process of the current resource occupancy rate of the target user, in combination with specific parameters in the heterogeneous resource allocation process.
In the embodiment of the disclosure, it is assumed that P pieces of equipment, m kinds of resources, and n users exist in a heterogeneous cluster, i is set to represent equipment, 1.ltoreq.i.ltoreq.p, j is set to represent resource types, 1.ltoreq.j.ltoreq.m, k is set to represent users, 1.ltoreq.k.ltoreq.n, and user h (i.e., k=h) is set as a target user.
Based on the above assumptions, the following matrices and vectors are defined:
R Pm(rij) is a p×m-dimensional matrix for representing the total amount of resources in the heterogeneous cluster, and the element R ij in the matrix is used to represent the total amount of j-type resources on device i.
A Pm(aij) is a p×m dimension matrix, which is used to represent the preset performance weight of the resource, and the element a ij in the matrix is used to represent the preset performance weight of the j-type resource on the device i.
C Pm(cij) is a p×m dimension matrix, which is used to represent the resource occupancy of each device in the heterogeneous cluster, and element C ij in the matrix is used to represent the occupancy of j-type resources on device i, and is initially 0.
U nm(ukj) is an n×m dimension matrix, which is used to represent the number of occupied resources of each user in the heterogeneous cluster, and the element U kj in the matrix is used to represent the occupied amount of the user k for the j type resource, and is initially 0.
S n(sk) is an n-dimensional vector, which is used to represent the current resource occupancy rate of each user in the heterogeneous cluster, and different elements S k in the vector are respectively used to represent the current resource occupancy rate of user k in the heterogeneous cluster, and are initially 0, where S h is the current resource occupancy rate of the target user, that is, S n(sk) is the minimum element in the vector.
On the premise that the heterogeneous cluster acquires the target resource demand D of the target user h, whether the idle resource in the heterogeneous cluster can meet the target resource demand is judged first, and based on the description, the formula (1) is obtained.
If the amount of free resources in the heterogeneous cluster can meet the target amount of resource demand, that is, when the inequality is satisfied, the amount of resources W Pm(wij is randomly extracted from the heterogeneous cluster), where W Pm(wij) is a p×m-dimensional matrix, which is used to represent the amount of resources allocated to the target user by the heterogeneous cluster, and element W ij in the matrix is used to represent the amount of resources of type j extracted from device i, as shown in the following formula (2).
It should be noted that, in the above formulas (1) and (2), when the target resource demand D includes multiple types of resources, that is, when j has multiple values, the corresponding type of resources should be extracted from the heterogeneous cluster and allocated to the target user h according to the resource type of the target user demand, for the target resource demand of each type of resources, and because the specific calculation modes are the same, the embodiments of the disclosure will not be repeated.
After the target user is allocated to the target user and the extraction meeting the target resource demand is performed, updating the resource occupation matrix C Pm(cij of each device in the heterogeneous cluster), where C 1Pm(c1ij) is the resource occupation matrix of each device before the resource allocation, and C 2Pm(c2ij) is the resource occupation matrix of each device after the resource allocation (i.e. the current device occupation matrix).
C2Pm(c2ij)=C1Pm(c1ij)+WPm(wij) (3)
Updating the resource occupation amount of the target user, wherein u 1hj is the occupation amount of the target user h on the j-type resource before the resource allocation, and u 2hj is the occupation amount of the target user h on the j-type resource after the resource allocation (namely the current user occupation amount), as shown in the following formula (4).
And further calculating the current resource occupancy rate S h of the target user h according to the current occupancy rate of the user h to the j-type resource as shown in the following formula (5).
It should be noted that, as described in the previous embodiment, when a certain user occupies multiple types of resources at the same time, the maximum value of the heterogeneous resource occupancy rates of the user for different types of resources may be selected as the current resource occupancy rate of the user.
Specifically, in the calculation process of the above formula (4) and formula (5), when the target user h occupies multiple types of resources, it is necessary to traverse all types of resources to perform calculation, even if j sequentially takes 1 to m, respectively calculate the resource occupancy rate of the target user h to each type of resources, and take the maximum value in the calculation result as the current resource occupancy rate, that is, as shown in the above formula (5), wherein,For representing taking j to take the maximum value of the calculation result of 1 to m respectively.
For ease of understanding, a specific application scenario will be provided below in conjunction with fig. 5, and by comparing the disclosure with the related art, the technical effects of the heterogeneous resource allocation method provided by the disclosure are shown, and the application scenario is merely for explaining the present invention and is not intended to limit the present invention.
As shown in fig. 5, assume that the cluster in the embodiment of the disclosure is composed of two devices, namely a device a and a device B, where the device a is configured as a 4-core CPU with a main frequency of 3.2GHz and an 8GB memory with a frequency of 3200 MHz; device B was configured as an 8-core CPU with a main frequency of 1.6GHz and a 16GB memory with a frequency of 1600 MHz.
At this time, the user a and the user B simultaneously request resource allocation from the cluster to start respective tasks to be executed. In the embodiment of the disclosure, in order to intuitively allocate and display the influence of the allocation method on allocation fairness, the resource requirements of the tasks to be executed by the user a and the user B are the same, for example, the tasks to be executed by the user a and the user B both need a 2-core CPU and a 2GB memory; and the user A and the user B respectively have a plurality of same tasks to be executed to wait for operation, namely, the user A and the user B respectively need to be allocated for a plurality of times to simultaneously execute a plurality of tasks to be executed, and each allocated resource is a 2-core CPU and a 2GB memory.
It should be noted that, because when the current resource occupancy rates of the plurality of users are the same and are the lowest, any user is selected to allocate the resource preferentially, and the allocation result is not affected, in the embodiment of the present disclosure, in order to avoid the influence of the random factor, the resource is allocated to the user a preferentially when the above-mentioned resource occupancy rates are the same.
Under the above application scenario, according to the allocation manner in the related art, that is, the allocation is only performed according to the number and the size of the resources during the allocation, so that the number of the allocated resources is balanced, and after the resource allocation is performed for multiple times, the final scheduling process and the final scheduling result are as shown in table 1 below.
Table 1 related art resource allocation results
After performing the resource allocation 6 times as in the above table 1, since neither device a nor device B has a free CPU, the resource demand of the task to be executed by the user a or the user B cannot be satisfied, and thus the resource allocation ends.
Specifically, as shown in table 1, after the allocation is finished, the user a obtains resources (6 core CPU,6GB memory) for simultaneously executing 3 tasks to be executed, the user B also obtains resources (6 core CPU,6GB memory) for simultaneously executing 3 tasks to be executed, and on the premise that the cluster resources are fully utilized, the user a and the user B obtain the same number of resources.
It should be noted that, since the resources allocated to the user a are concentrated in the device a, the resources allocated to the user B tend to be the device B, and since the resources in the cluster are heterogeneous resources, the performance of the CPU and the memory in the device a is much higher than that of the device B. Therefore, in this resource allocation manner, although both the user a and the user B obtain resources capable of executing 3 tasks to be executed, the user a obtains much more computing power than the user B from the viewpoint of obtaining computing power due to a large performance difference between the device a and the device B, that is, the user a performs the tasks at a much higher speed than the user B does, and thus unfairness in actually executing the tasks is caused.
Under the same application scenario, based on the heterogeneous resource allocation method provided by the present disclosure, performance weights can be preset for each type of resource on the device a and the device B respectively. For example, the performance weights may be set according to a performance scaling relationship between the same type of heterogeneous resources.
Specifically, in the embodiment of the present disclosure, since the CPU performance ratio of the device a and the device B is 2:1 (3.2 ghz:1.6 ghz) and the memory performance ratio is 2:1 (32000 mhz:1600 mhz). Therefore, the CPU preset performance weights of the device a and the device B may be set to 2 and 1, respectively, and the memory preset performance weights of the device a and the device B may be set to 2 and 1, respectively.
Table 2 resource allocation results of the present disclosure
After performing the resource allocation 6 times as in the above table 2, since neither device a nor device B has a free CPU, the resource demand of the task to be executed by the user a or the user B cannot be satisfied, and thus the resource allocation ends.
Specifically, as shown in table 2, after the allocation is finished, the user a obtains resources (4-core CPU,4GB memory) for simultaneously executing 2 tasks, and the user B also obtains resources (8-core CPU,8GB memory) for simultaneously executing 4 tasks, so that on the premise that the cluster resources are fully utilized, the user a and the user B obtain different numbers of resources.
It should be noted that, similar to the allocation process in table 1, the resources allocated to the user a are concentrated in the device a, the resources allocated to the user B tend to be in the device B, and the performance of the CPU and the memory in the device a is much higher than that of the device B. In this resource allocation method of the present disclosure, even if the number of resources obtained by the user B is twice that of the user a, since the performance of the unit resources obtained by the user a is twice that of the user B, the computing power obtained by the user a and the user B is substantially the same, and the same task can be performed in a unit time.
Further, by comparing the heterogeneous resource allocation method provided by the embodiment of the present disclosure with the related technology in the same application scenario, it can be found that by introducing the preset performance weight, the heterogeneous resource performance is clearly distinguished during execution, so that the problem that the related technology is unfair when applied to heterogeneous resource allocation is solved.
Based on the same inventive concept, the embodiment of the present disclosure also provides a heterogeneous resource allocation device, as described in the following embodiments. Since the principle of solving the problem of the embodiment of the device is similar to that of the embodiment of the method, the implementation of the embodiment of the device can be referred to the implementation of the embodiment of the method, and the repetition is omitted.
Fig. 6 shows a schematic diagram of a heterogeneous resource allocation apparatus according to an embodiment of the present disclosure, as shown in fig. 6, the apparatus 600 includes:
the user selection module 601 is configured to receive a resource allocation request sent by a target user to a heterogeneous cluster, where the target user is a user with a lowest current resource occupancy rate in the heterogeneous cluster, and the current resource occupancy rate is obtained based on a preset performance weight calculation.
The task obtaining module 602 is configured to obtain, in response to the resource allocation request, a target resource demand of the target user, where the target resource demand is a resource demand of a task to be executed by the target user.
And the resource allocation module 603 is configured to randomly select the idle resources and allocate the idle resources to the target user when the target resource demand is less than or equal to the total amount of idle resources of the corresponding type in the heterogeneous cluster, so that the target user obtains the resources that meet the target resource demand.
In some embodiments, the apparatus 600 further includes a resource calculation module configured to update the current resource occupancy of the target user according to a preset performance weight.
Specifically, when multiple types of resources are included in the heterogeneous cluster, the resource computation module is specifically configured to: according to the preset performance weight, calculating heterogeneous resource occupancy rates corresponding to each type of resources occupied by the target user respectively; and taking the maximum value of the heterogeneous resource occupancy rate of the target user for each type of resource as the current resource occupancy rate of the target user.
Further, the mode of calculating the heterogeneous resource occupancy rate by the resource calculation module is specifically as follows: according to the preset performance weight, carrying out weighted summation on heterogeneous resources of the same type occupied by the target user to obtain a first weighted sum; according to the preset performance weight, carrying out weighted summation on the corresponding type of heterogeneous resources in the heterogeneous cluster to obtain a second weighted sum; and taking the ratio of the first weighted sum and the second weighted sum of each type of resource as the heterogeneous resource occupancy rate respectively corresponding to each type of resource.
In some embodiments, the preset performance weights are generated based on resource performance, and the magnitude of the preset performance weights is proportional to the resource performance.
In some embodiments, the user selection module 601 is further configured to: when a plurality of users with the lowest resource occupancy rate exist in the heterogeneous cluster, randomly selecting one user with the lowest resource occupancy rate as a target user.
In some embodiments, the user selection module 601, the task acquisition module 602, and the resource allocation module 603 may be repeatedly invoked until the resource allocation module 603 fails to allocate resources meeting the resource requirements of the target user to the target user, and the resource allocation ends.
It should be noted that, when the heterogeneous resource allocation device provided in the foregoing embodiment is used for heterogeneous resource allocation, only the division of the foregoing functional modules is used for illustration, in practical application, the foregoing functional allocation may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the heterogeneous resource allocation device and the heterogeneous resource allocation method provided in the foregoing embodiments belong to the same concept, and specific implementation processes thereof are detailed in the method embodiments, which are not repeated herein.
Those skilled in the art will appreciate that the various aspects of the present disclosure may be implemented as a system, method, or program product. Accordingly, various aspects of the disclosure may be embodied in the following forms, namely: an entirely hardware embodiment, an entirely software embodiment (including firmware, micro-code, etc.) or an embodiment combining hardware and software aspects may be referred to herein as a "circuit," module "or" system.
An electronic device 700 according to such an embodiment of the present disclosure is described below with reference to fig. 7. The electronic device 700 shown in fig. 7 is merely an example and should not be construed to limit the functionality and scope of use of embodiments of the present disclosure in any way.
As shown in fig. 7, the electronic device 700 is embodied in the form of a general purpose computing device. Components of electronic device 700 may include, but are not limited to: the at least one processing unit 710, the at least one memory unit 720, and a bus 730 connecting the different system components, including the memory unit 720 and the processing unit 710.
Wherein the storage unit stores program code that is executable by the processing unit 710 such that the processing unit 710 performs steps according to various exemplary embodiments of the present disclosure described in the above-described "exemplary methods" section of the present specification. For example, the processing unit 710 may perform the following steps of the method embodiment described above: receiving a resource allocation request sent by a target user to a heterogeneous cluster, wherein the target user is a user with the lowest current resource occupancy rate in the heterogeneous cluster, and the current resource occupancy rate is obtained based on preset performance weight calculation; responding to a resource allocation request, and acquiring a target resource demand of a target user, wherein the target resource demand is the resource demand of a task to be executed by the target user; if the target resource demand is smaller than or equal to the total amount of the idle resources of the corresponding type in the heterogeneous cluster, the idle resources are randomly selected and distributed to the target user, so that the target user obtains the resources meeting the target resource demand.
The memory unit 720 may include readable media in the form of volatile memory units, such as Random Access Memory (RAM) 7201 and/or cache memory 7202, and may further include Read Only Memory (ROM) 7203.
The storage unit 720 may also include a program/utility 7204 having a set (at least one) of program modules 7205, such program modules 7205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
Bus 730 may be a bus representing one or more of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
The electronic device 700 may also communicate with one or more external devices 740 (e.g., keyboard, pointing device, bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 700, and/or any device (e.g., router, modem, etc.) that enables the electronic device 700 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 750. Also, electronic device 700 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network, such as the Internet, through network adapter 760. As shown, network adapter 760 communicates with other modules of electronic device 700 over bus 730. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 700, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.
From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.) or on a network, including several instructions to cause a computing device (may be a personal computer, a server, a terminal device, or a network device, etc.) to perform the method according to the embodiments of the present disclosure.
In an exemplary embodiment of the present disclosure, a computer-readable storage medium, which may be a readable signal medium or a readable storage medium, is also provided. On which a program product is stored which enables the implementation of the method described above of the present disclosure. In some possible implementations, various aspects of the disclosure may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the steps according to the various exemplary embodiments of the disclosure as described in the "exemplary methods" section of this specification, when the program product is run on the terminal device.
More specific examples of the computer readable storage medium in the present disclosure may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
In this disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Alternatively, the program code embodied on a computer readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
In particular implementations, the program code for carrying out operations of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).
It should be noted that although in the above detailed description several modules or units of a device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit in accordance with embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into a plurality of modules or units to be embodied.
Furthermore, although the steps of the methods in the present disclosure are depicted in a particular order in the drawings, this does not require or imply that the steps must be performed in that particular order, or that all illustrated steps be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform, etc.
From the description of the above embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.) or on a network, including several instructions to cause a computing device (may be a personal computer, a server, a mobile terminal, or a network device, etc.) to perform the method according to the embodiments of the present disclosure.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any adaptations, uses, or adaptations of the disclosure following the general principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

Claims (9)

1. A heterogeneous resource allocation method, comprising:
receiving a resource allocation request sent by a target user to a heterogeneous cluster, wherein the target user is a user with the lowest current resource occupancy rate in the heterogeneous cluster, and the current resource occupancy rate is obtained based on preset performance weight calculation, and the preset performance weight of the heterogeneous resource is positively correlated with the performance of a unit heterogeneous resource;
responding to the resource allocation request, and acquiring a target resource demand of the target user, wherein the target resource demand is the resource demand of a task to be executed by the target user;
if the target resource demand is smaller than or equal to the total amount of idle resources of the corresponding type in the heterogeneous cluster, randomly selecting idle resources and distributing the idle resources to a target user so that the target user can acquire the resources meeting the target resource demand;
According to a preset performance weight, calculating heterogeneous resource occupancy rates corresponding to each type of resources occupied by the target user respectively, wherein the heterogeneous resource occupancy rates specifically comprise:
according to a preset performance weight, carrying out weighted summation on the heterogeneous resources of the same type occupied by the target user to obtain a first weighted summation; according to the preset performance weight, carrying out weighted summation on the corresponding type of heterogeneous resources in the heterogeneous cluster to obtain a second weighted sum;
and taking the ratio of the first weighted sum and the second weighted sum of each type of resource as the heterogeneous resource occupancy rate respectively corresponding to each type of resource.
2. The heterogeneous resource allocation method according to claim 1, wherein after randomly selecting a free resource and allocating to a target user so that the target user obtains a resource satisfying the target resource demand, the method comprises:
And updating the current resource occupancy rate of the target user according to the preset performance weight.
3. The heterogeneous resource allocation method according to claim 2, wherein the heterogeneous cluster includes a plurality of types of resources;
The updating the current resource occupancy rate of the target user according to the preset performance weight specifically comprises:
According to the preset performance weight, calculating the heterogeneous resource occupancy rate corresponding to each type of resources occupied by the target user;
And taking the maximum value of the heterogeneous resource occupancy rate of the target user for each type of resource as the current resource occupancy rate of the target user.
4. A heterogeneous resource allocation method according to any one of claims 1 to 3, wherein the preset performance weight is generated based on resource performance and the magnitude of the preset performance weight is proportional to the resource performance.
5. The heterogeneous resource allocation method according to claim 1, wherein the method:
when a plurality of users with the lowest resource occupancy rate exist in the heterogeneous cluster, randomly selecting one user with the lowest resource occupancy rate as a target user.
6. The heterogeneous resource allocation method according to claim 2, wherein after updating the current resource occupancy of the target user according to a preset performance weight, the method further comprises:
Repeatedly receiving a resource allocation request sent by a target user to the heterogeneous cluster, and updating the current resource occupancy rate of the target user after the resource allocation is carried out until the target resource demand of the target user is greater than the total idle resource in the heterogeneous cluster, and ending the resource allocation.
7. A heterogeneous resource allocation apparatus, comprising:
The request receiving module is used for receiving a resource allocation request sent by a target user to the heterogeneous cluster, wherein the target user is a user with the lowest current resource occupancy rate in the heterogeneous cluster, the current resource occupancy rate is obtained based on preset performance weight calculation, and the preset performance weight of the heterogeneous resource is positively correlated with the performance of the unit heterogeneous resource;
The demand acquisition module is used for responding to the resource allocation request and acquiring the target resource demand of the target user, wherein the target resource demand is the resource demand of the task to be executed by the target user;
The resource allocation module is used for randomly selecting idle resources and allocating the idle resources to a target user when the target resource demand is smaller than or equal to the total idle resources of the corresponding type in the heterogeneous cluster, so that the target user can acquire the resources meeting the target resource demand;
According to a preset performance weight, calculating heterogeneous resource occupancy rates corresponding to each type of resources occupied by the target user respectively, wherein the heterogeneous resource occupancy rates specifically comprise:
according to a preset performance weight, carrying out weighted summation on the heterogeneous resources of the same type occupied by the target user to obtain a first weighted summation; according to the preset performance weight, carrying out weighted summation on the corresponding type of heterogeneous resources in the heterogeneous cluster to obtain a second weighted sum;
and taking the ratio of the first weighted sum and the second weighted sum of each type of resource as the heterogeneous resource occupancy rate respectively corresponding to each type of resource.
8. An electronic device, comprising:
A processor; and
A memory for storing executable instructions of the processor;
Wherein the processor is configured to perform the heterogeneous resource allocation method of any of claims 1-6 via execution of the executable instructions.
9. A computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the heterogeneous resource allocation method of any of claims 1 to 6.
CN202210516665.2A 2022-05-12 Heterogeneous resource allocation method and device, electronic equipment and storage medium Active CN114911618B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210516665.2A CN114911618B (en) 2022-05-12 Heterogeneous resource allocation method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210516665.2A CN114911618B (en) 2022-05-12 Heterogeneous resource allocation method and device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN114911618A CN114911618A (en) 2022-08-16
CN114911618B true CN114911618B (en) 2024-04-26

Family

ID=

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106548262A (en) * 2015-09-21 2017-03-29 阿里巴巴集团控股有限公司 For the dispatching method of the resource of process task, device and system
CN108268317A (en) * 2016-12-30 2018-07-10 华为技术有限公司 A kind of resource allocation methods and device
CN111045795A (en) * 2018-10-11 2020-04-21 浙江宇视科技有限公司 Resource scheduling method and device
CN111796933A (en) * 2020-06-28 2020-10-20 北京小米松果电子有限公司 Resource scheduling method, device, storage medium and electronic equipment
CN113157379A (en) * 2020-01-22 2021-07-23 株式会社日立制作所 Cluster node resource scheduling method and device
KR20220011063A (en) * 2020-07-20 2022-01-27 베이징 바이두 넷컴 사이언스 앤 테크놀로지 코., 엘티디. Server resource allocation method, device, electronic equipment and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106548262A (en) * 2015-09-21 2017-03-29 阿里巴巴集团控股有限公司 For the dispatching method of the resource of process task, device and system
CN108268317A (en) * 2016-12-30 2018-07-10 华为技术有限公司 A kind of resource allocation methods and device
CN111045795A (en) * 2018-10-11 2020-04-21 浙江宇视科技有限公司 Resource scheduling method and device
CN113157379A (en) * 2020-01-22 2021-07-23 株式会社日立制作所 Cluster node resource scheduling method and device
CN111796933A (en) * 2020-06-28 2020-10-20 北京小米松果电子有限公司 Resource scheduling method, device, storage medium and electronic equipment
KR20220011063A (en) * 2020-07-20 2022-01-27 베이징 바이두 넷컴 사이언스 앤 테크놀로지 코., 엘티디. Server resource allocation method, device, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
Hoang et al. Optimal admission control policy for mobile cloud computing hotspot with cloudlet
US9317336B2 (en) Method and apparatus for assignment of virtual resources within a cloud environment
US8112524B2 (en) Recommending moving resources in a partitioned computer
CN107070709B (en) NFV (network function virtualization) implementation method based on bottom NUMA (non uniform memory Access) perception
CN110166507B (en) Multi-resource scheduling method and device
CN109743751B (en) Resource allocation method and device for wireless access network
CN110493317B (en) Method for processing cloud platform resource fragments and related equipment
Li et al. PageRankVM: A pagerank based algorithm with anti-collocation constraints for virtual machine placement in cloud datacenters
CN111625339A (en) Cluster resource scheduling method, device, medium and computing equipment
JP2023543744A (en) Resource scheduling method, system, electronic device and computer readable storage medium
CN114116173A (en) Method, device and system for dynamically adjusting task allocation
CN109783236A (en) Method and apparatus for output information
CN115705247A (en) Process running method and related equipment
CN114911618B (en) Heterogeneous resource allocation method and device, electronic equipment and storage medium
WO2023082914A1 (en) Resource allocation method and apparatus, readable medium, and electronic device
CN116204302A (en) Equipment scheduling method based on K8S An Zhuoyun mobile phone
CN114911618A (en) Heterogeneous resource allocation method and device, electronic equipment and storage medium
CN115378879A (en) Data control method and related device
US11307889B2 (en) Schedule virtual machines
CN107977270A (en) Peers distribution method, peers distribution system and computer installation
CN114090234A (en) Request scheduling method and device, electronic equipment and storage medium
KR102064466B1 (en) Method for allocationing virtual desktop in virtualization system and virtualization system thereof
CN112231096A (en) Method, system, equipment and medium for task balancing of FPGA (field programmable Gate array) pooled resources
CN112181604B (en) Capacity-shrinking target selection method and device
CN111338782A (en) Node allocation method based on competition perception and oriented to shared burst data caching

Legal Events

Date Code Title Description
PB01 Publication
SE01 Entry into force of request for substantive examination
GR01 Patent grant