WO2017202108A1

WO2017202108A1 - Resource allocation method and apparatus for distributed system

Info

Publication number: WO2017202108A1
Application number: PCT/CN2017/076347
Authority: WO
Inventors: 杨磊
Original assignee: 中兴通讯股份有限公司
Priority date: 2016-05-23
Filing date: 2017-03-10
Publication date: 2017-11-30
Also published as: CN107423129A

Abstract

A resource allocation method and apparatus for a distributed system. An equivalent quantization relationship of various resources occupied by each process to be allocated is obtained; then standard calculation is performed on resources occupied by each process allocated to each calculation unit in various initial allocation schemes; at last, an initial allocation scheme satisfying a preset requirement is selected as a final allocation scheme according to calculation results. In the resource allocation method of a distributed system, various different resources occupied by a process can be uniformly quantized by using an equivalent quantization relationship of the resources, so that calculation and comparison conditions exist between the different resources. Based on this, an optimal resource allocation scheme can be selected according to a preset requirement, thereby implementing optimized resource allocation of a distributed system and improving the resource utilization.

Description

Resource configuration method and device for distributed system

Technical field

The present disclosure relates to the field of computer technologies, and in particular, to a resource configuration method and apparatus for a distributed system.

Background technique

Due to the increasing number of network management software in the communication system, the system needs to process more and more demands and responses. In order to improve the processing efficiency and enhance the user experience, the original process can be allocated to different hardware servers for operation. A distributed system is formed by distributed transformation of the communication system. A distributed system is a software system built on top of a network. It is a computer system composed of a plurality of interconnected computing units, wherein each computing unit performs the same task in cooperation under the control of the entire system, and can achieve at least a centralized program, data or hardware-dependent effect. These computing units may be geographically adjacent or geographically dispersed. The common features implied by distributed systems are: site distribution, data distribution, diversification of hardware platforms, diversification of operating systems, and diversification of application platforms.

But how can the processes that need to be run be reasonably allocated to each computing unit to achieve optimal resource allocation? The various resource optimization configurations proposed in the prior art are either for non-distributed systems or are solely for a certain aspect. For example, the resource configuration is performed separately according to the CPU or memory occupied by each process, or the network response model is separately. But in fact, it is unreasonable to use a solution for resource allocation for a certain resource in a distributed system, because each process has its own uniqueness. For example, some processes require a large amount of memory when they are running. To do data caching, but the demand for network bandwidth is not high; some processes cover complex algorithms, need to consume CPU resources to perform a large number of calculations, but almost no need to use the disk IO in the computing unit; The process needs to interact with the server frequently during the running process. Therefore, the network bandwidth requirements are very high.

Due to the different operational requirements of each process, it is difficult to reasonably allocate the above several different processes according to the existing resource optimization configuration scheme. Therefore, there is a lack of a resource optimization configuration scheme for comprehensively evaluating whether it is reasonable to allocate the process to a computing unit according to various resources occupied by a process, and to propose a better allocation scheme.

Summary of the invention

The main technical problem to be solved by the present disclosure is to provide a resource optimization configuration solution, which is used to solve the problem that the resource allocation in the prior art can only be performed for a certain resource and cannot be distributed according to multiple resources. Technical problems with unreasonable system resource allocation.

To solve the above technical problem, the present disclosure provides a resource configuration method for a distributed system, including:

Obtaining an equivalent quantitative relationship of each resource, where the equivalent quantization relationship is a quantification of each of the resources into a standard resource Change relationship

Obtaining a preliminary configuration scheme of each current configuration process on each computing unit;

And calculating, according to the equivalent quantization relationship, a planned occupied resource of each of the computing units in each of the preliminary configuration scenarios, where the planned occupied resource is occupied by the to-be-configured process to which the computing unit is to be allocated Standard resources

And selecting, according to the calculation result, a preliminary configuration scheme that the planned occupied resource of each of the computing units meets a preset condition is used as a destination configuration scheme.

In an embodiment of the present disclosure, the standard resource is any one of the resources or any one of the resources.

In an embodiment of the present disclosure, the preliminary configuration scheme of acquiring the current to-be-configured processes on each computing unit includes:

Determining, according to a preset requirement, a number of units of the computing unit and a specific process, where the specific process is a process to be configured in advance to be determined by the associated computing unit;

Configuring the particular process to the computing unit to which it belongs;

The to-be-configured processes other than the specific process are randomly configured to each of the computing units.

In an embodiment of the present disclosure, the manner of calculating, according to the equivalent quantization relationship, the planned occupied resources of each of the computing units in each of the preliminary configuration scenarios includes:

Calculating, according to the equivalent quantization relationship, standard resources that are required to be occupied when each of the to-be-configured processes is running;

Calculating, by each of the preliminary configuration schemes, a standard resource occupied by each of the computing units by the to-be-configured process;

or,

Calculating actual resources occupied by each of the computing units in each of the preliminary configuration schemes by each of the to-be-configured processes;

Calculating, according to the equivalent quantization relationship, standard resources occupied by each of the computing units in each of the preliminary configuration schemes by each of the to-be-configured processes.

In an embodiment of the present disclosure, the preliminary configuration scheme that selects the pseudo-occupied resource of each of the computing units to meet a preset condition according to the calculation result includes:

The preliminary configuration scheme of the most averaging resources of each of the computing units is selected as the destination configuration scheme.

In an embodiment of the present disclosure, the preliminary configuration scheme for selecting the most averaging of the resources to be occupied by each of the computing units as the destination configuration solution includes:

Calculating a variance of the planned occupied resources of each of the computing units in each of the preliminary configuration scenarios;

Selecting the preliminary configuration scheme with the smallest variance value as the destination configuration scheme;

or,

Calculating a difference between the two occupied resources of each of the computing units in each of the preliminary configuration schemes;

Selecting a maximum difference between the resources to be occupied in the preliminary configuration scheme as a characterization value to characterize the preliminary configuration scheme;

The preliminary configuration scheme in which the characterization value is the smallest is selected as the destination configuration scheme.

The present disclosure also provides a resource configuration apparatus for a distributed system, including:

The relationship obtaining module is configured to obtain an equivalent quantization relationship of each resource, where the equivalent quantization relationship is a conversion relationship that quantizes each of the resources into a standard resource;

Initially determining a module, configured to obtain a preliminary configuration scheme of each current configuration process on each computing unit;

An occupancy calculation module, configured to calculate, according to the equivalent quantization relationship, a planned occupied resource of each of the computing units in each of the preliminary configuration scenarios, where the planned occupied resource is the one to which the computing unit is to be dispatched Standard resources occupied by the process to be configured;

The solution selection module is configured to select, according to the calculation result, a preliminary configuration scheme that the planned occupied resource of each of the computing units meets a preset condition as a destination configuration scheme.

In an embodiment of the present disclosure, the occupancy calculation module includes at least one of the following:

a first calculation module, configured to calculate, according to the equivalent quantization relationship, a standard resource that is required to be used when each of the to-be-configured processes is running;

a second calculating module, configured to calculate a standard resource occupied by each of the computing units in each of the preliminary configuration schemes by the to-be-configured process;

or,

a third calculating module, configured to calculate an actual resource occupied by each of the computing units in each of the preliminary configuration schemes by each of the to-be-configured processes;

And a fourth calculating module, configured to calculate, according to the equivalent quantization relationship, a standard resource occupied by each of the computing units in each of the preliminary configuration scenarios by each of the to-be-configured processes.

In an embodiment of the present disclosure, the solution selection module includes:

The uniform selection module is configured to select a preliminary configuration scheme of the most averaging resources of each of the computing units as the destination configuration scheme.

In an embodiment of the present disclosure, the uniform selection module includes:

a fifth calculating module, configured to calculate a variance of the planned occupied resources of each of the computing units in each of the preliminary configuration scenarios;

a variance selection module, configured to select the preliminary configuration scheme with the smallest variance value as the destination configuration scheme;

or,

a sixth calculation module, configured to calculate a difference between the two occupied resources of each of the computing units in each of the preliminary configuration scenarios;

Characterizing a selection module, configured to select a maximum difference between the resources to be occupied in the preliminary configuration scheme as a characterization value to characterize the preliminary configuration scheme;

The most value selection module is configured to select the preliminary configuration scheme in which the representation value is the smallest as the destination configuration scheme.

The beneficial effects of the present disclosure are:

The resource configuration method of the distributed system proposed by the present disclosure obtains various resources occupied by each to-be-configured process. The equivalent quantitative relationship, and then normalize the resources occupied by the processes allocated to each of the computing units in the various preliminary configuration schemes, and finally select a preliminary configuration scheme that meets the preset requirements according to the calculation result as the final configuration. Program. In the resource allocation method of the distributed system proposed by the present disclosure, the various quantitative resources occupied by the process can be uniformly quantized by the equivalent quantitative relationship of each resource, so that different resources have the conditions for calculation and comparison. On this basis, the optimal resource configuration scheme can be selected according to the preset requirements, the resource optimization configuration of the distributed system is realized, and the resource utilization rate is improved.

DRAWINGS

FIG. 1 is a flowchart of a resource configuration method of a distributed system according to Embodiment 1 of the present disclosure;

2 is a flowchart of obtaining a preliminary configuration scheme according to Embodiment 1 of the present disclosure;

FIG. 3 is a flowchart of a selection destination configuration scheme according to Embodiment 1 of the present disclosure;

FIG. 4 is another flowchart of a selection destination configuration scheme according to Embodiment 1 of the present disclosure;

FIG. 5 is a schematic structural diagram of a resource configuration apparatus of a distributed system according to Embodiment 2 of the present disclosure;

6 is a schematic structural view of the preliminary determining module of FIG. 5;

7 is a schematic structural view of the occupancy calculation module of FIG. 5;

8 is another schematic structural diagram of the occupancy calculation module of FIG. 5;

9 is a schematic structural view of the scheme selection module of FIG. 5;

10 is a schematic structural view of the uniform selection module of FIG. 9;

FIG. 11 is another schematic structural view of the uniform selection module of FIG. 9.

detailed description

The present disclosure will be further described in detail below with reference to the accompanying drawings.

Embodiment 1:

This embodiment provides a resource configuration method for a distributed system. Referring to FIG. 1, the method includes:

S101. Acquire an equivalent quantitative relationship of each resource occupied by each to-be-configured process.

When a process runs on a computing unit, it consumes resources such as CPU, memory, hard disk, and network bandwidth of the computing unit. Usually, the CPU uses core ("core") as the unit of measurement, the unit of memory is GB (gigabytes), and the unit of hard disk I/O (Input/Output, input/output) is MB/s (megabits per second). ), while network bandwidth is measured in bps. These kinds of resources have different units of measurement because of their different natures. Therefore, in the usual sense, there is no way to put them together for comparison and comparison. This is why there is no resource allocation scheme in the prior art, which can fully consider all aspects of resources consumed by the process. For example, a process a runs on computing unit A, which requires CPUs of A computing unit 2core and 64 GB of memory, and hardly occupies hard disk and network bandwidth. Process b runs on computing unit B. Since the algorithm is not very complicated, the CPU requirements are very low, but it requires 32 GB of memory and 100 MB/s of hard disk I/O. According to the prior art scheme, it is difficult to judge. Which of the processes a and b consumes more computing units, too It is difficult to say whether the resource usage of the computing unit A is larger or the resource usage of the computing unit B is larger. Just as there are two metal blocks of A and B, the metal block weighs 0.5 kilograms and the volume is 3 cubic centimeters. The weight of B is 0.2 kilograms and the volume is 10 cubic centimeters. It is difficult to compare which metal block is more valuable from this information. Or maybe there is an A project, which is to build a bridge, and the B project is to lay a road. Because the construction of the bridge and the geographical environment are very different, we can’t simply take the bridge. The length and width of the roads indicate which of the two projects is more ambitious.

In this embodiment, various different resources can be converted into a standard resource according to the equivalent quantization relationship, so that various different resources can have the basis for comparison. The equivalent quantitative relationship is to convert each resource into a standard resource conversion relationship, which is equivalent to comparing the items of A and B according to the cost of the project. For example, the capital cost of a project is 1 million, and the funds of B are The consumption is 870,000. When the two projects of A and B are uniformly measured according to the capital expenditure, the size of the project can be compared, which is equivalent to a project with a capital cost of 1 million and a capital cost of 870,000. At the time of the project, it was obvious that the A project was even bigger. The conversion relationship between the bridge repairing project and the road construction project to the fund is equivalent to the equivalent quantitative relationship in the present embodiment.

In this embodiment, the equivalent quantization relationship can be customized by the user. Users can set the conversion relationship between CPU, memory, hard disk and network bandwidth according to their own considerations. For example, the user can determine the equivalent relationship from the aspect of capital consumption, if you extend a 2core CPU and expand a 64GB memory. As much as you need, you can think of a 2core CPU equivalent to 64GB of memory. Similarly, if the network bandwidth of 1000Mbps is the same as the price of the CPU of the extended 2core, the network bandwidth of 1000Mbps can be equivalent to the CPU of 2core, and the hard disk I/O of 100MB/s can be equivalent to the memory of 64GB, as long as it expands. 100MB/s of hard disk I/O costs as much as 64GB of expanded memory.

The standard resource is the basis for realizing the comparison of various types of resources, and the standard resource may be any one other than the resource that needs to be quantified by the standard. For example, in the above example, “bridge repair project” and “road construction project” can be uniformly converted into “funding cost”. At this time, “funding cost” belongs to one other than “bridge repair project” and “bridge repair project”. Kind of, "funding expenses" can also be replaced by "manpower consumption" and the like. In this embodiment, the standard resource may be any one of the CPU, the memory, the hard disk I/O, and the network bandwidth that needs to be quantized and unified, for example, the user customizes a “virtual resource”, for example, a user-defined “2core”. CPU=64GB memory=100MB/s hard disk I/O=1000Mbps network bandwidth=1v”, where 1v is a virtual resource that can convert CPU, memory, hard disk I/O and network bandwidth into this The virtual resources are then compared.

In this embodiment, the process is represented by "process=[CPU, memory, hard disk I/O, network bandwidth]", and it is assumed that the current process P requires 3 core CPU, 64 GB of memory, and 1000 MB/s hard disk. I / O and 500Mbps network bandwidth, then the process P = [3croe, 64GB, 1000MB / s, 500Mbps], if the standard resource is a virtual resource, then the process P = [1.5v, 1v, 10v, 0.5v].

It can be understood that the standard resource can also be any one of the resources that need to be quantified by the standard. For example, if the CPU is directly used as a standard resource, then the process P=[3croe, 2core, 20core, 1core], which is equivalent to The “bridge repair project” and “road construction project” will be uniformly converted into any one of “bridge repair project” or “bridge repair project”. For example, from the perspective of capital consumption, “bridge repair project” is equal to One "bridge construction project" and "road construction project" can be equal to 0.87 "bridge repair projects".

In the above embodiment, the user sets the equivalent quantitative relationship according to the capital expenditure of expanding various resources. It can be understood that the embodiment does not limit the determination manner of the equivalent quantitative relationship, because the user can according to his subjective will. Make any form of setting the equivalent quantization relationship. Of course, it is also feasible to set the equivalent quantitative relationship according to other factors than the capital consumption. For example, some resources are more difficult to expand, and it takes more time for the user, and some resources may be easier to expand. Obviously, the user can also set the equivalent quantization relationship according to the difficulty of expanding various resources.

S102. Acquire a preliminary configuration scheme of each current configuration process on each computing unit.

The initial configuration scheme generally includes multiple types. Since the initial configuration scheme only meets the most basic configuration requirements, there are many initial configuration schemes in which the process allocation is unreasonable, and the final destination configuration scheme needs to be filtered. .

In the following, this embodiment proposes a method for obtaining a preliminary configuration scheme, please refer to FIG. 2:

S201. Determine a number of units of the computing unit and a specific process according to a preset requirement.

Assume that the current default requirement is to configure the five processes A, B, C, D, and E to the three computing units A, B, and C of the distributed system. Then, the preliminary configuration scheme can have

Kind, that is, 60 species. In the actual application process, there are specific requirements for the configuration of some processes. This process is a specific process, and a specific process is a process that predetermines the owning computing unit. For example, in a distributed system, it is explicitly required to assign process B to a computing unit, then process B is a specific process.

S202. Configure a specific process to a computing unit to which it belongs.

According to the preset requirement, the specific process B is allocated to the computing unit A of the distributed system. Since the allocation scheme of the process B has been determined, there are no more than 60 preliminary configuration schemes that can be formed.

S203. Randomly configure a to-be-configured process other than a specific process to each computing unit.

In addition to process B, there are four processes A, C, D, and E that need to be assigned to three computing units. The four processes A, C, D, and E can be randomly assigned to A and B. And C on the three calculation units. At this time, the preliminary configuration that can be formed is

Kind, that is, 24 kinds.

S103. Calculate, according to the equivalent quantization relationship, the planned occupied resources of each computing unit in each preliminary configuration scheme.

Since each process in the preliminary configuration scheme is only intended to be configured on the corresponding computing unit, but has not been configured in practice, the standard resources occupied by the processes to which the computing units are configured in the preliminary configuration scheme are calculated. It is also only to calculate the resources that are occupied in the assumed situation. It is assumed that the occupied resources in the case are intended to occupy resources, and the resources to be occupied are the standard resources occupied by the to-be-configured process to which the computing unit is to be allocated. There are at least two ways to calculate the resources to be occupied:

The first type firstly converts the resources to be used in the running process of the to-be-configured process into standard resources according to the equivalent quantization relationship, and then calculates the processes occupied by the processes to which the computing units are to be dispatched according to the converted standard resources. Standard resources.

Secondly, the real resources occupied by the processes to which the computing units are to be dispatched are first calculated, and then the calculated results of the obtained real resources are converted into standard resources according to the equivalent quantitative relationship.

To put it simply, the first solution is to use the process as the computing object, to quantify the resources, convert them into standard resources, and then calculate the occupied standard resources by using the computing unit as the object. The calculation method of the second calculation method is just the opposite. The calculation unit calculates the resource occupancy first, and then converts the real resources into standard resources according to the equivalent quantization relationship. It should be understood by those skilled in the art that although the two calculation methods are different in order, the calculated results should be the same. For example, a computing unit has three processes A, B, and C, P _A = [3core, 64GB, 1000MB/s, 500Mbps], P _B = [2core, 64GB, 1000MB/s, 1000Mbps], P _C = [2core , 128GB, 550MB/s, 500Mbps], the equivalent quantization relationship is "2core CPU = 64GB of memory = 100MB / s of hard disk I / O = 1000Mbps network bandwidth = 1v", in order to obtain the intended resources of the computing unit You can use the two calculation methods shown below:

The first is to standardize the resources occupied by each process at the first run:

P _A = [3core, 64GB, 1000MB/s, 500Mbps] = [1.5v, 1v, 10v, 0.5v];

P _B = [2core, 64GB, 1000MB/s, 1000Mbps] = [1v, 1v, 10v, 1v];

P _C = [2core, 128 GB, 550 MB/s, 500 Mbps] = [1v, 2v, 5.5v, 0.5v].

Then calculate the pseudo-resource occupancy of the computing unit by using the computing unit as an object:

P _mark = [3.5v, 4v, 25.5v, 2v].

The second way is to first calculate the real resources occupied by the computing unit by processes A, B, and C:

P _true = [7core, 256GB, 2550MB / s, 2000Mbps];

Then, according to the equivalent quantization relationship - "2core CPU=64GB memory=100MB/s hard disk I/O=1000Mbps network bandwidth=1v", the calculated occupied resources of the computing unit are calculated:

P _mark = [(7core/2core) * v, (256GB / 64GB) * v, ((2550MB / s) / (100MB / s)) * v, (2000Mbps / 1000Mbps) * v] = [3.5v, 4v , 25.5v, 2v].

S104. Select, according to the calculation result, a preliminary configuration scheme that the planned occupied resources of each computing unit meet the preset condition as the destination configuration scheme.

After the initial configuration schemes are calculated in the foregoing steps, at least one scheme may be selected from the multiple schemes as the destination configuration scheme according to the calculation result. Since the hardware resources of the computing units in the distributed system are generally the same, the more even the allocation means the less resources are available, so the more optimized, the description in natural language is: assigned to different computing units. The difference in the resources occupied by the process is the smallest, that is, the scheme that the output value of the planned occupied resources of each computing unit is the smallest is an ideal resource allocation scheme. Therefore, in this embodiment, a preliminary configuration scheme in which the calculation unit is most averaging the resources to be occupied is selected as the destination configuration scheme.

How to determine whether the estimated resources occupied by each computing unit are average? An example of this embodiment provides an option, please refer to FIG. 3:

S301. Calculate a difference between the two occupied resources of each computing unit in the preliminary configuration scheme, for example, obtain one The planned occupied resources of the three computing units in the initial configuration scheme are:

P = ₁ = [3.5v, 4v, 20v, 2v], P ₂ = [2.5v, 6v, 10v, 4v], P ₃ = [3v, 2v, 5v, 4v].

The difference between the two occupied resources of the three computing units is:

ΔB _1-2 =|P _{mark 1} -P _{mark 2} |=[1v,2v,10v,2v]=15v;

ΔB _2-3 =|P _{mark 2} -P _{mark 3} |=[0.5v,4v,5v,0v]=9.5v;

ΔB _1-3 =|P _{is 1} -P _{is 3} |=[0.5v, 2v, 15v, 2v] = 19.5v.

S302. Select a maximum difference between the resources to be occupied in the preliminary configuration scheme as a characterization value to represent the initial configuration scheme.

In the above example, 19.5v should be selected as the characterization value to characterize the preliminary configuration scheme.

S303. Select a preliminary configuration scheme with the smallest characterization value as the destination configuration scheme.

Assuming that there are three preliminary configuration schemes, the characterization values are 16v, 17v, 19.5v, respectively, then according to the scheme of the present disclosure, the first preliminary configuration scheme should be selected as the destination configuration scheme to configure each computing unit.

It is noted that if the method of the configuration shown in FIG. 3 is used to select the destination configuration scheme, the standard corresponding to each resource of the process should not be used until the two-two difference between the planned occupied resources of each computing unit is calculated. The resource combination calculation, for example, cannot calculate the above P _{standard 1} as 29.5v. Because the difference between the two pairs is an absolute difference, if the standard resources corresponding to the resources are combined in advance, the calculation results of the two preliminary configuration schemes are similar, but the actual occupied resources are extremely unbalanced.

Since the variance has the ability to represent whether the distribution of the respective values is averaged, in another preferred example provided by the embodiment, the variance between the pseudo-occupied resources of each computing unit may be selected, and then the calculation result of the variance is selected according to the calculation result of the variance. Destination configuration scheme. The second way to select the destination configuration scheme is shown in Figure 4:

S401. Calculate a variance of a planned occupied resource of each computing unit in each preliminary configuration scheme.

For example, the proposed occupied resources of three computing units in a preliminary configuration scheme are:

Then, P _{is 1} = 29.5v, P _{is 2} = 22.5v, and P _{is 3} = 14v. Three unit calculates an average value for the proposed resource intensive, _P = 22v, the variance value of 40.17 can be obtained according to the formula of the variance.

S402. The initial configuration scheme with the smallest variance value is selected as the destination configuration scheme.

The smaller the variance, the more average the resources to be occupied by each computing unit, and the more optimized the resource allocation, that is, the more ideal the initial configuration scheme.

The resource allocation method of the distributed system provided in this embodiment can uniformly quantize various resources occupied by the process by using the equivalent quantization relationship of each resource, so that different resources have calculation and comparison between them. Conditions, on this basis, the optimal resource allocation scheme can be selected according to the preset requirements, the resource optimization configuration of the distributed system is realized, and the resource utilization rate is improved.

Embodiment 2:

This embodiment provides a resource configuration apparatus for a distributed system. The resource configuration method of the distributed system provided in Embodiment 1 can be used on the device. Please refer to FIG. 5:

The resource configuration apparatus 50 of the distributed system includes a relationship acquisition module 501, a preliminary determination module 502, an occupancy calculation module 503, and a scheme selection module 504.

The relationship obtaining module 501 is configured to obtain an equivalent quantization relationship of each resource occupied by each to-be-configured process.

When a process runs on a computing unit, it consumes resources such as CPU, memory, hard disk, and network bandwidth of the computing unit. Usually, the CPU uses core ("core") as the unit of measurement, the unit of memory is GB (gigabytes), and the unit of hard disk I/O (Input/Output, input/output) is MB/s (megabits per second). ), while network bandwidth is measured in bps. These kinds of resources have different units of measurement because of their different natures. Therefore, in the usual sense, there is no way to put them together for comparison and comparison. This is why there is no resource allocation scheme in the prior art, which can fully consider all aspects of resources consumed by the process. For example, a process a runs on computing unit A, which requires CPUs of A computing unit 2core and 64 GB of memory, and hardly occupies hard disk and network bandwidth. Process b runs on computing unit B. Since the algorithm is not very complicated, the CPU requirements are very low, but it requires 32 GB of memory and 100 MB/s of hard disk I/O. According to the prior art scheme, it is difficult to judge. Which of the processes a and b consumes more of the computing unit, and it is difficult to say whether the resource occupancy of the computing unit A is larger or the resource occupancy of the computing unit B is larger. Just as there are two metal blocks of A and B, the metal block weighs 0.5 kilograms and the volume is 3 cubic centimeters. The weight of B is 0.2 kilograms and the volume is 10 cubic centimeters. It's hard to compare a piece of metal from this information. Or maybe there is an A project, which is to build a bridge, and the B project is to lay a road. Because the construction of the bridge and the geographical environment are very different, we can’t simply take the bridge. The length and width of the roads indicate which of the two projects is more ambitious.

In this embodiment, the equivalent quantization relationship acquired by the relationship obtaining module 501 can be customized by the user. Users can set the conversion relationship between CPU, memory, hard disk and network bandwidth according to their own considerations. For example, the user can determine the equivalent relationship from the aspect of capital consumption, if you extend a 2core CPU and expand one. A 64GB of memory costs as much, so you can think of a 2core CPU equivalent to 64GB of memory. Similarly, if the network bandwidth of 1000Mbps is the same as the price of the CPU of the extended 2core, the network bandwidth of 1000Mbps can be equivalent to the CPU of 2core, and the hard disk I/O of 100MB/s can be equivalent to the memory of 64GB, as long as it expands. 100MB/s of hard disk I/O costs as much as 64GB of expanded memory.

The standard resource is the basis for realizing the comparison of various types of resources, and the standard resource may be any one other than the resource that needs to be quantified by the standard. For example, in the above example, “bridge repair project” and “road construction project” can be uniformly converted into “funding cost”. At this time, “funding cost” belongs to one other than “bridge repair project” and “bridge repair project”. Kind of, ""funding cost" can also be replaced by "manpower cost", etc. In this embodiment, the standard resource can be any one of the CPU, memory, hard disk I/O, and network bandwidth that need to be quantized and unified. For example, the user customizes a kind of "virtual resource", for example, user-defined "2core CPU=64GB memory=100MB/s hard disk I/O=1000Mbps network bandwidth=1v", where 1v is a virtual resource. The CPU, memory, hard disk I/O, and network bandwidth can be uniformly converted into such virtual resources and then compared.

It can be understood that the standard resource can also be any one of the resources that need to be quantified by the standard. For example, if the CPU is directly used as a standard resource, then the process P=[3croe, 2core, 20core, 1core], which is equivalent to The “bridge repair project” and the “road construction project” are uniformly converted into any one of “bridge repair project” or “bridge repair project”. For example, from the perspective of capital consumption, “bridge repair project” is equal to 1 A "bridge repair project" and "road construction project" can be equal to 0.87 "bridge repair projects".

In the above embodiment, the relationship obtaining module 501 is an equivalent quantitative relationship that is set by the user according to the cost of expanding various resources. It can be understood that the present disclosure does not limit the equivalent quantitative relationship acquired by the relationship obtaining module 501. How is it determined? Because users can make any form of equivalent quantitative relationship according to their subjective wishes. Of course, it is also feasible to set the equivalent quantitative relationship according to other factors than the capital consumption. For example, some resources are more difficult to expand, and it takes more time for the user, and some resources may be easier to expand. Obviously, the user can also set the equivalent quantization relationship according to the difficulty of expanding various resources.

The preliminary determining module 502 obtains a preliminary configuration scheme of each current configuration process on each computing unit.

In the following, this embodiment provides a preliminary determination module 502, please refer to FIG. 6:

The preliminary determination module 502 includes a first sub-module 5021, a second sub-module 5022, and a third sub-module 5023. The first sub-module 5021 is configured to determine a unit number and a specific process of the computing unit according to a preset requirement. It is assumed that the current preset requirement is to configure the five processes A, B, C, D, and E to the three computing units A, B, and C of the distributed system. Then, the preliminary configuration scheme can have

The second sub-module 5022 is configured to configure a particular process onto the computing unit to which it belongs. The second sub-module 5022 can allocate the specific process B to the computing unit A of the distributed system according to the preset requirement. Since the allocation scheme of the process B has been determined, there are no 60 preliminary configuration schemes that can be formed.

The third sub-module 5023 randomly configures the processes to be configured outside the specific process to the respective computing units. In addition to process B, there are four processes A, C, D, and E that need to be assigned to three computing units. The third sub-module 5023 can randomly assign four processes A, C, D, and E. Go to the three calculation units of A, B, and C. At this time, the preliminary configuration that can be formed is

Kind, that is, 24 kinds.

In this embodiment, the occupancy calculation module 503 calculates the planned occupied resources of each computing unit in each preliminary configuration scheme according to the equivalent quantization relationship.

Since each process in the preliminary configuration scheme is only intended to be configured on the corresponding computing unit, but has not been configured in practice, the standard resources occupied by the processes to which the computing units are configured in the preliminary configuration scheme are calculated. It is also only to calculate the resources that are occupied in the assumed situation. It is assumed that the occupied resources in the case are intended to occupy resources, and the resources to be occupied are the standard resources occupied by the to-be-configured process to which the computing unit is to be allocated. The occupancy calculation module 503 can be at least one of the following two types. Please refer to FIG. 7 and FIG. 8 for further reference:

The occupancy calculation module 503 shown in FIG. 7 includes a first calculation module 5031 and a second calculation module 5032. The first calculating module 5031 first converts the resources that need to be configured in the running process to the standard resources according to the equivalent quantization relationship, and then the second calculating module 5032 calculates, according to the converted standard resources, each computing unit is proposed to be The standard resources occupied by the process.

The occupancy calculation module 503 shown in FIG. 8 includes a third calculation module 5033 and a fourth calculation module 5034. The third calculating module 5033 first calculates the real resources occupied by the processes to which the computing units are to be dispatched, and then the fourth calculating module 5034 converts the calculated results of the obtained real resources into standard resources according to the equivalent quantization relationship.

In brief, the occupancy calculation module 503 shown in FIG. 7 is a standard resource that is first occupied by a process by using a process as a calculation object, equivalently quantizing a resource, converting it into a standard resource, and then taking the calculation unit as an object. The calculation process of the occupancy calculation module 503 in FIG. 8 is just the opposite. The calculation unit calculates the resource occupancy first, and then converts the real resource into a standard resource according to the equivalent quantization relationship. It should be understood by those skilled in the art that although the order in which the two occupancy calculation modules perform calculations is different, the calculated results should be the same. For example, a computing unit has three processes A, B, and C, P _A = [3core, 64GB, 1000MB/s, 500Mbps], P _B = [2core, 64GB, 1000MB/s, 1000Mbps], P _C = [2core , 128GB, 550MB/s, 500Mbps], the equivalent quantization relationship is "2core CPU = 64GB of memory = 100MB / s of hard disk I / O = 1000Mbps network bandwidth = 1v". In order to obtain the intended occupied resources of the computing unit, the first type of occupancy calculation module first performs standard quantization on the resources occupied by each process:

P _A = [3core, 64GB, 1000MB/s, 500Mbps] = [1.5v, 1v, 10v, 0.5v];

P _B = [2core, 64GB, 1000MB/s, 1000Mbps] = [1v, 1v, 10v, 1v];

P _C = [2core, 128 GB, 550 MB/s, 500 Mbps] = [1v, 2v, 5.5v, 0.5v].

P _mark = [3.5v, 4v, 25.5v, 2v].

The second occupancy calculation module first calculates the real resources occupied by the calculation unit by processes A, B, and C:

P _true = [7core, 256GB, 2550MB / s, 2000Mbps];

The solution selection module 504 is configured to select, according to the calculation result, a preliminary configuration scheme that each of the computing units whose planned occupied resources meet the preset condition is used as the destination configuration solution.

After the calculation of each preliminary configuration scheme is performed in the foregoing steps, the scheme selection module 504 may select at least one scheme from the multiple schemes as the destination configuration scheme according to the calculation result. Since the hardware resources of the computing units in the distributed system are generally the same, the more even the allocation means the less resources are available, so the more optimized, the description in natural language is: assigned to different computing units. The difference in the resources occupied by the process is the smallest, that is, the scheme that the output value of the planned occupied resources of each computing unit is the smallest is an ideal resource allocation scheme. Therefore, in this embodiment, as shown in FIG. 9, the scheme selection module 504 includes a uniform selection module 5041, and the uniform selection module 5041 is configured to select a preliminary configuration scheme of the calculation unit that is the most averaging of the resources to be used as the destination configuration scheme.

How to determine whether the estimated resources occupied by each computing unit are average? In an example of this embodiment, a uniform selection module is provided, please refer to FIG. 10:

The uniform selection module 5041 includes a sixth calculation module 50411 and a characterization selection module 50412, and a maximum value selection module 50413,

The sixth calculation module 50411 is configured to calculate the difference between the two occupied resources of each computing unit in the preliminary configuration scheme. For example, the estimated occupied resources of the three computing units in a preliminary configuration scheme are:

The sixth calculation module 50411 calculates that the difference between the two occupied resources of the three computing units is:

ΔB _1-2 =|P _{mark 1} -P _{mark 2} |=[1v,2v,10v,2v]=15v;

ΔB _2-3 =|P _{mark 2} -P _{mark 3} |=[0.5v,4v,5v,0v]=9.5v;

ΔB _1-3 =|P _{is 1} -P _{is 3} |=[0.5v, 2v, 15v, 2v] = 19.5v.

The characterization selection module 50412 is configured to select a maximum difference between the resources to be occupied in the preliminary configuration scheme as a characterization value to characterize the preliminary configuration scheme. For example, in the above example, the characterization selection module 50412 should select 19.5v as the characterization value to characterize the preliminary configuration scheme.

The most value selection module 50413 is configured to select a preliminary configuration scheme with the smallest characterization value as the destination configuration scheme. Assuming that there are three preliminary configuration schemes, the characterization values are 16v, 17v, 19.5v, respectively, then according to the solution of the present disclosure, the maximum value selection module 50413 should select the first preliminary configuration scheme as the destination configuration scheme to configure each computing unit.

It should be noted that if the uniform selection module 5041 is as shown in FIG. 10, the standard resources corresponding to the resources of the process should not be combined before calculating the difference between the two resources of the computing units. For example, the above P _{label 1} cannot be calculated as 29.5v. Because the difference between the two pairs is an absolute difference, if the standard resources corresponding to the resources are combined in advance, the calculation results of the two preliminary configuration schemes are similar, but the actual occupied resources are extremely unbalanced.

In the other preferred example provided by this embodiment, the uniform selection module 5041 includes a fifth calculation module 50414 and a variance selection module 50415. The uniform selection module 5041 can calculate a variance between the pseudo-occupied resources of each computing unit, and then select a destination configuration scheme according to the calculation result of the variance.

The fifth calculating module 50414 is configured to calculate a variance of the planned occupied resources of each computing unit in each preliminary configuration scheme. For example, the proposed occupied resources of three computing units in a preliminary configuration scheme are:

Then, P _{is 1} = 29.5v, P _{is 2} = 22.5v, and P _{is 3} = 14v. Three unit calculates an average value for the proposed resource intensive, _P = 22v, the fifth block 50414 calculates the variance value of 40.17 can be obtained according to the formula of the variance.

The variance selection module 50415 selects a preliminary configuration scheme in which the variance value is the smallest according to the calculation result of the fifth calculation module 50414 as the destination configuration scheme. The smaller the variance, the more average the resources to be occupied by each computing unit, and the more optimized the resource allocation, that is, the more ideal the initial configuration scheme.

The resource configuration apparatus of the distributed system provided by the embodiment can uniformly quantize various resources occupied by the process by using the equivalent quantization relationship of each resource, so that different resources have conditions for calculation and comparison. Based on the preset requirements, the optimal resource configuration scheme can be selected to optimize resource allocation of the distributed system and improve resource utilization.

Obviously, those skilled in the art should understand that the above modules or steps of the present disclosure can be implemented by a general computing device, which can be concentrated on a single computing device or distributed over a network composed of multiple computing devices. Alternatively, they may be implemented by program code executable by the computing device, such that They are stored in a storage medium (ROM/RAM, disk, optical disk) by a computing device, and in some cases, the steps shown or described may be performed in a different order than here, or they may be separately It is made by making individual integrated circuit modules, or by making a plurality of modules or steps of them into a single integrated circuit module. Therefore, the present disclosure is not limited to any specific combination of hardware and software.

The above content is a further detailed description of the present disclosure in conjunction with the specific embodiments, and the specific implementation of the present disclosure is not limited to the description. It is to be understood by those skilled in the art that the present invention may be construed as being limited to the scope of the present disclosure.

Industrial applicability

The present disclosure is applicable to the field of computer technology, and is used to uniformly quantify various resources occupied by processes through the equivalent quantitative relationship of resources, so that different resources have conditions for calculation and comparison, and on this basis, According to the preset requirements, the optimal resource configuration scheme is selected to implement resource optimization configuration of the distributed system and improve resource utilization.

Claims

A resource configuration method for a distributed system includes:

Obtaining an equivalent quantitative relationship of each resource, where the equivalent quantization relationship is a conversion relationship that quantizes each of the resources into a standard resource;

Obtaining a preliminary configuration scheme of each current configuration process on each computing unit;

And calculating, according to the equivalent quantization relationship, a planned occupied resource of each of the computing units in each of the preliminary configuration scenarios, where the planned occupied resource is occupied by the to-be-configured process to which the computing unit is to be allocated Standard resources

And selecting, according to the calculation result, a preliminary configuration scheme that the planned occupied resource of each of the computing units meets a preset condition is used as a destination configuration scheme.
The resource configuration method of a distributed system according to claim 1, wherein the standard resource is any one of the resources or any one of the resources.
The resource configuration method of the distributed system according to claim 1, wherein the obtaining a preliminary configuration scheme of each current configuration process on each computing unit comprises:

Determining, according to a preset requirement, a number of units of the computing unit and a specific process, where the specific process is a process to be configured in advance to be determined by the associated computing unit;

Configuring the particular process to the computing unit to which it belongs;

The to-be-configured processes other than the specific process are randomly configured to each of the computing units.
The method for configuring a resource of a distributed system according to claim 1, wherein the manner of calculating the planned occupied resources of each of the computing units in each of the preliminary configuration scenarios according to the equivalent quantization relationship comprises:

Calculating, according to the equivalent quantization relationship, standard resources that are required to be occupied when each of the to-be-configured processes is running;

Calculating, by each of the preliminary configuration schemes, a standard resource occupied by each of the computing units by the to-be-configured process;

or,

Calculating actual resources occupied by each of the computing units in each of the preliminary configuration schemes by each of the to-be-configured processes;

Calculating, according to the equivalent quantization relationship, standard resources occupied by each of the computing units in each of the preliminary configuration schemes by each of the to-be-configured processes.
The resource configuration method of the distributed system according to any one of claims 1 to 4, wherein the preliminary configuration scheme in which the pseudo-occupied resources of each of the computing units meet a preset condition is selected as a destination configuration according to a calculation result. The plan includes:

The preliminary configuration scheme of the most averaging resources of each of the computing units is selected as the destination configuration scheme.
The method for configuring a resource of a distributed system according to claim 5, wherein the preliminary configuration scheme of selecting the most-occupied resource of each of the computing units as the destination configuration scheme comprises:

Calculating a variance of the planned occupied resources of each of the computing units in each of the preliminary configuration scenarios;

Selecting the preliminary configuration scheme with the smallest variance value as the destination configuration scheme;

or,

Calculating a difference between the two occupied resources of each of the computing units in each of the preliminary configuration schemes;

Selecting a maximum difference between the resources to be occupied in the preliminary configuration scheme as a characterization value to characterize the preliminary configuration scheme;

The preliminary configuration scheme in which the characterization value is the smallest is selected as the destination configuration scheme.
A resource configuration device for a distributed system, comprising:

The relationship obtaining module is configured to obtain an equivalent quantization relationship of each resource, where the equivalent quantization relationship is a conversion relationship that quantizes each of the resources into a standard resource;

Initially determining a module, configured to obtain a preliminary configuration scheme of each current configuration process on each computing unit;

An occupancy calculation module, configured to calculate, according to the equivalent quantization relationship, a planned occupied resource of each of the computing units in each of the preliminary configuration scenarios, where the planned occupied resource is the one to which the computing unit is to be dispatched Standard resources occupied by the process to be configured;

The solution selection module is configured to select, according to the calculation result, a preliminary configuration scheme that the planned occupied resource of each of the computing units meets a preset condition as a destination configuration scheme.
The resource configuration apparatus of a distributed system according to claim 7, wherein the occupancy calculation module comprises at least one of the following:

a first calculation module, configured to calculate, according to the equivalent quantization relationship, a standard resource that is required to be used when each of the to-be-configured processes is running;

a second calculating module, configured to calculate a standard resource occupied by each of the computing units in each of the preliminary configuration schemes by the to-be-configured process;

or,

a third calculating module, configured to calculate an actual resource occupied by each of the computing units in each of the preliminary configuration schemes by each of the to-be-configured processes;

And a fourth calculating module, configured to calculate, according to the equivalent quantization relationship, a standard resource occupied by each of the computing units in each of the preliminary configuration scenarios by each of the to-be-configured processes.
The resource configuration apparatus of the distributed system according to claim 7 or 8, wherein the scheme selection module comprises:

The uniform selection module is configured to select a preliminary configuration scheme of the most averaging resources of each of the computing units as the destination configuration scheme.
The resource configuration apparatus of a distributed system according to claim 9, wherein the uniform selection module comprises:

a fifth calculating module, configured to calculate the proposed occupancy of each of the computing units in each of the preliminary configuration scenarios The variance of the resources used;

a variance selection module, configured to select the preliminary configuration scheme with the smallest variance value as the destination configuration scheme;

or,

a sixth calculation module, configured to calculate a difference between the two occupied resources of each of the computing units in each of the preliminary configuration scenarios;

Characterizing a selection module, configured to select a maximum difference between the resources to be occupied in the preliminary configuration scheme as a characterization value to characterize the preliminary configuration scheme;

The most value selection module is configured to select the preliminary configuration scheme in which the representation value is the smallest as the destination configuration scheme.
A computer storage medium storing execution instructions for performing the method of any one of claims 1 to 6.