CN106506594B - A kind of concurrent computation resource distribution method and device - Google Patents
A kind of concurrent computation resource distribution method and device Download PDFInfo
- Publication number
- CN106506594B CN106506594B CN201610875024.0A CN201610875024A CN106506594B CN 106506594 B CN106506594 B CN 106506594B CN 201610875024 A CN201610875024 A CN 201610875024A CN 106506594 B CN106506594 B CN 106506594B
- Authority
- CN
- China
- Prior art keywords
- resource
- cost
- clustered node
- node
- resource allocation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
Abstract
The invention discloses a kind of concurrent computation resource distribution method and devices, this method comprises: receiving Current resource distribution request, obtain the total number resource of the current resource allocation request；Obtain the resource related information that each clustered node includes in parallel computing trunking；According to the resource related information of each clustered node, the resource allocation cost of each clustered node is calculated；Resource allocation is carried out to the corresponding request task of the current resource allocation request according to the total number resource of the resource allocation cost of each clustered node and the current resource allocation request, is passed through, the present invention can effectively improve the utilization rate of entire cluster resource.
Description
Technical field
The present invention relates to cluster resource management and parallel computation fields, more particularly to a kind of concurrent computation resource distribution side
Method and device.
Background technique
In recent years, parallel computing trunking is increasingly used in speech recognition, image recognition, natural language understanding et al.
Work intelligence related fields, so that the task processing capacity in the field is obviously improved, the deep learning in the especially described field
The processing capacity of task gets a promotion significantly；And with the increase of data processing amount, single resource can not often carry complicated
Calculating task needs more and more resources in parallel computing trunking to execute corresponding task, thus to meet practical application need
It asks, such as forms the depth that a GPU cluster executes mass data using multiple GPU (Graphic Processing Unit, GPU)
The calculating demand in learning tasks is spent, this just needs efficiently to distribute cluster resource, to give full play to its parallel computation
Ability.
Existing parallel computing trunking resource allocation methods be usually using resource in cluster as CPU, memory, hard disk except
A kind of attached resource simply distributed, clustered node carry out resource allocation when, generally to collection by the way of mutual exclusion, exclusive
Resource is allocated in group, that is, when idling-resource meets mission requirements in clustered node, is then allocated, allocated resource
Exclusive resource resource will be discharged after the completion of the task always, however, this resource allocation methods will cause task dispatching when
Between it is longer, cluster parallel processing capability decline and resource utilization it is lower the problems such as.
Summary of the invention
In order to overcome the deficiencies of the above existing technologies, purpose of the present invention is to provide a kind of distribution of concurrent computation resource
Method and device, to improve the utilization rate of cluster resource.
In order to achieve the above object, the present invention proposes a kind of concurrent computation resource distribution method, include the following steps:
Step 1 receives Current resource distribution request, obtains the total number resource of the current resource allocation request；
Step 2 obtains the resource related information that each clustered node includes in parallel computing trunking；
Step 3 calculates the resource allocation cost of each clustered node according to the resource related information of each clustered node；
Step 4, according to the resource allocation cost of each clustered node and the total number resource to the current resource allocation request pair
The request task answered carries out resource allocation.
Further, step 3 further comprises:
The transition cost generated when calculating each clustered node resource allocation according to the resource related information of each clustered node,
The transition cost is the cost that energy level transition occurs when splitting resource group in clustered node every time and generates；
Each cluster after having distributed resource release in each clustered node is calculated according to the resource related information of each clustered node
The remaining cost of node；
Resource allocation when each clustered node distribution resource is calculated according to the transition cost of each clustered node and remaining cost
Cost.
Further, the transition cost obtains as follows:
The fractionation cost that resource group generates is split when calculating each clustered node resource allocation；
Transition cost when each clustered node resource allocation is calculated according to the fractionation cost for splitting resource group.
Further, the number of resources for splitting cost and including by surplus resources number in current cluster node and resource group
It calculates and obtains, or directly setting obtains.
Further, the remaining cost is obtained by surplus resources number in clustered node.
Further, the resource related information include each resource group includes in current cluster node number of resources, when
All total number resources in preceding clustered node use total number resource and surplus resources sum.
Further, the resource related information is obtained by the heartbeat signal that each clustered node is sent.
Further, in step 4, select the smallest clustered node distribution of resource allocation cost mutually should total number resource
Resource give the request task.
In order to achieve the above objectives, the present invention also provides a kind of concurrent computation resource distributors, comprising:
Resource allocation request receiving module obtains Current resource distribution request for receiving Current resource distribution request
Total number resource；
Resource information obtains module, the resource correlation letter for including for obtaining each clustered node in parallel computing trunking
Breath；
Resource allocation cost computing module calculates the money of each clustered node according to the resource related information of each clustered node
Cost is distributed in source；
Resource distribution module, according to the resource allocation cost of each clustered node and the resource of the current resource allocation request
Sum carries out resource allocation to the corresponding request task of the current resource allocation request.
Further, the resource allocation cost computing module includes:
Transition cost calculating unit, for calculating each clustered node resource point according to the resource related information of each clustered node
The transition cost that timing generates, the transition cost are that energy level transition occurs and generates when splitting resource group in clustered node every time
Cost；
Remaining cost calculating unit has divided for being calculated in each clustered node according to the resource related information of each clustered node
Remaining cost with each clustered node after resource release；
Cost calculating unit is distributed, each clustered node is calculated according to the transition cost of each clustered node and remaining cost and is distributed
Resource allocation cost when resource.
Further, resource group is split when the transition cost calculating unit calculates each clustered node resource allocation first
The fractionation cost of generation calculates transition cost when each clustered node resource allocation further according to the fractionation cost for splitting resource group.
Further, the number of resources for splitting cost and including by surplus resources number in current cluster node and resource group
It calculates and obtains, or directly setting obtains.
Further, the remaining cost is obtained by surplus resources number in clustered node.
Compared with prior art, a kind of concurrent computation resource distribution method of the present invention and device are by carrying out resource allocation
Before according to the resource related information for including in each clustered node, when calculating each clustered node distribution respective numbers resource
Cost is distributed, so that selection generates the less clustered node of fragment and carries out resource allocation, effectively when ensure that request resource every time
Ground improves the utilization rate of entire cluster resource.
Detailed description of the invention
Fig. 1 is a kind of step flow chart of concurrent computation resource distribution method of first embodiment of the invention；
Fig. 2 is the detailed flowchart of step 103 in first embodiment of the invention
Fig. 3 is the energy level transition schematic diagram generated when splitting resource group in clustered node in the specific embodiment of the invention；
Fig. 4 is a kind of structural schematic diagram of concurrent computation resource distributor of second embodiment of the invention
Fig. 5 is the detail structure chart of resource allocation cost computing module in second embodiment of the invention.
Specific embodiment
Below by way of specific specific example and embodiments of the present invention are described with reference to the drawings, those skilled in the art can
Understand further advantage and effect of the invention easily by content disclosed in the present specification.The present invention can also pass through other differences
Specific example implemented or applied, details in this specification can also be based on different perspectives and applications, without departing substantially from
Various modifications and change are carried out under spirit of the invention.
Fig. 1 is a kind of step flow chart of concurrent computation resource distribution method of first embodiment of the invention.As shown in Figure 1,
A kind of concurrent computation resource distribution method of the present invention, includes the following steps:
Step 101, Current resource distribution request is received, the total number resource of Current resource distribution request is obtained.
The total number resource of the Current resource distribution request is generally the number of resources needed when executing new task；The resource
Refer to that each clustered node distributes least resource unit when resource in the task of execution, such as in parallel computing trunking node
Carry out parallel computation using multiple GPU, then when single GPU is that clustered node executes task, the least resource unit of distribution.This
It include multiple clustered nodes in cluster when parallel computation, each clustered node includes one or more resource groups, each in invention
It include the resource of fixed quantity in resource group.
Step 102, the resource related information that each clustered node includes in parallel computing trunking is obtained.The resource is related
Information includes the number of resources that each resource group includes in current cluster node, all total number resources, use in current cluster node
Total number resource and surplus resources sum in embodiments of the present invention, the heartbeat that resource related information can be sent by each clustered node
Signal obtains, that is to say, that each clustered node periodically can send heartbeat signal to distributor, to prove oneself to be still active in collection
Group in, the heartbeat signal include each clustered node itself ID number and corresponding resource related information, but not limited to this.
Step 103, according to the resource related information of each clustered node, the resource allocation cost of each clustered node is calculated.
In the present invention, resource allocation cost consists of two parts: when each clustered node resource allocation, splitting resource group
The remaining cost after resource is released has been distributed in the energy level transition cost of generation and each clustered node.Fig. 2 is the present invention the
The detailed flowchart of step 103 in one embodiment.As shown in Fig. 2, step 203 further comprises following steps:
Step S1, the jump generated when calculating each clustered node resource allocation according to the resource related information of each clustered node
Move cost.The transition cost refers to the energy level transition that occurs when splitting resource group in clustered node every time and the cost that generates.By
It is limited to hardware, when distributing resource in clustered node, if necessary to split resource group, the cost of cost is often higher, i.e. fractionation generation
Valence is higher；If you do not need to splitting set of resources matches resource, the cost of cost is often lower, and well below fractionation resource group
When the fractionation cost that spends, can be ignored, therefore, the calculating process of transition cost is as follows:
1) the fractionation cost that resource group generates is split first in computing cluster node when resource allocation
Splitting cost only can just generate when resource group in splitting clustered node, split in cost and current cluster node
The number of resources that surplus resources number and resource group include is related, can pass through surplus resources number in current cluster node and resource group packet
The number of resources contained, which calculates, to be obtained, in the specific embodiment of the invention, specific formula for calculation such as following formula (1):
Wherein, k indicates the fractionation cost generated when splitting resource group in clustered node；R indicates to remain in current cluster node
Remaining number of resources, g are the number of resources that resource group includes in clustered node, and j is to split cost adjustment parameter, and value is more than or equal to 1
Integer, specific value can be determined according to application demand or experimental result, it acts as guaranteeing that the value of k is 0 or 1, i.e., ought be torn open
When dividing resource group, value 1, when not splitting resource group, value 0；
It should be noted that above-mentioned formula (1) is merely illustrative calculation method, it is of course also possible to not use what above formula provided to tear open
Divide cost calculation method, directly setting clustered node splits the fractionation cost of resource group every time, such as directly set when distributing resource
Fractionation cost when the fixed resource group of fractionation every time is 1, when not splitting resource group, and splitting cost is 0, and invention is not limited thereto.
2) transition cost when each clustered node resource allocation is calculated according to the fractionation cost for splitting resource group
When there are multiple resource groups in clustered node, cluster is caused splitting resource group progress resource allocation Shi Douhui every time
The transition of node energy level generates transition cost, and the energy level transition refers to when not splitting resource group in cluster when resource allocation, energy level
It does not change, after splitting resource group, energy level is reduced, and transition occurs for energy level, and then generates transition cost.
It is generated when all fractionation resource groups in each clustered node of transition cost when each clustered node resource allocation
The sum of cost is split, if Fig. 3 is that the energy level transition generated when splitting resource group in clustered node in the specific embodiment of the invention is shown
It is intended to, wherein the clustered node includes 8 resources altogether, and 2 resource groups, fixed each resource group includes 4 resources, works as cluster
When middle resource drops to 4 by 8 or drops to 0 by 4, transition occurs for energy level, generates transition cost, and the transition cost passes through such as following formula
(2) it calculates and obtains:
Wherein, TR-aFrom the transition cost after a resource of R resource allocation after expression clustered node distribution resource, a is indicated
Request number of resources, kiIndicate that fractionation cost when clustered node distribution resource has split money only when distributing i-th of resource
Source group, kiValue is 1, otherwise, value 0.
Illustrate the calculating process of each clustered node transition cost below by way of a specific example.It is assumed that request number of resources
It is 2, and includes two clustered nodes node1 and node2, the relevant information of each clustered node such as following table in parallel computing trunking
Shown in 1:
Table 1
Wherein, g indicates the number of resources that resource group includes in each clustered node, and a indicates request number of resources, and Total is indicated
The total number resource that each clustered node includes, Used indicate the assigned number of resources of each clustered node, R1、R2Table respectively
Show clustered node surplus resources number after distributing resource；The calculating of the transition cost of each clustered node in the specific embodiment of the invention
Process is as follows:
1) the transition cost of node1 node is calculated
It first calculates node1 node and distributes the fractionation cost after two resources, since remaining 5 resources of node1 node are available,
Each resource group includes 4 resources, when node distributes the 1st resource, resource group is not split, when node distributes the 2nd money
When source, need to split a resource group, therefore, can calculate according to formula (1) and split cost is 1；Due to node1 node distribution 2
A resource group is split when a resource altogether；Therefore, transition cost when node1 node is reduced to 3 resources by 5 resources is institute
Have and splits the sum of cost, i.e., 1；
2) the transition cost of node2 node is calculated
It equally first calculates node2 node and distributes the fractionation cost after two resources, due to remaining 3 resources of node2 node
It can use, each resource group includes 4 resources, after node distributes two resources, does not split resource group, and splitting cost is 0；
It therefore is 0 by the transition cost that 3 resources are reduced to 1 resource after node2 node distributes 2 resources.
Step S2 is calculated according to the resource related information of each clustered node after having distributed resource release in each clustered node
The remaining cost of each clustered node.
It, can be whole by resource in clustered node due in actual use, guaranteeing the integrality of resource in each clustered node
Body distributes to the biggish prior task of resource consumption, therefore value with higher only lacks a money in clustered node
When source, the potential value having is higher, because only needing to discharge the integrality that a resource ensures that the clustered node,
When carrying out resource allocation, task directly that resource allocation in entire clustered node is bigger to resource consumption.For this purpose, the present invention exists
When calculating the resource allocation cost of each clustered node, it is also contemplated that allocated resource collects upon discharge in each clustered node
The remaining cost of group node, the residue cost refers to since clustered node is when distributing resource, does not consider to have divided in clustered node
Lead to the impaired generated cost of potential value with the integrality of clustered node can be kept after node release, specific residue generation
The calculation method of valence is as follows:
The remaining cost of the clustered node is related to surplus resources number in clustered node, can be by remaining in clustered node
Number of resources obtains, and when surplus resources are more in clustered node, and surplus resources are distributed to the task of request resource, loss is dived
It is bigger being worth, to keep the remaining cost of clustered node bigger；Therefore, the remaining cost and clustered node of the clustered node
In remaining number of resources be inversely proportional, remaining cost can be used such as following formula (3) calculate obtain:
Wherein, LRRemaining cost when for clustered node R resource of residue, R are that clustered node distributes the residue before resource
Number of resources；
Step S3, when calculating each clustered node distribution resource according to the transition cost of each clustered node and remaining cost
Distribute cost.
In the specific embodiment of the invention, it is described distribution cost be each clustered node distribute resource after transition cost with
The sum of remaining cost is obtained using such as following formula (4):
CR-a=TR-a+LR (4)
Wherein, CR-aFor the distribution cost in clustered node when remaining R resource after a resource of distribution.It should be noted herein
It is that formula (4) is merely illustrative, transition cost and remaining cost can also be weighted after processing and calculates distribution generation
Valence, invention is not limited thereto.
Illustrate the calculating process for distributing cost below by way of a specific example: such as current request number of resources is 2
The resource related information of clustered node node1 and node2 are as shown in table 2 below, and design parameter meaning is identical as table 1 in table, wherein C
Distribution cost when resource is distributed for each clustered node, other parameters meaning is identical as table 1, clustered node node1 and node2
Distribution cost C calculating process it is as described below:
Table 2
Transition cost after first calculating each clustered node distribution resource, since node1 and node2 node distributes 2 moneys
Resource group is not split behind source, therefore fractionation cost and transition cost are all 0；
Calculate the remaining cost of node1 node: 1-1/7=6/7；
Calculate the remaining cost of node2 node: 1-1/2=1/2；
Therefore, the distribution cost of node1 node distribution resource is 0+6/7=6/7, and node2 node distributes the distribution of resource
Cost is 0+1/2=1/2.
Step 104, according to the resource allocation cost of each clustered node and resource request sum to the current resource allocation
Corresponding request task is requested to carry out resource allocation.In the embodiment of the present invention, in specific distribution, then select distribution cost minimum
Clustered node distribute the resource of corresponding requested resource sum to the request task.
Since clustered node each in parallel computing trunking can include one or more resource groups, each resource group includes to fix
The resource allocation methods of the resource of quantity, the prior art are allocated resource in cluster only with mutual exclusion, exclusive mode, not
Consider the distribution cost generated when resource allocation, but directly give the resource allocation for meeting demand to demand task, often tears open
The set of resources divided in each clustered node matches resource.In this case, if surplus resources are less in a resource group, next demand
When task requests resource, it is often unable to satisfy requirement, thus the fragment for generating resource fragmentation, and generating over time
Can be more and more, when there is larger task to need more resource, resource fragmentation is unable to satisfy demand, larger task in clustered node
After needing to wait and having distributed resource release, the resource of quantity required can be just obtained, thus it is longer to will cause task waiting time, collection
The problems such as group's parallel processing capability decline, resource utilization is lower, the present invention not only considers each clustered node in resource allocation
Transition cost after distribution request number of resources resource, while considering the clustered node that each clustered node has distributed after resource release
Remaining cost, resource point is carried out to request task according to the resource allocation cost of each clustered node and resource request sum
Match, problem of the existing technology can be well solved.
Fig. 4 is a kind of structural schematic diagram of concurrent computation resource distributor of second embodiment of the invention.As shown in figure 4,
A kind of concurrent computation resource distributor of the present invention, comprising: resource allocation request receiving module 10, resource information obtain module
11, resource allocation cost computing module 12 and resource distribution module 13.
Wherein, resource allocation request receiving module 10 obtains Current resource distribution for receiving Current resource distribution request
The total number resource of request.The total number resource of the Current resource distribution request is generally the number of resources needed when executing new task；
The resource refers to that each clustered node in the task of execution, distributes least resource unit when resource, such as parallel computing trunking section
Carry out parallel computation using multiple GPU in point, then when single GPU is that clustered node executes task, the least resource list of distribution
Position.It include multiple clustered nodes in cluster when parallel computation in the present invention, each clustered node includes one or more resource groups,
It include the resource of fixed quantity in each resource group.
Resource information obtains module 11, the resource correlation letter for including for obtaining each clustered node in parallel computing trunking
Breath.The resource related information includes the number of resources that each resource group includes in current cluster node, institute in current cluster node
There is total number resource, using total number resource and surplus resources sum.In the specific embodiment of the invention, resource related information can pass through
The heartbeat signal that each clustered node is sent obtains, that is to say, that and each clustered node periodically can send heartbeat signal to distributor,
To prove oneself to be still active in cluster, the heartbeat signal includes the ID number and corresponding resource phase of each clustered node itself
Close information.
Resource allocation cost computing module 12 calculates each clustered node according to the resource related information of each clustered node
Resource allocation cost.
In the present invention, resource allocation cost consists of two parts: when each clustered node resource allocation, splitting resource group
The remaining cost after resource is released has been distributed in the energy level transition cost of generation and each clustered node.Fig. 5 is the present invention the
The detail structure chart of resource allocation cost computing module in two embodiments.As shown in figure 5, resource allocation cost computing module 12 into
One step includes: transition cost calculating unit 120, remaining cost calculating unit 121, distribution cost calculating unit 122.
Transition cost calculating unit 120, for calculating each clustered node according to the resource related information of each clustered node
The transition cost generated when resource allocation.The transition cost refers to the energy level jump occurred when splitting resource group in clustered node every time
The cost moved and generated.It is limited to hardware, when distributing resource in clustered node, if necessary to split resource group, the cost of cost
Often higher, i.e. fractionation cost is higher；If you do not need to splitting set of resources matches resource, the cost of cost is often lower, and
Well below the fractionation cost spent when splitting resource group, can be ignored, therefore, the meter of transition cost calculating unit 120
Calculation process is as follows:
1) the fractionation cost that resource group generates is split when first calculating resource allocation in each clustered node
The fractionation cost only can just generate when resource group in splitting clustered node, the fractionation cost and current collection
Surplus resources number and resource group include that number of resources is related in group node, can pass through surplus resources number and money in current cluster node
The number of resources that source group includes, which calculates, to be obtained, and in the specific embodiment of the invention, circular is obtained using following formula:
Wherein, k indicates the fractionation cost generated when splitting resource group in clustered node；R indicates to remain in current cluster node
Remaining number of resources, g are that resource group includes number of resources in clustered node, and j is to split cost adjustment parameter, for the integer more than or equal to 1,
Specific value can be determined according to application demand or experimental result, it acts as guaranteeing that the value of k is 0 or 1, i.e., when fractionation resource group
When, value 1, when not splitting resource group, value 0；
It should be noted that above formula is merely illustrative calculation method, it is of course also possible to which not using above formula to provide splits cost meter
Calculation method, directly setting clustered node split the fractionation cost of resource group, such as directly setting is torn open every time when distributing resource every time
Dividing fractionation cost when resource group is 1, when not splitting resource group, and splitting cost is 0；Invention is not limited thereto.
2) transition cost when being distributed according to the fractionation cost computing cluster node resource for splitting resource group
When there are multiple resource groups in clustered node, cluster is caused splitting resource group progress resource allocation Shi Douhui every time
The transition of node energy level generates transition cost, and the energy level transition refers to when not splitting resource group in cluster when resource allocation, energy level
It does not change, after splitting resource group, energy level is reduced, and transition occurs for energy level, and then generates transition cost.
What transition cost when clustered node resource allocation generated when being all fractionation resource groups in each clustered node tears open
Divide the sum of cost.Following formula acquisition can be used in the transition cost:
Wherein, TR-aFrom the transition cost after a resource of R resource allocation after expression clustered node distribution resource, a is indicated
Request number of resources, kiIndicate that fractionation cost when clustered node distribution resource has split money only when distributing i-th of resource
Source group, kiValue is 1, otherwise, value 0.
Remaining cost calculating unit 121, for calculating each clustered node according to the resource related information of each clustered node
In distributed resource release after each clustered node remaining cost.
It, can be whole by resource in clustered node due in actual use, guaranteeing the integrality of resource in each clustered node
Body distributes to the biggish prior task of resource consumption, therefore value with higher only lacks a money in clustered node
When source, the potential value having is higher, because only needing to discharge the integrality that a resource ensures that the clustered node,
When carrying out resource allocation, task directly that resource allocation in entire clustered node is bigger to resource consumption.For this purpose, the present invention exists
When calculating the resource allocation cost of each clustered node, it is also contemplated that allocated resource collects upon discharge in each clustered node
The remaining cost of group node, the residue cost refers to since clustered node is when distributing resource, does not consider to have divided in clustered node
Lead to the impaired generated cost of potential value with the integrality of clustered node can be kept after node release, specific residue generation
The calculating process of valence is as follows:
The remaining cost of the clustered node is related to surplus resources number in clustered node, can be by remaining in clustered node
Number of resources obtains, and when surplus resources are more in clustered node, and surplus resources are distributed to the task of request resource, loss is dived
It is bigger being worth, to keep the remaining cost of clustered node bigger；Therefore, the remaining cost and clustered node of the clustered node
In remaining number of resources be inversely proportional, it is described residue cost calculation method be shown below:
Wherein, LRRemaining cost when for clustered node R resource of residue, R are that clustered node distributes the residue before resource
Number of resources；
Cost calculating unit 122 is distributed, for calculating each collection according to the transition cost and remaining cost of each clustered node
Group node distributes distribution cost when resource.
In embodiments of the present invention, the distribution cost is that each clustered node distributes transition cost and residue after resource
The sum of cost is obtained using following formula:
CR-a=TR-a+LR
Wherein, CR-aFor the distribution cost in clustered node when remaining R resource after a resource of distribution.
Resource distribution module 13, according to the resource allocation cost of each clustered node and resource request sum to request task
Carry out resource allocation.In the embodiment of the present invention, in specific distribution, then select the smallest clustered node distribution of distribution cost corresponding
The resource of resource request sum is to request task.
In conclusion a kind of concurrent computation resource distribution method of the present invention and device pass through the root before carrying out resource allocation
According to the resource related information for including in each clustered node, distribution generation when each clustered node distribution respective numbers resource is calculated
Valence effectively improves so that selection generates the less clustered node of fragment and carries out resource allocation when ensure that request resource every time
The utilization rate of entire cluster resource.
Anyone skilled in the art without departing from the spirit and scope of the present invention, repair above-described embodiment
Decorations and change.Therefore, the scope of the present invention, should be as listed in the claims.
Claims (13)
1. a kind of concurrent computation resource distribution method, includes the following steps:
Step 1 receives Current resource distribution request, obtains the total number resource of the current resource allocation request；
Step 2 obtains the resource related information that each clustered node includes in parallel computing trunking；
Step 3 calculates the resource allocation cost of each clustered node, wherein providing according to the resource related information of each clustered node
Source distribution cost consist of two parts: when each clustered node resource allocation, split resource group generate energy level transition cost and
The remaining cost after resource is released has been distributed in each clustered node；
Step 4, it is corresponding to the current resource allocation request according to the resource allocation cost of each clustered node and the total number resource
Request task carries out resource allocation.
2. a kind of concurrent computation resource distribution method as described in claim 1, which is characterized in that step 3 further comprises:
The transition cost generated when calculating each clustered node resource allocation according to the resource related information of each clustered node, it is described
Transition cost is the cost that energy level transition occurs when splitting resource group in clustered node every time and generates；
Each clustered node after having distributed resource release in each clustered node is calculated according to the resource related information of each clustered node
Remaining cost；
Resource allocation cost when each clustered node distribution resource is calculated according to the transition cost of each clustered node and remaining cost.
3. a kind of concurrent computation resource distribution method as claimed in claim 2, which is characterized in that the transition cost is by such as
Lower step obtains:
The fractionation cost that resource group generates is split when calculating each clustered node resource allocation；
Transition cost when each clustered node resource allocation is calculated according to the fractionation cost for splitting resource group.
4. a kind of concurrent computation resource distribution method as claimed in claim 3, it is characterised in that: the fractionation cost is by working as
The number of resources that surplus resources number and resource group include in preceding clustered node, which calculates, to be obtained, or directly setting obtains.
5. a kind of concurrent computation resource distribution method as claimed in claim 2, it is characterised in that: the residue cost passes through collection
Surplus resources number obtains in group node.
6. a kind of concurrent computation resource distribution method as described in claim 1, it is characterised in that: the resource related information packet
Include the number of resources that each resource group includes in current cluster node, all total number resources in current cluster node, total using resource
Several and surplus resources sum.
7. a kind of concurrent computation resource distribution method as claimed in claim 6, it is characterised in that: the resource related information is logical
The heartbeat signal that each clustered node is sent is crossed to obtain.
8. a kind of concurrent computation resource distribution method as described in claim 1, it is characterised in that: in step 4, selection money
Source distribute the distribution of cost the smallest clustered node mutually should total number resource resource to the request task.
9. a kind of concurrent computation resource distributor, comprising:
Resource allocation request receiving module obtains the resource of Current resource distribution request for receiving Current resource distribution request
Sum；
Resource information obtains module, for obtaining the resource related information that each clustered node includes in parallel computing trunking；
Resource allocation cost computing module calculates the resource of each clustered node according to the resource related information of each clustered node
Cost is distributed, wherein resource allocation cost consists of two parts: when each clustered node resource allocation, splitting what resource group generated
The remaining cost after resource is released has been distributed in energy level transition cost and each clustered node；
Resource distribution module, according to the total number resource of the resource allocation cost of each clustered node and the current resource allocation request
Resource allocation is carried out to the corresponding request task of the current resource allocation request.
10. concurrent computation resource distributor as claimed in claim 9, which is characterized in that the resource allocation cost calculates
Module includes:
Transition cost calculating unit, when for calculating each clustered node resource allocation according to the resource related information of each clustered node
The transition cost of generation, the transition cost are the generation that energy level transition occurs when splitting resource group in clustered node every time and generates
Valence；
Remaining cost calculating unit has distributed money for calculating in each clustered node according to the resource related information of each clustered node
The remaining cost of each clustered node after the release of source；
Cost calculating unit is distributed, each clustered node is calculated according to the transition cost of each clustered node and remaining cost and distributes resource
When resource allocation cost.
11. concurrent computation resource distributor as claimed in claim 10, which is characterized in that the transition cost calculating unit
The fractionation cost that resource group generates is split when calculating each clustered node resource allocation first, further according to the fractionation for splitting resource group
Cost calculates transition cost when each clustered node resource allocation.
12. concurrent computation resource distributor as claimed in claim 11, it is characterised in that: the fractionation cost passes through current
The number of resources that surplus resources number and resource group include in clustered node, which calculates, to be obtained, or directly setting obtains.
13. concurrent computation resource distributor as claimed in claim 10, it is characterised in that: the residue cost passes through cluster
Surplus resources number obtains in node.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610875024.0A CN106506594B (en) | 2016-09-30 | 2016-09-30 | A kind of concurrent computation resource distribution method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610875024.0A CN106506594B (en) | 2016-09-30 | 2016-09-30 | A kind of concurrent computation resource distribution method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106506594A CN106506594A (en) | 2017-03-15 |
| CN106506594B true CN106506594B (en) | 2019-09-03 |
Family
ID=58293625
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610875024.0A Active CN106506594B (en) | 2016-09-30 | 2016-09-30 | A kind of concurrent computation resource distribution method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106506594B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11144828B2 (en) * | 2017-06-09 | 2021-10-12 | Htc Corporation | Training task optimization system, training task optimization method and non-transitory computer readable medium for operating the same |
| CN107577534A (en) * | 2017-08-31 | 2018-01-12 | 郑州云海信息技术有限公司 | A kind of resource regulating method and device |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102043659A (en) * | 2010-12-08 | 2011-05-04 | 上海交通大学 | Compiling device for eliminating memory access conflict and implementation method thereof |
| CN102185713B (en) * | 2011-05-04 | 2014-07-30 | 浙江大学 | Global optimization method of internet service resource distribution |
| CN103218233B (en) * | 2013-05-09 | 2015-11-18 | 福州大学 | Data allocation strategy in Hadoop isomeric group |
| CN104536804A (en) * | 2014-12-23 | 2015-04-22 | 西安电子科技大学 | Virtual resource dispatching system for related task requests and dispatching and distributing method for related task requests |
| CN104881322B (en) * | 2015-05-18 | 2018-10-09 | 中国科学院计算技术研究所 | A kind of cluster resource dispatching method and device based on vanning model |
-
2016
- 2016-09-30 CN CN201610875024.0A patent/CN106506594B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN106506594A (en) | 2017-03-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107659433B (en) | Cloud resource scheduling method and equipment | |
| Levin et al. | DP-FAIR: A simple model for understanding optimal multiprocessor scheduling | |
| CN103534687B (en) | Extensible centralized dynamic resource distribution in a clustered data grid | |
| CN105912403B (en) | The method for managing resource and device of Docker container | |
| CN109992422A (en) | A kind of method for scheduling task towards GPU resource, device and system | |
| CN103853618B (en) | Resource allocation method with minimized cloud system cost based on expiration date drive | |
| CN109254842A (en) | Method for managing resource, device and the readable storage medium storing program for executing of distributive type system | |
| CN104881325A (en) | Resource scheduling method and resource scheduling system | |
| CN103297499B (en) | Scheduling method and system based on cloud platform | |
| CN103414752B (en) | A kind of cloud data center virtual machine distribution method of network aware | |
| CN103561055B (en) | Web application automatic elastic extended method under conversation-based cloud computing environment | |
| CN108845874B (en) | Dynamic resource allocation method and server | |
| CN103810047B (en) | Dynamic improves the memory affinity of logical partition | |
| CN103942098A (en) | System and method for task processing | |
| CN103336722A (en) | Virtual machine CPU source monitoring and dynamic distributing method | |
| CN105975340A (en) | Deployment algorithm of application distribution of virtual machine | |
| CN103401939A (en) | Load balancing method adopting mixing scheduling strategy | |
| CN105808341B (en) | A kind of methods, devices and systems of scheduling of resource | |
| CN106034153B (en) | Method for allocating bandwidth resources, data transmission method, apparatus and system | |
| CN110109756A (en) | A kind of network target range construction method, system and storage medium | |
| CN101685480A (en) | Parallel computing method for security and stability analysis of large power grid and computing platform | |
| WO2016101996A1 (en) | Allocating cloud computing resources in a cloud computing environment | |
| CN104461748A (en) | Optimal localized task scheduling method based on MapReduce | |
| CN106506594B (en) | A kind of concurrent computation resource distribution method and device | |
| CN106790332A (en) | A kind of resource regulating method, system and host node |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |