CN107704320A - Task allocation method and system for distributed system - Google Patents

Task allocation method and system for distributed system Download PDF

Info

Publication number
CN107704320A
CN107704320A CN 201710333404 CN201710333404A CN107704320A CN 107704320 A CN107704320 A CN 107704320A CN 201710333404 CN201710333404 CN 201710333404 CN 201710333404 A CN201710333404 A CN 201710333404A CN 107704320 A CN107704320 A CN 107704320A
Authority
CN
Grant status
Application
Patent type
Prior art keywords
resource
processing
set
score
out
Prior art date
Application number
CN 201710333404
Other languages
Chinese (zh)
Other versions
CN107704320B (en )
Inventor
沈杰
张炎泼
Original Assignee
贵州白山云科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5044Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering hardware capabilities
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/50Indexing scheme relating to G06F9/50
    • G06F2209/501Performance criteria

Abstract

The invention discloses a task allocation method and system for a distributed system. The method comprises the steps of 1, obtaining current resource states of multiple processing resources of each processing resource combination comprised in multiple servers; 2, by utilizing the obtained current resource states of the multiple processing resources of each processing resource combination and weights of the multiple processing resources of each processing resource combination, calculating a total score of each processing resource combination; and 3, based on the calculated total score of each processing resource combination, selecting the processing resource combination, and allocating a processing task to the selected processing resource combination. According to the method and the system,the resources can be fully utilized, so that the cost is reduced and the service availability is greatly improved. In addition, task allocation is automatic, so that the system robustness is improved.

Description

_种分布式系统的任务分配方法及系统 _ Kind of task allocation method and system for distributed systems

技术领域 FIELD

[0001] 本发明涉及数据处理领域,尤其涉及一种分布式系统的任务分配方法及系统。 [0001] The present invention relates to data processing, and more particularly relates to a distributed system task allocation method and system.

背景技术 Background technique

[0002] 分布式系统通常涉及多个服务器,每个服务器上包括多个磁盘。 [0002] Distributed systems typically involve multiple servers, each server comprising a plurality of disks. 分布式系统需要处理的数据量通常是非常大的,如果将数据分配给某个或某几个磁盘进行处理,则会导致有些磁盘一直处于繁忙状态,而有些磁盘则一直处于空闲状态,从而不能合理地利用服务器上的处理资源。 The amount of data distributed systems need to be addressed are usually very large, if the data is assigned to one or several disks for processing, it will lead to some disk has been in a busy state, and some disks are in an idle state, which can not rational use of processing resources on the server.

[0003] 因此,当通过分布式系统对数据进行读写、处理等操作时,需要一种能够合理利用各个服务器上的处理资源,提高服务器处理效率的方法和系统。 [0003] Thus, when reading and writing, operating a distributed processing system for data, a need for a rational use of the various processing resources on the server, the server processing efficiency to improve methods and systems.

发明内容 SUMMARY

[0004] 为了解决现有技术中向服务器上的处理资源分配任务的问题,提出了一种用于分布式系统的任务分配方法和系统。 [0004] In order to solve the problem of the prior art processing resources assigned tasks on the server, the proposed task allocation method and system for distributed systems.

[0005] 根据本发明的一个方面,提供了一种分布式系统的任务分配方法,所述方法包括: [0005] In accordance with one aspect of the present invention, there is provided a distributed system task allocation, the method comprising:

[0006] 步骤1,获取多个服务器包括的每个处理资源组合的多个处理资源的当前资源状况; [0006] Step 1, the current resource status acquiring processing a plurality of processing resources for each resource combination comprising a plurality of servers;

[0007] 步骤2,利用获取的每个处理资源组合的多个处理资源的当前资源状况以及每个处理资源组合的多个处理资源的权重,计算每个所述处理资源组合的总分; [0007] Step 2, a plurality of current resource status acquisition processing resources using a combination of each processing resources and processing resources of a plurality of weights each processing resource combination weight, calculated for each combination out of the processing resource;

[0008] 步骤3,基于计算的各处理资源组合的总分,选择处理资源组合,并向所选择的处理资源组合分配处理任务。 [0008] Step 3, based on the score calculated for each combination of processing resources, resource combination selection processing, combination processing resources allocated to the selected processing task.

[0009] 其中,所述步骤2包括: [0009] wherein the step 2 comprises:

[0010] 基于每个处理资源组合的多个处理资源的当前资源状况确定各处理资源的评分, 将每个处理资源组合中处理资源的评分乘以该处理资源的权重后得到该处理资源的加权评分,并将每个处理资源组合中各处理资源的加权评分相加,分别得到每个处理资源组合的总分。 [0010] determination score for each processing resource based on the current resource status of the plurality of processing resources for each resource combination process, the combination processing resources each resource processing rates multiplied by the weight after re-processing resources to obtain the weighting processing resources score and weighting each process resource score for each combination of processing resources are added, respectively out of each processing resource combination.

[0011] 其中,所述步骤3包括: [0011] wherein said step 3 comprises:

[0012] 将各处理资源组合的总分按从高到低排序后进行分段,得到η个总分集合,第i个总分集合中的总分均大于第i+Ι个总分集合至第η个总分集合中的总分,从所述η个总分集合中选择前m个总分集合,并从所述前m个总分集合的每个总分集合中分别选择设定数量的总分,向与所选择的总分对应的处理资源组合分配处理任务,其中n、m和i均为大于0的正整数,且m<n,i<n〇 [0012] The score of each resource combination process performed by the descending segment sorted out to obtain a set of η, out of the i-th set out of a set of greater than one score to i + Ι the first set of scores η th score, score selected from the set top m out of a set of the η, the number and select from a set out of the front of each set of m out of a set of respectively the total score, the score corresponding to the selected combination of processing resources allocation processing tasks, wherein n, m and i are both positive integers greater than 0, and m <n, i <n〇

[0013] 其中,所述步骤3包括: [0013] wherein said step 3 comprises:

[0014] 将各处理资源组合的总分按从低到高排序后进行分段,得到η个总分集合,第i个总分集合中的总分均小于第i+Ι个总分集合至第η个总分集合中的总分,从所述η个总分集合中选择前m个总分集合,并从所述前m个总分集合的每个总分集合中分别选择设定数量的总分,向与所选择的总分对应的处理资源组合分配处理任务,其中n、m和i均为大于O的正整数,且m<n,i<n〇 [0014] Each resource combination treatment was carried out after segmentation from lowest to highest, to give a total score η set, the i-th set out of the total score is less than the first set out to th i + Ι the first set of scores η th score, score selected from the set top m out of a set of the η, the number and select from a set out of the front of each set of m out of a set of respectively the total score, the score corresponding to the selected combination of processing resources allocation processing tasks, wherein n, m and i are both positive integers greater than O, and m <n, i <n〇

[0015] 其中,所述步骤2还包括根据以下方式确定各处理资源的评分: [0015] wherein said step 2 further comprises determining rates of each processing resources in the following manner:

[0016] 方式一,当一处理资源的当前资源状况表示该处理资源的可用资源占比大于等于设定阈值时,将该处理资源的评分确定为该可用资源占比; [0016] one embodiment, when a current processing resources indicates the resource status accounting processing resources available resource is larger than equal to a set threshold value, the score is determined for the processing resources available resource accounting;

[0017] 方式二,当一处理资源的当前资源状况表示该处理资源的可用资源占比小于所述设定阈值时,将该处理资源的评分确定为零分。 [0017] The second approach, when a current resource situation of the resources available processing resources indicates the proportion of processing resources is less than the set threshold value, the determination processing resources score of zero.

[0018] 其中,所述步骤2还包括根据以下方式确定各处理资源的评分: [0018] wherein said step 2 further comprises determining rates of each processing resources in the following manner:

[0019] 方式一,当一处理资源的当前资源状况表示该处理资源的已用资源占比小于等于设定阈值时,将该处理资源的评分确定为该已用资源占比; [0019] In one embodiment, the processing resources when a resource status indicates the current processing resource has less resource share the set threshold value, the determination processing resources ratings for the resource has been accounted for;

[0020] 方式二,当一处理资源的当前资源状况表示该处理资源的已用资源占比大于所述设定阈值时,将该处理资源的评分确定为零分。 [0020] The second approach, when a current resource situation of the processing resources indicates processing resources when the set threshold, determining a score of the processing resources is greater than zero resource accounting.

[0021] 其中,在所述步骤2之前还包括: [0021] wherein, prior to said step 2 further comprises:

[0022] 根据所述处理资源组合的属性,为所述处理资源组合的多个处理资源设置相应的权重。 [0022] The combination of properties of the processing resource, for said plurality of processing resources setting processing resource combination corresponding weights.

[0023] 其中,从所述前m个总分集合的每个总分集合中分别选择设定数量的总分包括:基于哈希运算,从每个总分集合中分别选择设定数量的总分。 [0023] wherein the set number selected from each set out of the front set of m out out respectively comprising: based on a hash calculation of the total number selected from each set out of the collection, respectively Minute.

[0024] 根据本发明的另一方面,还提供了一种分布式系统的任务分配系统,所述系统包括: [0024] According to another aspect of the present invention, there is provided a system for distributed task allocation system, the system comprising:

[0025] 资源状况获取模块,获取多个服务器包括的每个处理资源组合的多个处理资源的当前资源状况; [0025] The resource status acquisition module acquiring a plurality of current resource status of each processing resource combination processing resources include a plurality of servers;

[0026] 总分计算模块,用于利用获取的每个处理资源组合的多个处理资源的当前资源状况以及每个处理资源组合的多个处理资源的权重,计算每个所述处理资源组合的总分; A plurality of current resource situation of the processing resources of each processing resource combination [0026] score calculating means for obtaining and utilizing processing resources of a plurality of weights for each combination of heavy processing resources, computing resources for each of the processing composition total score;

[0027] 任务分配模块,用于基于计算的各处理资源组合的总分,选择处理资源组合,并向所选择的处理资源组合分配处理任务。 [0027] Task assignment module, based on the total score is calculated for each combination of processing resources, resource combination selection processing, combination processing resources allocated to the selected processing task.

[0028] 其中,所述总分计算模块还用于: [0028] wherein said score calculating module is further configured to:

[0029] 基于每个处理资源组合的多个处理资源的当前资源状况确定各处理资源的评分, 将每个处理资源组合中处理资源的评分乘以该处理资源的权重后得到该处理资源的加权评分,并将每个处理资源组合中各处理资源的加权评分相加,分别得到每个处理资源组合的总分。 [0029] The score for each determination processing resources based on the current resource status of the plurality of processing resources for each resource combination process, the combination processing resources each resource processing rates multiplied by the weight after re-processing resources to obtain the weighting processing resources score and weighting each process resource score for each combination of processing resources are added, respectively out of each processing resource combination.

[0030] 其中,所述任务分配模块还用于: [0030] wherein the task assignment module is further configured to:

[0031] 将各处理资源组合的总分按从高到低排序后进行分段,得到η个总分集合,第i个总分集合中的总分均大于第i+Ι个总分集合至第η个总分集合中的总分,从所述η个总分集合中选择前m个总分集合,并从所述前m个总分集合的每个总分集合中分别选择设定数量的总分,向与所选择的总分对应的处理资源组合分配处理任务,其中n、m和i均为大于0的正整数,且m<n,i<n〇 [0031] The score of each resource combination process performed by the descending segment sorted out to obtain a set of η, out of the i-th set out of a set of greater than one score to i + Ι the first set of scores η th score, score selected from the set top m out of a set of the η, the number and select from a set out of the front of each set of m out of a set of respectively the total score, the score corresponding to the selected combination of processing resources allocation processing tasks, wherein n, m and i are both positive integers greater than 0, and m <n, i <n〇

[0032] 其中,所述任务分配模块还用于: [0032] wherein the task assignment module is further configured to:

[0033] 将各处理资源组合的总分按从低到高排序后进行分段,得到η个总分集合,第i个总分集合中的总分均小于第i+Ι个总分集合至第η个总分集合中的总分,从所述η个总分集合中选择前m个总分集合,并从所述前m个总分集合的每个总分集合中分别选择设定数量的总分,向与所选择的总分对应的处理资源组合分配处理任务,其中n、m和i均为大于O的正整数,且m<n,i<n〇 [0033] Each resource combination treatment was carried out after segmentation from lowest to highest, to give a total score η set, the i-th set out of the total score is less than the first set out to th i + Ι the first set of scores η th score, score selected from the set top m out of a set of the η, the number and select from a set out of the front of each set of m out of a set of respectively the total score, the score corresponding to the selected combination of processing resources allocation processing tasks, wherein n, m and i are both positive integers greater than O, and m <n, i <n〇

[0034] 其中,所述总分计算模块还用于根据以下方式确定各处理资源的评分: [0034] wherein said score calculating module is further for determining a score for each processing resource in the following manner:

[0035] 方式一,当一处理资源的当前资源状况表示该处理资源的可用资源占比大于等于设定阈值时,将该处理资源的评分确定为该可用资源占比; [0035] In one embodiment, the processing resources when a resource status indicates the current share of processing resources available resource is larger than equal to a set threshold value, the score is determined for the processing resources available resource accounting;

[0036] 方式二,当一处理资源的当前资源状况表示该处理资源的可用资源占比小于所述设定阈值时,将该处理资源的评分确定为零分。 [0036] The second approach, when a current resource situation of the resources available processing resources indicates the proportion of processing resources is less than the set threshold value, the determination processing resources score of zero.

[0037] 其中,所述总分计算模块还用于根据以下方式确定各处理资源的评分: [0037] wherein said score calculating module is further for determining a score for each processing resource in the following manner:

[0038] 方式一,当一处理资源的当前资源状况表示该处理资源的已用资源占比小于等于设定阈值时,将该处理资源的评分确定为该已用资源占比; [0038] one embodiment, processing resources when a resource status indicates the current processing resource has less resource share the set threshold value, the determination processing resources ratings for the resource has been accounted for;

[0039] 方式二,当一处理资源的当前资源状况表示该处理资源的已用资源占比大于所述设定阈值时,将该处理资源的评分确定为零分。 [0039] Mode 2 when the current resource situation indicates that a processing resource of processing resources when the set threshold, determining a score of the processing resources is greater than zero resource accounting.

[0040] 其中,所述系统还包括: [0040] wherein said system further comprises:

[0041] 权重设置模块,用于根据所述处理资源组合的属性,为所述处理资源组合的多个处理资源设置相应的权重。 [0041] a weight setting means for the processing resource according to an attribute combination, said plurality of processing resources setting processing resource combination corresponding weights.

[0042] 其中,所述任务分配模块还用于: [0042] wherein the task assignment module is further configured to:

[0043] 基于哈希运算,从每个总分集合中分别选择设定数量的总分。 [0043] Based on hashing selected out of a set number of out from each set, respectively.

[0044] 本发明中的分布式系统的任务分配方法和系统,通过利用处理资源组合中各处理资源的评分和权重计算处理资源的总分,并将总分排序后分段,选择处理资源组合分配任务,可以有效地利用处理资源,避免部分处理资源过于繁忙,部分处理资源却过于空闲,从而提高系统的处理效率。 [0044] The distributed system of the present invention is a method and system for allocating tasks, the weight calculation processing resources out by utilizing processing resources of the score and weight of each combination of processing resources, and to sort out segment, resource combination selection processing assign tasks, efficient use of processing resources can avoid part of the processing resources are too busy, too idle resources are part of the process, thus improving the processing efficiency of the system. 因此,采用本发明的方法和系统,可以实现资源的充分利用,节约成本,极大提高服务可用性。 Accordingly, the present invention is a method and system that can achieve full utilization of resources, cost savings, greatly improves service availability. 此外,在进行任务分配时实现自动分配,提高系统的健壮性。 In addition, automated dispensing carrying out the task assigned to improve the robustness of the system.

附图说明 BRIEF DESCRIPTION

[0045] 构成本发明的一部分的附图用来提供对本发明的进一步理解,本发明的示意性实施例及其说明用于解释本发明,并不构成对本发明的不当限定。 [0045] The drawings constitute a part of the present invention to provide a further understanding of the invention, exemplary embodiments of the present invention are used to explain the present invention without unduly limiting the present invention. 在附图中: In the drawings:

[0046] 图1是根据本发明的分布式系统的任务分配方法流程图; [0046] FIG. 1 is a flowchart of a task allocation method for distributed system according to the present invention;

[0047] 图2是根据本发明的分布式系统的任务分配系统模块图。 [0047] FIG. 2 is a system block diagram of a distributed task allocation system according to the invention.

具体实施方式 detailed description

[0048] 为使本发明实施例的目的、技术方案和优点更加清楚,下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例是本发明一部分实施例,而不是全部的实施例。 [0048] In order that the invention object, technical solutions, and advantages of the embodiments more clearly, the following the present invention in the accompanying drawings, technical solutions of embodiments of the present invention are clearly and completely described, obviously, the described the embodiment is an embodiment of the present invention is a part, but not all embodiments. 基于本发明中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本发明保护的范围。 Based on the embodiments of the present invention, all other embodiments of ordinary skill in the art without any creative effort shall fall within the scope of the present invention. 需要说明的是,在不冲突的情况下,本申请中的实施例及实施例中的特征可以相互任意组合。 Incidentally, in the case of no conflict, embodiments and features of the embodiments of the present application may be arbitrarily combined with each other.

[0049] 本发明提供了一种分布式系统的任务分配方法,如图1所述,该方法包括: [0049] The present invention provides a method for task allocation in a distributed system, as in claim 1, the method comprising:

[0050] 步骤101,获取多个服务器包括的每个处理资源组合的多个处理资源的当前资源状况; [0050] Step 101, the current resource status acquiring a plurality of processing resources for each of the plurality of server processing resources include a combination of;

[0051] 步骤102,利用获取的每个处理资源组合的多个处理资源的当前资源状况以及每个处理资源组合的多个处理资源的权重,计算每个处理资源组合的总分; [0051] Step 102, the current resource status acquisition by a plurality of processing resources for each combination of processing resources and processing resources of a plurality of weights each processing resource combination weight, calculated for each combination of processing resources score;

[0052] 步骤103,基于计算的各处理资源组合的总分,选择处理资源组合,并向所选择的处理资源组合分配处理任务。 [0052] Step 103, based on the score calculated for each combination of processing resources, resource combination selection processing, combination processing resources allocated to the selected processing task.

[0053] 上述步骤102包括:基于每个处理资源组合的多个处理资源的当前资源状况确定各处理资源的评分,将每个处理资源组合中处理资源的评分乘以该处理资源的权重后得到该处理资源的加权评分,并将每个处理资源组合中各处理资源的加权评分相加,分别得到每个处理资源组合的总分。 [0053] The step 102 comprises: determining a process resource score for each resource based on the current status of the plurality of processing resources for each resource combination process, the combination of each processing resource processing resources score obtained by multiplying the weight of the heavy processing resources the weighted scoring processing resources, and each resource combination process weighted score for each processing resource summing scores were obtained for each combination of processing resources. 即,处理资源的总分是由各处理资源的评分和相应的权重进行计算得到的。 That is, the total score by the score processing resources and processing resources corresponding weight for weight calculated.

[0054] 上述步骤103包括:将各处理资源组合的总分按高低排序后进行分段,得到η个总分集合,第i个总分集合中的总分均大于第i+Ι个总分集合至第η个总分集合中的总分,从所述η个总分集合中选择前m个总分集合,并从所述前m个总分集合的每个总分集合中分别选择设定数量的总分,向与所选择的总分对应的处理资源组合分配处理任务,其中n、m和i均为大于0的正整数,且m彡n,i彡η。 [0054] The step 103 comprises: the score of each resource combination process by the level segmenting sorted out to obtain a set of η, the i-th set out of the total score greater than i + Ι out th set to the second set of scores η th score, score selected from the set top m out of a set of the η, and select from the set provided out of the front of each set of scores of m, respectively a given number of scores, the score corresponding to the selected combination of processing resources allocation processing tasks, wherein n, m and i are both positive integers greater than 0, and m San n, i San η.

[0055] S卩,将各处理资源组合的总分按高低排序后分段,得到多个总分集合,例如,第1总分集合、第2总分集合、…、第η总分集合。 [0055] S Jie, the score of each resource combination process by the sorting level segment, set out to obtain a plurality of, e.g., a first set of scores, the second set of score, ..., η first set score. 排在前面的总分集合中的总分均大于排在后面的总分集合中的总分。 Out of the top surface is set out greater than set out in the back row in the score. 另外,在选择处理资源组合时,并不是全部从排名最靠前的总分集合中选择,而是首先选择几个排名靠前的总分集合,然后分别从所选的总分集合中选择设定数量的总分。 Further, when selecting the combination of processing resources, not all be selected from the highest ranking score set, but rather a set of first selected out of several top-ranking, then selecting the setting from the selected set out in given the number of scores. 这是因为,在步骤102中计算的总分是根据各处理资源上报的当前资源状况计算的,而各处理资源的资源使用状况可能会实时改变,即计算出的总分可能会实时调整,在这种情况下,如果全部从排名最靠前的总分集合中进行选择,则这个分配方法不一定是最优的。 This is because, calculated in step 102 are calculated based on the total score for each processing resource reported current resources, and the resource usage of each processing resource may be changed in real time, i.e., the calculated score may be adjusted in real time in in this case, if all choose from the collection of the highest ranking score, then this allocation method is not necessarily optimal. 而采用本发明这种排序分段,并从多个集合中分别选择多个总分的方法,可以有效保证处理资源的实时优化利用。 The use of the present invention, this sort segment, and selecting out of the plurality of collection from the plurality of methods, respectively, can effectively guarantee real-time to optimize the use of processing resources.

[0056] 在这种情况下,步骤102还包括根据以下方式确定各处理资源的评分: [0056] In this case, the step 102 further comprises determining the respective rates of processing resources in the following manner:

[0057] 方式一,当一处理资源的当前资源状况表示该处理资源的可用资源占比大于等于设定阈值时,将该处理资源的评分确定为该可用资源占比; [0057] one embodiment, when a current processing resources indicates the resource status accounting processing resources available resource is larger than equal to a set threshold value, the score is determined for the processing resources available resource accounting;

[0058] 方式二,当一处理资源的当前资源状况表示该处理资源的可用资源占比小于所述设定阈值时,将该处理资源的评分确定为零分。 [0058] The second approach, when a current resource situation of the resources available processing resources indicates the proportion of processing resources is less than the set threshold, the processing resource is determined as zero score.

[0059] 在上述确定处理资源的评分的方式中,当处理资源的可用资源占比比较小时,则认为该处理资源不可用,即将该处理资源的评分确定为零分。 [0059] In determining the score processing resources, the processing resources when a relatively small proportion of the available resources, it is considered that the processing resource is not available, i.e. the process resource score is determined as zero. 另外,步骤101中获取的处理资源的当前资源状况可以是该处理资源的资源已使用状况或资源可用状况,即已用资源占比或可用资源占比。 Further, in step 101 the current resource status acquiring processing resources may be resources of the processing resources used condition or status of resources available, already available resources or resource accounting accounting. 当获取的是已用资源占比时,计算1-已用资源占比即可得到可用资源占比,其中的占比可以是百分比。 When the acquired resource is already accounted for when calculating 1- proportion of available resources have been used to share resources, which may be a percentage proportion.

[0060] 另外,上述步骤103还可以包括:将各处理资源组合的总分按从低到高排序后进行分段,得到η个总分集合,第i个总分集合中的总分均小于第i+Ι个总分集合至第η个总分集合中的总分,从所述η个总分集合中选择前m个总分集合,并从所述前m个总分集合的每个总分集合中分别选择设定数量的总分,向与所选择的总分对应的处理资源组合分配处理任务,其中n、m和i均为大于0的正整数,且m<n,i<n。 [0060] Further, the above step 103 may further comprise: processing the score of each resource combination according segmented from lowest to highest, to give a total score η set, the i-th set out of the total score less than i + Ι out of a set of η to set a score in the score, score selected from the first m out of a set of η set, and set out from the front of each of the m were selected out of a set of scores set number, the score corresponding to the selected combination of processing resources allocation processing tasks, wherein n, m and i are both positive integers greater than 0, and m <n, i < n. 该排序方法与前面的从高到低的方法相反,是因为所使用的处理资源的评分是已用资源占比,即已用资源占比越小越好。 In contrast with the method of the foregoing method of the descending sort, because the processing resources are already used resource accounting rates, ie the smaller the better resource accounting.

[0061] 在这种情况下,步骤102还包括根据以下方式确定各处理资源的评分: [0061] In this case, the step 102 further comprises determining the respective rates of processing resources in the following manner:

[0062] 方式一,当一处理资源的当前资源状况表示该处理资源的已用资源占比小于等于设定阈值时,将该处理资源的评分确定为该已用资源占比; [0062] one embodiment, processing resources when a resource status indicates the current processing resource has less resource share the set threshold value, the determination processing resources ratings for the resource has been accounted for;

[0063] 方式二,当一处理资源的当前资源状况表示该处理资源的已用资源占比大于所述设定阈值时,将该处理资源的评分确定为零分。 [0063] The second approach, when a current resource situation of the processing resources indicates processing resources when the set threshold, determining a score of the processing resources is greater than zero resource accounting.

[0064] 在上述确定处理资源的评分的方式中,当处理资源的已用资源占比比较大时,则认为该处理资源不可用,即将该处理资源的评分确定为零分。 [0064] In the score of the above-identified processing resources, the resources have been used when dealing with resource accounting is relatively large, it is considered that processing resources are not available, the processing resources is about to score determined as zero. 另外,步骤101中获取的处理资源的当前资源状况可以是该处理资源的资源已使用状况或资源可用状况,即已用资源占比或可用资源占比。 Further, in step 101 the current resource status acquiring processing resources may be resources of the processing resources used condition or status of resources available, already available resources or resource accounting accounting. 当获取的是可用资源占比时,计算1-可用资源占比即可得到已用资源占比,其中的占比可以是百分比。 When the acquired share of resources is available, calculate the proportion of available resources to obtain 1- resource has been accounted for, which may be a percentage proportion.

[0065] 需要说明的是,从前m个总分集合的每个总分集合中分别选择设定数量的总分包括:基于哈希运算,从每个总分集合中分别选择设定数量的总分。 [0065] Incidentally, each set of m out of the front of the collection were selected out of a set number of out comprising: based on a hash calculation of the total number selected from each set out of the collection, respectively Minute. 即,先按照总分从高到低的顺序排序后分段,或者按照总分从低到高的顺序排序后分段。 That is, the first sorted out according to a descending order segment, or segment out according to the order from low to high after sorting. 分段后,从每个总分集合中选择设定数量的总分时可以采用随机哈希运算进行选择。 After segmentation, may be employed to select a random hash operation is selected out of a set number of scores from each set. 在一个总分集合中,尽管各个总分值不完全相同,但是非常接近,因此,从该总分集合中可随机选择设定数量的总分。 In one set out, although individual total score are not identical, but very close, therefore, may be randomly selected from a set number of out out the collection. 除了哈希运算,也可以采用其他随机选择算法。 Hashing addition, other random selection algorithm may also be used.

[0066] 在上述步骤102之前还包括:根据所述处理资源组合的属性,为所述处理资源组合的多个处理资源设置相应的权重。 [0066] further comprising, prior to step 102: according to the attributes of the combined processing resources, the processing corresponding to the plurality of processing resources provided weights resource combination weight. 即,在计算每个处理资源组合的总分之前,还需要为各处理资源设置权重。 That is, before score calculation for each combination of processing resources, we need to re-set the right each processing resources. 处理资源组合的属性可以是,例如,计算性、数据写入、数据读取。 Attribute processing resource combination may be, for example, computational, data writing, data reading. 其中计算性资源组合中,CPU、内存的优先级高,即权重大;数据写入资源组合中,磁盘1/0、网络I/O 的优先级高,即权重大;数据读取资源组合中,网络I/O、磁盘I/O、网络I/O的优先级高,即权重大。 Wherein the combination computing resources, CPU, memory, high-priority, i.e., significant weight; resource combination data is written, the disk 1/0, high network I / O priority, i.e., significant weight; resource combination data read high network I / O, disk I / O, network I / O priorities, namely the right major. 下表列出了一个示例性实施例中各属性对应的处理资源的权重。 The following table shows an embodiment of processing resources each attribute weights corresponding to the weight of the exemplary embodiment.

[0067] 表1 [0067] TABLE 1

Figure CN107704320AD00091

[0069] 这种对各处理资源的评分进行加权求和的方法,基于处理资源组合的属性,为各处理资源设置合适的权重,计算该处理资源组合的合理的总分,该总分代表该处理资源组合的处理能力,从而依据各处理资源组合的总分选择处理能力较强的处理资源组合,来分配任务。 [0069] This method is a weighted sum of scores of each processing resources, processing resources based on the combined properties, for the right to set the appropriate processing resource weight calculated out of the reasonable process resource combination, the score representing the processing capability combined resources to select strong capabilities combined processing resources based on the total score of each combination of processing resources to assign tasks. 这样,能够有效地利用分布式系统的资源,提高分布式系统的处理效率。 Thus, resources can be effectively utilized in a distributed system, improve the processing efficiency of the distributed system.

[0070] 上述处理资源至少包括:CPU、内存、网络输入/输出、磁盘输入/输出、磁盘容量。 [0070] The processing resource comprising at least: CPU, memory, network I / O, disk I / O, disk capacity.

[0071] 另外,各处理资源会周期性地上报自己的当前资源状况,例如每分钟上报一次,以便系统可以根据实时的数据进行计算以及任务分配。 [0071] Further, each processing resources periodically report their current resource situation, for example, reports once per minute, so that the system can be calculated and task allocation according to real-time data.

[0072] 下面给出根据本发明的分布式系统的任务分配方法的具体实施例。 [0072] According to a particular embodiment given below task allocation method for a distributed system of the present invention. 在该实施例中,分布式系统包括3个服务器,每个服务器包括4个磁盘,S卩4个处理资源组合,处理资源包括CPU、内存、网络输入/输出、磁盘输入/输出、磁盘容量。 In this embodiment, the distributed system comprises three servers, each comprising a disk 4, S 4 Jie combination processing resources, processing resources, including CPU, memory, network I / O, disk I / O, disk capacity. 其中,服务器一为用于数据读取的服务器,服务器二为用于数据写入的服务器,服务器三为用于数据计算的服务器。 Wherein the server is a server for data read, two servers for data write server, a server for the server three calculated data. 具体的任务分配方法如下: Specific tasks assigned as follows:

[0073] 第一步,获取3个服务器中每个磁盘的CPU、内存、网络输入/输出、磁盘输入/输出、 磁盘容量的当前资源状况。 [0073] The first step, the server 3 acquires the current resource status of each disk CPU, memory, network I / O, disk I / O, disk capacity.

[0074] 第二步,根据获取的当前资源状况确定每个磁盘的CPU、内存、、网络输入/输出、磁盘输入/输出磁盘容量的可用资源占比,并进一步确定每个磁盘的CPU、内存、网络输入/输出、磁盘输入/输出、磁盘容量的评分。 [0074] The second step, the disk is determined for each CPU, memory ,, network input / output, a disk input / output accounting for the resources available disk capacity according to the obtained current status of the resources, and further determines each disk CPU, memory , network I / O, disk I / O, disk capacity rating. 例如,确定的服务器一的4个磁盘的CPU、内存、网络输入/输出、磁盘输入/输出、磁盘容量的评分如表2所示,其中的评分为可用资源占比乘以100: For example, a server identified four disks CPU, memory, network I / O, disk I / O, disk capacity ratings shown in Table 2, where the score is multiplied by the proportion of available resources 100:

[0075] 表2 [0075] TABLE 2

Figure CN107704320AD00101

[0077] 第三步,基于每个磁盘的评分及其相应的权重,计算每个磁盘的总分。 [0077] The third step, based on each disk and its corresponding weighting score, overall score for each disk. 其中,服务器一的属性为数据读取,对于服务器一中的磁盘,设定CPU、内存、网络输入/输出、磁盘输入/输出、磁盘容量的权重分别为1 〇%、1 〇%、20 %、30 %、30 %。 Wherein, the server reads a data attribute, for a disk in the server, the set CPU, memory, network I / O, disk I / O, disk capacity right weight percent 1 billion, 1 billion%, 20% , 30%, 30%. 将各处理资源的评分与权重相乘并求和,得到磁盘一、磁盘二、磁盘三、磁盘四的总分分别为:36、79、71、55。 The score and the weight of each processing resources multiplying and summed to give a disk, two disks, disk three out of four disks were: 36,79,71,55. 以此类推, 分别计算服务器二和服务器三中各磁盘的总分。 And so, out of three in the second and each disk server servers were calculated. 其中,服务器二的属性为数据写入,其中各磁盘的CPU、内存、网络输入/输出、磁盘输入/输出、磁盘容量的权重分别为10 %、10 %、 20 %、30 %、30 % ;服务器三的属性为数据计算,其中各磁盘的CPU、内存、网络输入/输出、磁盘输入/输出、磁盘容量的权重分别为10 %、10%、40%、40%、0 %。 Wherein the server two attributes for the data writing, wherein the CPU, memory, network I / O, disk input / output, the right disk capacity of each disk weight, respectively 10%, 10%, 20%, 30%, 30%; three attribute data server computing, in which the weight of each disk CPU, memory, network I / O, disk I / O, disk capacity, weight of 10%, 10%, 40%, 40%, 0%. 例如,计算得到的服务器二中四个磁盘的总分分别为:58、76、76、37,计算得到的服务器三中四个磁盘的总分分别为:58、56、56、56。 For example, the server calculated out in two four disks are: 58,76,76,37, calculated in three out of four disk server are: 58,56,56,56.

[0078] 第四步,利用哈希运算将计算得到12个磁盘的总分排序并分段。 [0078] The fourth step, the calculated hash operation using a sort out segment 12 and the disk. 按总分从高到低排序后得到79、76、76、71、58、58、56、56、56、55、37、36。 After descending sort by total score obtained 79,76,76,71,58,58,56,56,56,55,37,36. 其中,将总分在71-80内的分值分到第1总分集合中,将总分在51-60内的分值分到第2总分集合中,将总分在31-40内的分值分到第3总分集合中。 Wherein the scores in the score assigned to the first 71-80 set out, the scores in the score of 51-60 in the collection assigned to the second score, the score in the 31-40 Total score assigned to the third set. 分段后,第1总分集合包括4个总分,第2总分集合包括6个总分,第3总分集合包括2个总分。 After segmentation, the first set comprises four score score, the second score comprises a set of six out, out of the third set comprises two scores.

[0079] 第五步,从第1总分集合中选择2个总分,从第2总分集合中选择1个总分,从第3总分集合中选择1个总分。 [0079] The fifth step, selected from the first set of two score score, a score selected from the second set of scores, selecting one out of scores from the third set. 该步骤中从各总分集合中选择总分的个数是根据大量实验得出的经验值进行设定的。 The step out from each selected out of the set number is set based on empirical values ​​of a large number of experimentally derived. 并且,在各总分集合中的选择是随机的。 And, in each selected out of the set is random.

[0080] 第六步,向与所选择的总分对应的磁盘分配任务。 [0080] The sixth step, corresponding to the selected disk out of the assigned tasks.

[0081 ]该实施例中,选择CPU、内存、网络输入/输出、磁盘输入/输出、磁盘容量的可用资源占比作为其评分,并且在对磁盘总分进行排序时,按照从高到低的顺序进行排序并分段。 [0081] When the proportion of available resources embodiment, the selected CPU, memory, network I / O, disk I / O, disk capacity rating as this embodiment, and sort out the disk, in a descending and sort order segment. 在其他实施例中,可以选择已用资源占比作为评分,按照从低到高的顺序对磁盘总分进行排序并分段。 In other embodiments, the proportion may be selected as the used resource score to rank the disk out of order from low to high and segments.

[0082]本发明还提供了一种分布式系统的任务分配系统,所述系统包括: [0082] The present invention further provides a distributed system task allocation, the system comprising:

[0083] 资源状况获取模块201,获取多个服务器包括的每个处理资源组合的多个处理资源的当前资源状况; [0083] The resource status acquiring module 201 acquires a plurality of current resource situation of processing resources for each combination of processing resources include a plurality of servers;

[0084] 总分计算模块202,用于利用获取的每个处理资源组合的多个处理资源的当前资源状况以及每个处理资源组合的多个处理资源的权重,计算每个所述处理资源组合的总分; [0084] The score computation module 202, a plurality of current resource status acquisition processing resources using a combination of resources and a plurality of weights for each processing process resources for each processing resource combination weight is calculated for each combination of the processing resource total score;

[0085] 任务分配模块203,用于基于计算的各处理资源组合的总分,选择处理资源组合, 并向所选择的处理资源组合分配处理任务。 [0085] The task allocation module 203, based on a combination of each score calculation processing resources, resource combination selection processing, combination processing resources allocated to the selected processing task.

[0086] 其中,所述总分计算模块202还用于: [0086] wherein, the score computation module 202 is further configured to:

[0087] 基于每个处理资源组合的多个处理资源的当前资源状况确定各处理资源的评分, 将每个处理资源组合中处理资源的评分乘以该处理资源的权重后得到该处理资源的加权评分,并将每个处理资源组合中各处理资源的加权评分相加,分别得到每个处理资源组合的总分。 [0087] The score for each determination processing resources based on the current resource status of the plurality of processing resources for each resource combination process, the combination processing resources each resource processing rates multiplied by the weight after re-processing resources to obtain the weighting processing resources score and weighting each process resource score for each combination of processing resources are added, respectively out of each processing resource combination.

[0088] 其中,所述任务分配模块203还用于: [0088] wherein the task assignment module 203 is further configured to:

[0089] 将各处理资源组合的总分按从高到低排序后进行分段,得到η个总分集合,第i个总分集合中的总分均大于第i+Ι个总分集合至第η个总分集合中的总分,从所述η个总分集合中选择前m个总分集合,并从所述前m个总分集合的每个总分集合中分别选择设定数量的总分,向与所选择的总分对应的处理资源组合分配处理任务,其中n、m和i均为大于0的正整数,且m<n,i<n〇 [0089] The score of each resource combination process performed by the descending segment sorted out to obtain a set of η, out of the i-th set out of a set of greater than one score to i + Ι the first set of scores η th score, score selected from the set top m out of a set of the η, the number and select from a set out of the front of each set of m out of a set of respectively the total score, the score corresponding to the selected combination of processing resources allocation processing tasks, wherein n, m and i are both positive integers greater than 0, and m <n, i <n〇

[0090] 其中,所述任务分配模块203还用于: [0090] wherein the task assignment module 203 is further configured to:

[0091] 将各处理资源组合的总分按从低到高排序后进行分段,得到η个总分集合,第i个总分集合中的总分均小于第i+Ι个总分集合至第η个总分集合中的总分,从所述η个总分集合中选择前m个总分集合,并从所述前m个总分集合的每个总分集合中分别选择设定数量的总分,向与所选择的总分对应的处理资源组合分配处理任务,其中n、m和i均为大于0的正整数,且m<n,i<n〇 [0091] Each resource combination treatment was carried out after segmentation from lowest to highest, to give a total score η set, the i-th set out of the total score is less than the first set out to th i + Ι the first set of scores η th score, score selected from the set top m out of a set of the η, the number and select from a set out of the front of each set of m out of a set of respectively the total score, the score corresponding to the selected combination of processing resources allocation processing tasks, wherein n, m and i are both positive integers greater than 0, and m <n, i <n〇

[0092] 其中,所述总分计算模块202还用于根据以下方式确定各处理资源的评分: [0092] wherein, the score calculation module 202 is further configured to determine the score for each processing resource in the following manner:

[0093] 方式一,当一处理资源的当前资源状况表示该处理资源的可用资源占比大于等于设定阈值时,将该处理资源的评分确定为该可用资源占比; [0093] one embodiment, when a current processing resources indicates the resource status accounting processing resources available resource is larger than equal to a set threshold value, the score is determined for the processing resources available resource accounting;

[0094] 方式二,当一处理资源的当前资源状况表示该处理资源的可用资源占比小于所述设定阈值时,将该处理资源的评分确定为零分。 [0094] The second approach, when a current resource situation of the resources available processing resources indicates the proportion of processing resources is less than the set threshold, the processing resource is determined as zero score.

[0095] 其中,所述总分计算模块202还用于根据以下方式确定各处理资源的评分: [0095] wherein, the score calculation module 202 is further configured to determine the score for each processing resource in the following manner:

[0096] 方式一,当一处理资源的当前资源状况表示该处理资源的已用资源占比小于等于设定阈值时,将该处理资源的评分确定为该已用资源占比; [0096] one embodiment, processing resources when a resource status indicates the current processing resource has less resource share the set threshold value, the determination processing resources ratings for the resource has been accounted for;

[0097] 方式二,当一处理资源的当前资源状况表示该处理资源的已用资源占比大于所述设定阈值时,将该处理资源的评分确定为零分。 [0097] The second approach, when a current resource situation of the processing resources indicates processing resources when the set threshold, determining a score of the processing resources is greater than zero resource accounting.

[0098] 其中,所述系统还包括: [0098] wherein said system further comprises:

[0099] 权重设置模块204,用于根据所述处理资源组合的属性,为所述处理资源组合的多个处理资源设置相应的权重。 [0099] The weight setting block 204, for the processing resource according to an attribute combination, said plurality of processing resources setting processing resource combination corresponding weights.

[0100] 其中,所述处理资源至少包括:cpu、内存、磁盘输入/输出、网络输入/输出。 [0100] wherein the processing resource comprising at least: cpu, memory, disk input / output, network input / output.

[0101] 其中,所述任务分配模块203还用于:基于哈希运算,从每个总分集合中分别选择设定数量的总分。 [0101] wherein the task assignment module 203 is further configured to: based on hashing selected out of a set number of scores from each set, respectively.

[0102] 本发明中的分布式系统的任务分配方法和系统,通过利用处理资源组合中各处理资源的评分和权重计算处理资源的总分,并将总分排序后分段,选择处理资源组合分配任务,可以有效地利用处理资源,避免部分处理资源过于繁忙,部分处理资源却过于空闲,从而提高系统的处理效率。 [0102] in the distributed system of the present invention is a method and system for allocating tasks, the weight calculation processing resources out by utilizing processing resources of the score and weight of each combination of processing resources, and to sort out segment, resource combination selection processing assign tasks, efficient use of processing resources can avoid part of the processing resources are too busy, too idle resources are part of the process, thus improving the processing efficiency of the system. 因此,采用本发明的方法和系统,可以实现资源的充分利用,节约成本,极大提高服务可用性。 Accordingly, the present invention is a method and system that can achieve full utilization of resources, cost savings, greatly improves service availability. 此外,在进行任务分配时实现自动分配,提高系统的健壮性。 In addition, automated dispensing carrying out the task assigned to improve the robustness of the system.

[0103] 上面描述的内容可以单独地或者以各种方式组合起来实施,而这些变型方式都在本发明的保护范围之内。 [0103] The above described may be used alone or in combination of embodiments in various ways, and these variations are within the scope of the present invention.

[0104] 需要说明的是,在本文中,术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的物品或者设备不仅包括那些要素,而且还包括没有明确列出的其他要素,或者是还包括为这种物品或者设备所固有的要素。 [0104] Incidentally, herein, the terms "comprises", "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that the article or device comprising a series of elements includes not only those elements, and also includes other elements not explicitly listed, or further includes elements of this article or device inherent. 在没有更多限制的情况下,由语句“包括……”限定的要素,并不排除在包括所述要素的物品或者设备中还存在另外的相同要素。 Without more constraints, by the wording "comprising ......" defining element does not exclude the existence of additional identical elements in the element or article comprising the device.

[0105] 以上实施例仅用以说明本发明的技术方案而非限制,仅仅参照较佳实施例对本发明进行了详细说明。 [0105] Example embodiments above are intended to illustrate and not limit the present invention only with reference to the preferred embodiments of the present invention has been described in detail. 本领域的普通技术人员应当理解,可以对本发明的技术方案进行修改或者等同替换,而不脱离本发明技术方案的精神和范围,均应涵盖在本发明的权利要求范围当中。 Those skilled in the art will appreciate that modifications may be made to the technical solutions of the present invention, or equivalent replacements without departing from the spirit and scope of the technical solutions of the present invention shall be encompassed in the scope of the present invention accompanying claims.

Claims (16)

  1. 1. 一种分布式系统的任务分配方法,其特征在于,所述方法包括: 步骤1,获取多个服务器包括的每个处理资源组合的多个处理资源的当前资源状况; 步骤2,利用获取的每个处理资源组合的多个处理资源的当前资源状况以及每个处理资源组合的多个处理资源的权重,计算每个所述处理资源组合的总分; 步骤3,基于计算的各处理资源组合的总分,选择处理资源组合,并向所选择的处理资源组合分配处理任务。 Task allocation method for distributed system, wherein, said method comprising: step 1, obtaining current resource condition of each processing resource combination plurality of processing resources include a plurality of servers; Step 2, using the obtained current resource status of the plurality of processing resources for each processing resource combination weights and a plurality of processing resources for each processing resource combination weight is calculated for each combination out of the processing resource; step 3, based on the respective calculation processing resources combined total score, choose a combination of processing resources, and selected resource allocation process combination processing tasks.
  2. 2. 如权利要求1所述的方法,其特征在于,所述步骤2包括: 基于每个处理资源组合的多个处理资源的当前资源状况确定各处理资源的评分,将每个处理资源组合中处理资源的评分乘以该处理资源的权重后得到该处理资源的加权评分, 并将每个处理资源组合中各处理资源的加权评分相加,分别得到每个处理资源组合的总分。 2. The method according to claim 1, wherein said step 2 comprises: determining a process resource score for each resource based on the current status of each of the plurality of processing resource combination of processing resources, processing resources each combination processing resources rates multiplied by the weight of heavy processing resources of the processing resources to obtain a weighted score, and the score for each weighting addition processing resources each resource combination process, respectively out of each processing resource combination.
  3. 3. 如权利要求2所述的方法,其特征在于,所述步骤3包括: 将各处理资源组合的总分按从高到低排序后进行分段,得到η个总分集合,第i个总分集合中的总分均大于第i + Ι个总分集合至第η个总分集合中的总分,从所述η个总分集合中选择前m个总分集合,并从所述前m个总分集合的每个总分集合中分别选择设定数量的总分,向与所选择的总分对应的处理资源组合分配处理任务,其中n、m和i均为大于O的正整数,且m<n,i<n〇 3. The method according to claim 2, wherein said step 3 comprises: the score of each combination of processing resources by segmenting the highest to lowest, to give a total score η set, the i-th Total scores were set larger than i + Ι η th through th set out of scores set out, selected from the first m out of one out of the set of η set, and from the each set of the first m out of the collection were selected out of a set number of scores, the score corresponding to the selected combination of processing resources allocation processing tasks, wherein n, m and i are greater than the positive O integers and m <n, i <n〇
  4. 4. 如权利要求2所述的方法,其特征在于,所述步骤3包括: 将各处理资源组合的总分按从低到高排序后进行分段,得到η个总分集合,第i个总分集合中的总分均小于第i + Ι个总分集合至第η个总分集合中的总分,从所述η个总分集合中选择前m个总分集合,并从所述前m个总分集合的每个总分集合中分别选择设定数量的总分,向与所选择的总分对应的处理资源组合分配处理任务,其中n、m和i均为大于O的正整数,且m<n,i<n〇 4. The method according to claim 2, wherein said step 3 comprises: the score of each combination by the processing resource is segmented from lowest to highest, to give a total score η set, the i-th out of the collection of the score is less than i + Ι η th through th set out of scores set out, selected from the first m out of one out of the set of η set, and from the each set of the first m out of the collection were selected out of a set number of scores, the score corresponding to the selected combination of processing resources allocation processing tasks, wherein n, m and i are greater than the positive O integers and m <n, i <n〇
  5. 5. 如权利要求3所述的方法,其特征在于,所述步骤2还包括根据以下方式确定各处理资源的评分: 方式一,当一处理资源的当前资源状况表示该处理资源的可用资源占比大于等于设定阈值时,将该处理资源的评分确定为该可用资源占比; 方式二,当一处理资源的当前资源状况表示该处理资源的可用资源占比小于所述设定阈值时,将该处理资源的评分确定为零分。 5. The method according to claim 3, wherein said step 2 further comprises determining rates of each processing resource according to the following manner: a mode, when a processing resource of the current resource status of available resources indicates processing resources accounted when the ratio is greater than equal to a set threshold, the score is determined for the processing resources available resource accounting; Second way, when a processing resource of said current resource status indicates the set threshold value accounting for the resources available is less than the processing resources, the processing resources of the score is determined as zero.
  6. 6. 如权利要求4所述的方法,其特征在于,所述步骤2还包括根据以下方式确定各处理资源的评分: 方式一,当一处理资源的当前资源状况表示该处理资源的已用资源占比小于等于设定阈值时,将该处理资源的评分确定为该已用资源占比; 方式二,当一处理资源的当前资源状况表示该处理资源的已用资源占比大于所述设定阈值时,将该处理资源的评分确定为零分。 6. The method according to claim 4, wherein said step 2 further comprises determining rates of each processing resource according to the following methods: Method 1, when a current resource status of processing resources indicates the resource is the resource handle when the proportion is less than equal to the set threshold value, the determination processing resource ratings for the resource has been accounted for; the proportion is greater than the resource, a processing resource when the current resource situation of the processing resources indicates the set manner two when the threshold value, the processing resource is determined as zero score.
  7. 7. 如权利要求1或2所述的方法,其特征在于,在所述步骤2之前还包括: 根据所述处理资源组合的属性,为所述处理资源组合的多个处理资源设置相应的权重。 7. The method of claim 1 or claim 2, wherein, prior to said step 2 further comprises: the processing resource according to an attribute combination, said treatment of the corresponding plurality of processing resources provided weights resource combination weight .
  8. 8. 如权利要求3或4所述的方法,其特征在于, 从所述前m个总分集合的每个总分集合中分别选择设定数量的总分包括:基于哈希运算,从每个总分集合中分别选择设定数量的总分。 8. The method of claim 3 or claim 4, characterized in that, from each set were selected out of the front of the m-th set out of a set number of out comprising: based on a hash calculation, from each They were selected out of a set of a set number of scores.
  9. 9. 一种分布式系统的任务分配系统,其特征在于,所述系统包括: 资源状况获取模块,获取多个服务器包括的每个处理资源组合的多个处理资源的当前资源状况; 总分计算模块,用于利用获取的每个处理资源组合的多个处理资源的当前资源状况以及每个处理资源组合的多个处理资源的权重,计算每个所述处理资源组合的总分; 任务分配模块,用于基于计算的各处理资源组合的总分,选择处理资源组合,并向所选择的处理资源组合分配处理任务。 A distributed system task allocation system, characterized in that, the system comprising: a resource status acquisition module acquiring a plurality of current resource status of each processing resource combination processing resources include a plurality of servers; score calculated weight means for utilizing the plurality of processing resources available resources for each combination of process conditions and the current resources of the plurality of processing resources for each processing resource combination weight is calculated for each combination out of the processing resource; task assignment module , each based on the combined score calculation processing resources, resource combination selection processing, combination processing resources allocated to the selected processing task.
  10. 10. 如权利要求9所述的系统,其特征在于,所述总分计算模块还用于: 基于每个处理资源组合的多个处理资源的当前资源状况确定各处理资源的评分,将每个处理资源组合中处理资源的评分乘以该处理资源的权重后得到该处理资源的加权评分, 并将每个处理资源组合中各处理资源的加权评分相加,分别得到每个处理资源组合的总分。 10. The system according to claim 9, wherein said score calculating module is further configured to: determine a score for each processing resource based on the current resource situation of the plurality of processing resources for each processing resource combination of each processing resources right combination of processing resources multiplied by the score processing resources obtained after the re-weighting process resource score, and the score for each weighting addition processing resources each resource combination process, respectively of the total composition of each processing resources Minute.
  11. 11. 如权利要求10所述的系统,其特征在于,所述任务分配模块还用于: 将各处理资源组合的总分按从高到低排序后进行分段,得到η个总分集合,第i个总分集合中的总分均大于第i+Ι个总分集合至第η个总分集合中的总分,从所述η个总分集合中选择前m个总分集合,并从所述前m个总分集合的每个总分集合中分别选择设定数量的总分,向与所选择的总分对应的处理资源组合分配处理任务,其中n、m和i均为大于0的正整数,且m<n,i<n〇 11. The system according to claim 10, wherein the task assignment module is further configured to: the composition of each processing resource out by segmenting the highest to lowest, to give a total score η set, the i-th set out of the total score greater than i + Ι η th through th set out of scores set out, selected from the first m η out one set out of the collection, and selecting a set number of out out from the front of each set of m out of the collection, respectively, to the selected process resource score corresponding to distribute processing composition, wherein n, m and i are greater than are positive integers and m <n, i <n〇
  12. 12. 如权利要求10所述的系统,其特征在于,所述任务分配模块还用于: 将各处理资源组合的总分按从低到高排序后进行分段,得到η个总分集合,第i个总分集合中的总分均小于第i + Ι个总分集合至第η个总分集合中的总分,从所述η个总分集合中选择前m个总分集合,并从所述前m个总分集合的每个总分集合中分别选择设定数量的总分,向与所选择的总分对应的处理资源组合分配处理任务,其中n、m和i均为大于0的正整数,且m<n,i<n〇 12. The system according to claim 10, wherein the task assignment module is further configured to: process each score is segmented by combination of resources from the lowest to highest, to give a total score η set, the i-th set out of the total score is less than i + Ι th set out of η to set a score out, selected from the first m η out one set out of the collection, and selecting a set number of out out from the front of each set of m out of the collection, respectively, to the selected process resource score corresponding to distribute processing composition, wherein n, m and i are greater than are positive integers and m <n, i <n〇
  13. 13. 如权利要求11所述的系统,其特征在于,所述总分计算模块还用于根据以下方式确定各处理资源的评分: 方式一,当一处理资源的当前资源状况表示该处理资源的可用资源占比大于等于设定阈值时,将该处理资源的评分确定为该可用资源占比; 方式二,当一处理资源的当前资源状况表示该处理资源的可用资源占比小于所述设定阈值时,将该处理资源的评分确定为零分。 13. The system of claim 11, wherein said score calculating module is further for determining a process resource score for each of the following ways: Method 1, when a current processing resources indicates the resource status processing resources when the proportion of available resources than or equal to the set threshold value, the score is determined for the processing resources available resource accounting; Second way, when a processing resource of the current resource status represents the proportion of available resources is less than the set processing resources when the threshold value, the processing resource is determined as zero score.
  14. 14. 如权利要求12所述的系统,其特征在于,所述总分计算模块还用于根据以下方式确定各处理资源的评分: 方式一,当一处理资源的当前资源状况表示该处理资源的已用资源占比小于等于设定阈值时,将该处理资源的评分确定为该已用资源占比; 方式二,当一处理资源的当前资源状况表示该处理资源的已用资源占比大于所述设定阈值时,将该处理资源的评分确定为零分。 14. The system of claim 12, wherein said score calculating module is further for determining a process resource score for each of the following ways: Method 1, when a current processing resources indicates the resource status processing resources when the proportion is less resource set threshold, the score processing resources that have been determined by the proportion of resources;, processing resources when a resource status indicates the current processing resource embodiment II is greater than the proportion of resources used when said threshold value is set, the processing resource is determined as zero score.
  15. 15. 如权利要求9或10所述的系统,其特征在于,所述系统还包括: 权重设置模块,用于根据所述处理资源组合的属性,为所述处理资源组合的多个处理资源设置相应的权重。 15. The system of claim 9 or claim 10, wherein said system further comprises: a weight setting means for the processing resource according to an attribute combination, a plurality of processing resources for the processing resources provided in combination corresponding weight.
  16. 16.如权利要求11或12所述的系统,其特征在于,所述任务分配模块还用于: 基于哈希运算,从每个总分集合中分别选择设定数量的总分。 16. The system of claim 11 or claim 12, wherein the task assignment module is further configured to: based on hashing selected out of a set number of scores from each set, respectively.
CN 201710333404 2017-05-12 2017-05-12 Kind of task allocation method and system for distributed systems CN107704320B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 201710333404 CN107704320B (en) 2017-05-12 2017-05-12 Kind of task allocation method and system for distributed systems

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 201710333404 CN107704320B (en) 2017-05-12 2017-05-12 Kind of task allocation method and system for distributed systems

Publications (2)

Publication Number Publication Date
CN107704320A true true CN107704320A (en) 2018-02-16
CN107704320B CN107704320B (en) 2018-08-17

Family

ID=61169482

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 201710333404 CN107704320B (en) 2017-05-12 2017-05-12 Kind of task allocation method and system for distributed systems

Country Status (1)

Country Link
CN (1) CN107704320B (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101351784A (en) * 2005-12-30 2009-01-21 阿西式·A·潘迪亚 Runtime adaptable search processor
CN102902587A (en) * 2011-07-28 2013-01-30 中国移动通信集团四川有限公司 Distribution type task scheduling method, distribution type task scheduling system and distribution type task scheduling device
CN105022662A (en) * 2014-04-22 2015-11-04 中国银联股份有限公司 Distributive task allocation method
CN105427171A (en) * 2015-11-30 2016-03-23 北京口袋财富信息科技有限公司 Data processing method of Internet lending platform rating
US20160226789A1 (en) * 2015-02-04 2016-08-04 Telefonaktiebolaget L M Ericsson (Publ) Method and system to rebalance constrained services in a cloud using a genetic algorithm
US20160292011A1 (en) * 2015-03-31 2016-10-06 Stitch Fix, Inc. Systems and methods for intelligently distributing tasks received from clients among a plurality of worker resources
CN106126346A (en) * 2016-07-05 2016-11-16 东北大学 Large-scale distributed data collection system and method
CN106445676A (en) * 2015-08-05 2017-02-22 杭州海康威视系统技术有限公司 Distributed data calculation-based task allocation method and task allocation apparatus

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101351784A (en) * 2005-12-30 2009-01-21 阿西式·A·潘迪亚 Runtime adaptable search processor
CN102902587A (en) * 2011-07-28 2013-01-30 中国移动通信集团四川有限公司 Distribution type task scheduling method, distribution type task scheduling system and distribution type task scheduling device
CN105022662A (en) * 2014-04-22 2015-11-04 中国银联股份有限公司 Distributive task allocation method
US20160226789A1 (en) * 2015-02-04 2016-08-04 Telefonaktiebolaget L M Ericsson (Publ) Method and system to rebalance constrained services in a cloud using a genetic algorithm
US20160292011A1 (en) * 2015-03-31 2016-10-06 Stitch Fix, Inc. Systems and methods for intelligently distributing tasks received from clients among a plurality of worker resources
CN106445676A (en) * 2015-08-05 2017-02-22 杭州海康威视系统技术有限公司 Distributed data calculation-based task allocation method and task allocation apparatus
CN105427171A (en) * 2015-11-30 2016-03-23 北京口袋财富信息科技有限公司 Data processing method of Internet lending platform rating
CN106126346A (en) * 2016-07-05 2016-11-16 东北大学 Large-scale distributed data collection system and method

Also Published As

Publication number Publication date Type
CN107704320B (en) 2018-08-17 grant

Similar Documents

Publication Publication Date Title
Sharma et al. Performance analysis of load balancing algorithms
Downey Predicting queue times on space-sharing parallel computers
Chen et al. Effective VM sizing in virtualized data centers.
Bichler et al. Capacity planning for virtualized servers
US20120198466A1 (en) Determining an allocation of resources for a job
Lee et al. Heterogeneity-Aware Resource Allocation and Scheduling in the Cloud.
US20070169127A1 (en) Method, system and computer program product for optimizing allocation of resources on partitions of a data processing system
US20090313635A1 (en) System and/or method for balancing allocation of data among reduce processes by reallocation
Zhang et al. Improving data locality of MapReduce by scheduling in homogeneous computing environments
US20140282540A1 (en) Performant host selection for virtualization centers
US20060161920A1 (en) Method, system, and computer program for managing a queuing system
Rao et al. QoS guarantees and service differentiation for dynamic cloud applications
US20130060834A1 (en) Distributed messaging system connectivity and resource management
US7036123B2 (en) System using fair-share scheduling technique to schedule processes within each processor set based on the number of shares assigned to each process group
US20140059207A1 (en) Client placement in a computer network system using dynamic weight assignments on resource utilization metrics
US20060031242A1 (en) Method, system, and program for distributing application transactions among work servers
US20100251253A1 (en) Priority-based management of system load level
US20050177833A1 (en) Method and apparatus for reassigning objects to processing units
EP1533699A1 (en) A method of assigning objects to processing units
US20060020767A1 (en) Data processing system and method for assigning objects to processing units
Rajguru et al. A comparative performance analysis of load balancing algorithms in distributed system using qualitative parameters
US8434088B2 (en) Optimized capacity planning
Jung et al. Agent-based adaptive resource allocation on the cloud computing environment
US20140165070A1 (en) Ranking and scheduling of monitoring tasks
CN103150263A (en) Hierarchical storage method

Legal Events

Date Code Title Description
PB01
SE01
GR01