CN107704320A - The method for allocating tasks and system of a kind of distributed system - Google Patents
The method for allocating tasks and system of a kind of distributed system Download PDFInfo
- Publication number
- CN107704320A CN107704320A CN201710333404.6A CN201710333404A CN107704320A CN 107704320 A CN107704320 A CN 107704320A CN 201710333404 A CN201710333404 A CN 201710333404A CN 107704320 A CN107704320 A CN 107704320A
- Authority
- CN
- China
- Prior art keywords
- total score
- resource
- process resource
- combination
- scoring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5044—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering hardware capabilities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/501—Performance criteria
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a kind of method for allocating tasks of distributed system and system.This method includes：Step 1, the Current resource situation of the multiple process resources for each process resource combination that multiple servers include is obtained；Step 2, the weight of the Current resource situation of the multiple process resources combined using each process resource of acquisition and multiple process resources of each process resource combination, the total score of each process resource combination is calculated；Step 3, the total score of each processing combination of resources based on calculating, process resource combination is selected, and processing task is distributed to selected process resource combination.The inventive method and system can realize making full use of for resource, cost-effective, and service availability is greatly improved.In addition, realizing automatic distribution when carrying out task distribution, the robustness of system is improved.
Description
Technical field
The present invention relates to the method for allocating tasks and system of data processing field, more particularly to a kind of distributed system.
Background technology
Distributed system is usually directed to multiple servers, includes multiple disks on each server.Distributed system needs
The data volume of processing is typically very big, is handled, can caused if assigning data to some or certain several disks
Some disks are constantly in busy state, and some disks are then constantly in idle condition, so as to can not reasonably utilize service
Process resource on device.
Therefore, when being written and read by distributed system to data, the operation such as handling, it is necessary to which one kind can be utilized rationally
Process resource on each server, improve the method and system of server process efficiency.
The content of the invention
In order to solve the problems, such as in the prior art to the processing resource allocation task on server, it is proposed that one kind is used to divide
The method for allocating tasks and system of cloth system.
According to an aspect of the invention, there is provided a kind of method for allocating tasks of distributed system, methods described include：
Step 1, the Current resource shape of the multiple process resources for each process resource combination that multiple servers include is obtained
Condition；
Step 2, the Current resource situation of the multiple process resources combined using each process resource of acquisition and each
The weight of multiple process resources of process resource combination, calculate the total score of each process resource combination；
Step 3, the total score of each processing combination of resources based on calculating, process resource combination is selected, and to selected place
Manage combination of resources distribution processing task.
Wherein, the step 2 includes：
The Current resource situation of multiple process resources based on the combination of each process resource determines the scoring of each process resource,
The scoring of process resource, which is multiplied by after the weight of the process resource, during each process resource is combined obtains the weighting of the process resource
Scoring, and the weighted scoring of each process resource in the combination of each process resource is added, respectively obtain each process resource combination
Total score.
Wherein, the step 3 includes：
By the total score of each processing combination of resources by being segmented after sorting from high to low, n total score set is obtained, i-th
Total score in total score set is all higher than the total score that i+1 total diversity is bonded in n-th of total score set, from the n total diversity
M total score set before being selected in conjunction, and selection sets quantity respectively from each total score set of the preceding m total score set
Total score, to selected total score corresponding to process resource combine distribution processing task, wherein n, m and i is just whole more than 0
Number, and m≤n, i≤n.
Wherein, the step 3 includes：
By the total score of each processing combination of resources by being segmented after sorting from low to high, n total score set is obtained, i-th
Total score in total score set is respectively less than the total score that i+1 total diversity is bonded in n-th of total score set, from the n total diversity
M total score set before being selected in conjunction, and selection sets quantity respectively from each total score set of the preceding m total score set
Total score, to selected total score corresponding to process resource combine distribution processing task, wherein n, m and i is just whole more than 0
Number, and m≤n, i≤n.
Wherein, the step 2 also includes the scoring that each process resource is determined according in the following manner：
Mode one, when the Current resource situation of a process resource represents that the available resources accounting of the process resource is more than or equal to
During given threshold, the scoring of the process resource is defined as the available resources accounting；
Mode two, when the Current resource situation of a process resource represents the available resources accounting of the process resource less than described
During given threshold, the scoring of the process resource is defined as zero.
Wherein, the step 2 also includes the scoring that each process resource is determined according in the following manner：
Mode one, when the Current resource situation of a process resource represents being less than or equal to resource accounting for the process resource
During given threshold, the scoring of the process resource is defined as this and uses resource accounting；
Mode two, when the Current resource situation of a process resource represents that being more than with resource accounting for the process resource is described
During given threshold, the scoring of the process resource is defined as zero.
Wherein, also include before the step 2：
The attribute combined according to the process resource, multiple process resources for process resource combination are set accordingly
Weight.
Wherein, the total score of selection setting quantity includes respectively from each total score set of the preceding m total score set：Base
In Hash operation, selection sets the total score of quantity respectively from each total score set.
According to another aspect of the present invention, a kind of task distribution system of distributed system, the system bag are additionally provided
Include：
Resource situation acquisition module, obtain the multiple process resources for each process resource combination that multiple servers include
Current resource situation；
Total score computing module, the Current resource shape for multiple process resources using each process resource combination obtained
The weight of condition and multiple process resources of each process resource combination, calculate the total score of each process resource combination；
Task allocating module, for the total score of each processing combination of resources based on calculating, process resource is selected to combine, and to
Selected process resource combination distribution processing task.
Wherein, the total score computing module is additionally operable to：
The Current resource situation of multiple process resources based on the combination of each process resource determines the scoring of each process resource,
The scoring of process resource, which is multiplied by after the weight of the process resource, during each process resource is combined obtains the weighting of the process resource
Scoring, and the weighted scoring of each process resource in the combination of each process resource is added, respectively obtain each process resource combination
Total score.
Wherein, the task allocating module is additionally operable to：
By the total score of each processing combination of resources by being segmented after sorting from high to low, n total score set is obtained, i-th
Total score in total score set is all higher than the total score that i+1 total diversity is bonded in n-th of total score set, from the n total diversity
M total score set before being selected in conjunction, and selection sets quantity respectively from each total score set of the preceding m total score set
Total score, to selected total score corresponding to process resource combine distribution processing task, wherein n, m and i is just whole more than 0
Number, and m≤n, i≤n.
Wherein, the task allocating module is additionally operable to：
By the total score of each processing combination of resources by being segmented after sorting from low to high, n total score set is obtained, i-th
Total score in total score set is respectively less than the total score that i+1 total diversity is bonded in n-th of total score set, from the n total diversity
M total score set before being selected in conjunction, and selection sets quantity respectively from each total score set of the preceding m total score set
Total score, to selected total score corresponding to process resource combine distribution processing task, wherein n, m and i is just whole more than 0
Number, and m≤n, i≤n.
Wherein, the total score computing module is additionally operable to determine the scoring of each process resource according in the following manner：
Mode one, when the Current resource situation of a process resource represents that the available resources accounting of the process resource is more than or equal to
During given threshold, the scoring of the process resource is defined as the available resources accounting；
Mode two, when the Current resource situation of a process resource represents the available resources accounting of the process resource less than described
During given threshold, the scoring of the process resource is defined as zero.
Wherein, the total score computing module is additionally operable to determine the scoring of each process resource according in the following manner：
Mode one, when the Current resource situation of a process resource represents being less than or equal to resource accounting for the process resource
During given threshold, the scoring of the process resource is defined as this and uses resource accounting；
Mode two, when the Current resource situation of a process resource represents that being more than with resource accounting for the process resource is described
During given threshold, the scoring of the process resource is defined as zero.
Wherein, the system also includes：
Weight setting module, for the attribute combined according to the process resource, for the multiple of process resource combination
Process resource sets corresponding weight.
Wherein, the task allocating module is additionally operable to：
Based on Hash operation, selection sets the total score of quantity respectively from each total score set.
The method for allocating tasks and system of distributed system in the present invention, are respectively handled in being combined by using process resource
The scoring of resource and the total score of weight calculation process resource, and be segmented after total score is sorted, select process resource combination distribution to appoint
Business, can effectively utilize process resource, avoid part process resource excessively busy, and part process resource is but excessively idle, from
And improve the treatment effeciency of system.Therefore, using the method and system of the present invention, it is possible to achieve resource makes full use of, and saves
Cost, service availability is greatly improved.In addition, realizing automatic distribution when carrying out task distribution, the robustness of system is improved.
Brief description of the drawings
The accompanying drawing for forming the part of the present invention is used for providing a further understanding of the present invention, schematic reality of the invention
Apply example and its illustrate to be used to explain the present invention, do not form inappropriate limitation of the present invention.In the accompanying drawings：
Fig. 1 is the method for allocating tasks flow chart according to the distributed system of the present invention；
Fig. 2 is the task distribution system module map according to the distributed system of the present invention.
Embodiment
To make the purpose, technical scheme and advantage of the embodiment of the present invention clearer, below in conjunction with the embodiment of the present invention
In accompanying drawing, the technical scheme in the embodiment of the present invention is clearly and completely described, it is clear that described embodiment is
Part of the embodiment of the present invention, rather than whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art
The every other embodiment obtained under the premise of creative work is not made, belongs to the scope of protection of the invention.Need
Illustrate, in the case where not conflicting, the feature in embodiment and embodiment in the application can be mutually combined.
The invention provides a kind of method for allocating tasks of distributed system, as described in Figure 1, this method includes：
Step 101, the Current resource of the multiple process resources for each process resource combination that multiple servers include is obtained
Situation；
Step 102, the Current resource situation of the multiple process resources combined using each process resource of acquisition and every
The weight of multiple process resources of individual process resource combination, calculate the total score of each process resource combination；
Step 103, the total score of each processing combination of resources based on calculating, process resource combination is selected, and to selected
Process resource combination distribution processing task.
Above-mentioned steps 102 include：The Current resource situation of multiple process resources based on the combination of each process resource determines
The scoring of each process resource, obtained after the scoring of process resource in the combination of each process resource is multiplied by into the weight of the process resource
The weighted scoring of the process resource, and the weighted scoring of each process resource in the combination of each process resource is added, respectively obtain
The total score of each process resource combination.That is, the total score of process resource is by the scoring of each process resource and the progress of corresponding weight
It is calculated.
Above-mentioned steps 103 include：It is segmented after the total score of each processing combination of resources is sorted by height, obtains n always
Divide set, the total score in i-th of total score set is all higher than the total score that i+1 total diversity is bonded in n-th of total score set, from institute
M total score set before being selected in n total score set is stated, and is selected respectively from each total score set of the preceding m total score set
Select the total score of setting quantity, to selected total score corresponding to process resource combine and distributes processing task, wherein n, m and i be equal
For the positive integer more than 0, and m≤n, i≤n.
That is, it is segmented after the total score of each processing combination of resources is sorted by height, multiple total score set is obtained, for example, the 1st is total
Point set, the 2nd total score set ..., the n-th total score set.The total score come in total score set above is all higher than coming below
Total score in total score set.It is not all from the most forward total score set of ranking in addition, when selecting process resource combination
Selection, but several total score set in the top are selected first, then selection sets number from selected total score set respectively
The total score of amount.Because the total score calculated in a step 102 is calculated according to the Current resource situation of each processing resource reporting
, and the resource behaviour in service of each process resource may change in real time, that is, the total score calculated may adjust in real time, at this
In the case of kind, if all selected from the most forward total score set of ranking, this distribution method is not necessarily optimal
's.And this sequencingsegmenting of the present invention is used, and select the method for multiple total score respectively from multiple set, it can be effectively ensured
The real-time optimization of process resource utilizes.
In this case, step 102 also includes the scoring that each process resource is determined according in the following manner：
Mode one, when the Current resource situation of a process resource represents that the available resources accounting of the process resource is more than or equal to
During given threshold, the scoring of the process resource is defined as the available resources accounting；
Mode two, when the Current resource situation of a process resource represents the available resources accounting of the process resource less than described
During given threshold, the scoring of the process resource is defined as zero.
In the mode of the scoring of above-mentioned determination process resource, when the available resources accounting of process resource is smaller, then
Think that the process resource is unavailable, will the scoring of the process resource be defined as zero.In addition, the processing obtained in step 101
The Current resource situation of resource can be that behaviour in service or resource can use situation for the resource of the process resource, i.e., accounted for resource
Than or available resources accounting.When acquisition be to use resource accounting when, calculate 1- with resource accounting i.e. can obtain available resources
Accounting, accounting therein can be percentage.
In addition, above-mentioned steps 103 can also include：By it is each processing combination of resources total score by from low to high sort after carry out
Segmentation, obtains n total score set, the total score in i-th of total score set is respectively less than i+1 total diversity and is bonded to n-th of total diversity
Total score in conjunction, from the n total score set select before m total score set, and from the preceding m total score set it is each always
Point set in respectively selection setting quantity total score, to selected total score corresponding to process resource combine distribution processing times
Business, wherein n, m and i are the positive integer more than 0, and m≤n, i≤n.The sort method and method phase from high to low above
Instead, it is because the scoring of used process resource is to use resource accounting, i.e., has used resource accounting the smaller the better.
In this case, step 102 also includes the scoring that each process resource is determined according in the following manner：
Mode one, when the Current resource situation of a process resource represents being less than or equal to resource accounting for the process resource
During given threshold, the scoring of the process resource is defined as this and uses resource accounting；
Mode two, when the Current resource situation of a process resource represents that being more than with resource accounting for the process resource is described
During given threshold, the scoring of the process resource is defined as zero.
In the mode of the scoring of above-mentioned determination process resource, when process resource with resource accounting it is bigger when, then
Think that the process resource is unavailable, will the scoring of the process resource be defined as zero.In addition, the processing obtained in step 101
The Current resource situation of resource can be that behaviour in service or resource can use situation for the resource of the process resource, i.e., accounted for resource
Than or available resources accounting.When acquisition be available resources accounting when, calculate 1- available resources accountings and used resource
Accounting, accounting therein can be percentage.
It should be noted that total subpackage of selection setting quantity respectively from each total score set of preceding m total score set
Include：Based on Hash operation, selection sets the total score of quantity respectively from each total score set.That is, first according to total score from high to low
Order sequence after be segmented, or according to total score from low to high order sequence after be segmented.After segmentation, from each total score set
It can be selected during the total score of selection setting quantity using random Harsh computing.In a total score set, although each total
Score value is incomplete same, but closely, therefore, the total score of setting quantity can be randomly choosed from the total score set.Except
Hash operation, other stochastic selection algorithms can also be used.
Also include before above-mentioned steps 102：The attribute combined according to the process resource, combined for the process resource
Multiple process resources corresponding weight is set.That is, before the total score of each process resource combination is calculated, it is also necessary to for everywhere
Manage resource and weight is set.The attribute of process resource combination can be, for example, computational, data write-in, digital independent.It is fallen into a trap
In the property calculated combination of resources, CPU, the priority of internal memory are high, i.e., weight is big；In data write-in combination of resources, magnetic disc i/o, network I/O
Priority it is high, i.e., weight is big；In digital independent combination of resources, network I/O, magnetic disc i/o, network I/O priority are high, that is, weigh
It is great.Following table lists the weight of process resource corresponding to each attribute in an exemplary embodiment.
Table 1
| CPU weight | Internal memory weight | Network I/O weights | Magnetic disc i/o weight | Disk size weight | |
| It is computational | 50 | 40 | 10 | 0 | 0 |
| Data write | 10 | 10 | 20 | 30 | 30 |
| Digital independent | 10 | 10 | 40 | 40 | 0 |
The method that this scoring to each process resource is weighted summation, it is each based on the attribute of process resource combination
Process resource sets suitable weight, calculates the rational total score of process resource combination, and the total score represents the process resource group
The disposal ability of conjunction, so as to the stronger process resource combination of total component selections disposal ability according to each processing combination of resources, to divide
With task.In such manner, it is possible to effectively utilize the resource of distributed system, the treatment effeciency of distributed system is improved.
Above-mentioned process resource comprises at least：CPU, internal memory, network inputs/output, disk input/output, disk size.
In addition, each process resource can periodically report the Current resource situation of oneself, for example, it is per minute report once, with
Just system can be calculated according to real-time data and task is distributed.
The specific embodiment of the method for allocating tasks of the distributed system according to the present invention is given below.In the embodiment
In, distributed system includes 3 servers, and each server includes 4 disks, i.e. 4 process resource combinations, process resource bag
Include CPU, internal memory, network inputs/output, disk input/output, disk size.Wherein, server one is for digital independent
Server, server two are the server for data write-in, and server three is the server calculated for data.It is specific to appoint
Distribution method of being engaged in is as follows：
The first step, obtain the CPU of each disk in 3 servers, internal memory, network inputs/output, disk input/output,
The Current resource situation of disk size.
Second step, according to the Current resource situation of acquisition determine the CPU of each disk, internal memory, network inputs/output, magnetic
The available resources accounting of disk input/output disk size, and further determine that the CPU, internal memory, network inputs of each disk/defeated
Go out, disk input/output, the scoring of disk size.For example, CPU, internal memory, the network of 4 disks of the server one determined are defeated
Enter/export, disk input/output, the scoring of disk size it is as shown in table 2, scoring therein is multiplied by for available resources accounting
100：
Table 2
| CPU scores | Internal memory scores | Network I/O scores | Magnetic disc i/o scores | Disk size scores | |
| Disk one | 80 | 40 | 20 | 40 | 100 |
| Disk two | 50 | 60 | 70 | 100 | 70 |
| Disk three | 50 | 60 | 70 | 80 | 90 |
| Disk four | 50 | 60 | 70 | 40 | 0 |
3rd step, scoring and its corresponding weight based on each disk, calculate the total score of each disk.Wherein, service
The attribute of device one is digital independent, and for the disk in server one, setting CPU, internal memory, network inputs/output, disk are defeated
Enter/export, the weight of disk size is respectively 10%, 10%, 20%, 30%, 30%.By the scoring of each process resource and weight
It is multiplied and sums, obtaining disk one, disk two, disk three, the total score of disk four is respectively：36、79、71、55.By that analogy,
The total score of each disk in difference calculation server two and server three.Wherein, the attribute of server two writes for data, wherein respectively
The CPU of disk, internal memory, network inputs/output, disk input/output, the weight of disk size are respectively 10%, 10%,
20%th, 30%, 30%；The attribute of server three calculates for data, wherein the CPU of each disk, internal memory, network inputs/output, magnetic
Disk input/output, the weight of disk size are respectively 10%, 10%, 40%, 40%, 0%.For example, the server being calculated
The total score of four disks is respectively in two：58th, 76,76,37, the total score difference of four disks in the server three being calculated
For：58、56、56、56.
4th step, the total score that 12 disks are calculated is sorted and is segmented using Hash operation.By total score from high to low
79,76,76,71,58,58,56,56,56,55,37,36 are obtained after sequence.Wherein, score value of the total score in 71-80 is assigned to
In 1st total score set, score value of the total score in 51-60 is assigned in the 2nd total score set, by the total score score value in 31-40 point
Into the 3rd total score set.After segmentation, the 1st total score set includes 4 total scores, and the 2nd total score set includes 6 total scores, the 3rd total score
Set includes 2 total scores.
5th step, 2 total scores are selected from the 1st total score set, 1 total score is selected from the 2nd total score set, it is total from the 3rd
Divide and 1 total score is selected in set.The number of total score is selected to be drawn according to many experiments from each total score set in the step
What empirical value was set.Also, the selection in each total score set is random.
6th step, task is distributed to disk corresponding with selected total score.
In the embodiment, CPU, internal memory, network inputs/output, disk input/output, the available money of disk size are selected
Source accounting scores as it, and when being ranked up to disk total score, is ranked up and is segmented according to order from high to low.
In other embodiments, can select to carry out disk total score according to order from low to high by the use of resource accounting as scoring
Sort and be segmented.
Present invention also offers a kind of task distribution system of distributed system, the system includes：
Resource situation acquisition module 201, obtain the multiple processing money for each process resource combination that multiple servers include
The Current resource situation in source；
Total score computing module 202, the current money for multiple process resources using each process resource combination obtained
The weight of source situation and multiple process resources of each process resource combination, calculate the total of each process resource combination
Point；
Task allocating module 203, for the total score of each processing combination of resources based on calculating, process resource combination is selected,
And distribute processing task to selected process resource combination.
Wherein, the total score computing module 202 is additionally operable to：
The Current resource situation of multiple process resources based on the combination of each process resource determines the scoring of each process resource,
The scoring of process resource, which is multiplied by after the weight of the process resource, during each process resource is combined obtains the weighting of the process resource
Scoring, and the weighted scoring of each process resource in the combination of each process resource is added, respectively obtain each process resource combination
Total score.
Wherein, the task allocating module 203 is additionally operable to：
By the total score of each processing combination of resources by being segmented after sorting from high to low, n total score set is obtained, i-th
Total score in total score set is all higher than the total score that i+1 total diversity is bonded in n-th of total score set, from the n total diversity
M total score set before being selected in conjunction, and selection sets quantity respectively from each total score set of the preceding m total score set
Total score, to selected total score corresponding to process resource combine distribution processing task, wherein n, m and i is just whole more than 0
Number, and m≤n, i≤n.
Wherein, the task allocating module 203 is additionally operable to：
By the total score of each processing combination of resources by being segmented after sorting from low to high, n total score set is obtained, i-th
Total score in total score set is respectively less than the total score that i+1 total diversity is bonded in n-th of total score set, from the n total diversity
M total score set before being selected in conjunction, and selection sets quantity respectively from each total score set of the preceding m total score set
Total score, to selected total score corresponding to process resource combine distribution processing task, wherein n, m and i is just whole more than 0
Number, and m≤n, i≤n.
Wherein, the total score computing module 202 is additionally operable to determine the scoring of each process resource according in the following manner：
Mode one, when the Current resource situation of a process resource represents that the available resources accounting of the process resource is more than or equal to
During given threshold, the scoring of the process resource is defined as the available resources accounting；
Mode two, when the Current resource situation of a process resource represents the available resources accounting of the process resource less than described
During given threshold, the scoring of the process resource is defined as zero.
Wherein, the total score computing module 202 is additionally operable to determine the scoring of each process resource according in the following manner：
Mode one, when the Current resource situation of a process resource represents being less than or equal to resource accounting for the process resource
During given threshold, the scoring of the process resource is defined as this and uses resource accounting；
Mode two, when the Current resource situation of a process resource represents that being more than with resource accounting for the process resource is described
During given threshold, the scoring of the process resource is defined as zero.
Wherein, the system also includes：
Weight setting module 204, for the attribute combined according to the process resource, for the more of process resource combination
Individual process resource sets corresponding weight.
Wherein, the process resource comprises at least：CPU, internal memory, disk input/output, network inputs/output.
Wherein, the task allocating module 203 is additionally operable to：Based on Hash operation, selected respectively from each total score set
Set the total score of quantity.
The method for allocating tasks and system of distributed system in the present invention, are respectively handled in being combined by using process resource
The scoring of resource and the total score of weight calculation process resource, and be segmented after total score is sorted, select process resource combination distribution to appoint
Business, can effectively utilize process resource, avoid part process resource excessively busy, and part process resource is but excessively idle, from
And improve the treatment effeciency of system.Therefore, using the method and system of the present invention, it is possible to achieve resource makes full use of, and saves
Cost, service availability is greatly improved.In addition, realizing automatic distribution when carrying out task distribution, the robustness of system is improved.
Descriptions above can combine implementation individually or in a variety of ways, and these variants all exist
Within protection scope of the present invention.
It should be noted that herein, term " comprising ", "comprising" or its any other variant are intended to non-row
His property includes, so that article or equipment including a series of elements not only include those key elements, but also including not
There is an other element being expressly recited, or also include for this article or the intrinsic key element of equipment.Do not limiting more
In the case of system, the key element that is limited by sentence " including ... ", it is not excluded that in the article including the key element or equipment
Other identical element also be present.
The above embodiments are merely illustrative of the technical solutions of the present invention and it is unrestricted, reference only to preferred embodiment to this hair
It is bright to be described in detail.It will be understood by those within the art that technical scheme can be modified
Or equivalent substitution, without departing from the spirit and scope of technical solution of the present invention, the claim model in the present invention all should be covered
Among enclosing.
Claims (16)
1. a kind of method for allocating tasks of distributed system, it is characterised in that methods described includes：
Step 1, the Current resource situation of the multiple process resources for each process resource combination that multiple servers include is obtained；
Step 2, the Current resource situation of the multiple process resources combined using each process resource of acquisition and each processing
The weight of multiple process resources of combination of resources, calculate the total score of each process resource combination；
Step 3, the total score of each processing combination of resources based on calculating, process resource combination is selected, and is provided to selected processing
Source combination distribution processing task.
2. the method as described in claim 1, it is characterised in that the step 2 includes：
The Current resource situation of multiple process resources based on the combination of each process resource determines the scoring of each process resource, will be every
The scoring of process resource obtains the weighted scoring of the process resource after being multiplied by the weight of the process resource in individual process resource combination,
And the weighted scoring of each process resource is added during each process resource is combined, the total of each process resource combination is respectively obtained
Point.
3. method as claimed in claim 2, it is characterised in that the step 3 includes：
By the total score of each processing combination of resources by being segmented after sorting from high to low, n total score set, i-th of total score are obtained
Total score in set is all higher than the total score that i+1 total diversity is bonded in n-th of total score set, from the n total score set
M total score set before selection, and selection sets the total of quantity respectively from each total score set of the preceding m total score set
Point, to selected total score corresponding to process resource combine distribute processing task, wherein n, m and i is just whole more than 0
Number, and m≤n, i≤n.
4. method as claimed in claim 2, it is characterised in that the step 3 includes：
By the total score of each processing combination of resources by being segmented after sorting from low to high, n total score set, i-th of total score are obtained
Total score in set is respectively less than the total score that i+1 total diversity is bonded in n-th of total score set, from the n total score set
M total score set before selection, and selection sets the total of quantity respectively from each total score set of the preceding m total score set
Point, to selected total score corresponding to process resource combine distribute processing task, wherein n, m and i is just whole more than 0
Number, and m≤n, i≤n.
5. method as claimed in claim 3, it is characterised in that the step 2 also includes determining each processing according in the following manner
The scoring of resource：
Mode one, when the Current resource situation of a process resource represents that the available resources accounting of the process resource is more than or equal to setting
During threshold value, the scoring of the process resource is defined as the available resources accounting；
Mode two, when the Current resource situation of a process resource represents that the available resources accounting of the process resource is less than the setting
During threshold value, the scoring of the process resource is defined as zero.
6. method as claimed in claim 4, it is characterised in that the step 2 also includes determining each processing according in the following manner
The scoring of resource：
Mode one, when the Current resource situation of a process resource represent the process resource with resource accounting be less than or equal to setting
During threshold value, the scoring of the process resource is defined as this and uses resource accounting；
Mode two, when what the Current resource situation of a process resource represented the process resource is more than the setting with resource accounting
During threshold value, the scoring of the process resource is defined as zero.
7. method as claimed in claim 1 or 2, it is characterised in that also include before the step 2：
The attribute combined according to the process resource, corresponding power is set for multiple process resources of process resource combination
Weight.
8. the method as described in claim 3 or 4, it is characterised in that
The total score of selection setting quantity includes respectively from each total score set of the preceding m total score set：Transported based on Hash
Calculate, selection sets the total score of quantity respectively from each total score set.
9. the task distribution system of a kind of distributed system, it is characterised in that the system includes：
Resource situation acquisition module, obtain each process resource combination that multiple servers include multiple process resources it is current
Resource situation；
Total score computing module, for using obtain each process resource combination multiple process resources Current resource situation with
And the weight of multiple process resources of each process resource combination, calculate the total score that each process resource combines；
Task allocating module, for the total score of each processing combination of resources based on calculating, select process resource combination, and to selected
The process resource combination distribution processing task selected.
10. system as claimed in claim 9, it is characterised in that the total score computing module is additionally operable to：
The Current resource situation of multiple process resources based on the combination of each process resource determines the scoring of each process resource, will be every
The scoring of process resource obtains the weighted scoring of the process resource after being multiplied by the weight of the process resource in individual process resource combination,
And the weighted scoring of each process resource is added during each process resource is combined, the total of each process resource combination is respectively obtained
Point.
11. system as claimed in claim 10, it is characterised in that the task allocating module is additionally operable to：
By the total score of each processing combination of resources by being segmented after sorting from high to low, n total score set, i-th of total score are obtained
Total score in set is all higher than the total score that i+1 total diversity is bonded in n-th of total score set, from the n total score set
M total score set before selection, and selection sets the total of quantity respectively from each total score set of the preceding m total score set
Point, to selected total score corresponding to process resource combine distribute processing task, wherein n, m and i is just whole more than 0
Number, and m≤n, i≤n.
12. system as claimed in claim 10, it is characterised in that the task allocating module is additionally operable to：
By the total score of each processing combination of resources by being segmented after sorting from low to high, n total score set, i-th of total score are obtained
Total score in set is respectively less than the total score that i+1 total diversity is bonded in n-th of total score set, from the n total score set
M total score set before selection, and selection sets the total of quantity respectively from each total score set of the preceding m total score set
Point, to selected total score corresponding to process resource combine distribute processing task, wherein n, m and i is just whole more than 0
Number, and m≤n, i≤n.
13. system as claimed in claim 11, it is characterised in that the total score computing module is additionally operable to true according in the following manner
The scoring of fixed each process resource：
Mode one, when the Current resource situation of a process resource represents that the available resources accounting of the process resource is more than or equal to setting
During threshold value, the scoring of the process resource is defined as the available resources accounting；
Mode two, when the Current resource situation of a process resource represents that the available resources accounting of the process resource is less than the setting
During threshold value, the scoring of the process resource is defined as zero.
14. system as claimed in claim 12, it is characterised in that the total score computing module is additionally operable to true according in the following manner
The scoring of fixed each process resource：
Mode one, when the Current resource situation of a process resource represent the process resource with resource accounting be less than or equal to setting
During threshold value, the scoring of the process resource is defined as this and uses resource accounting；
Mode two, when what the Current resource situation of a process resource represented the process resource is more than the setting with resource accounting
During threshold value, the scoring of the process resource is defined as zero.
15. the system as described in claim 9 or 10, it is characterised in that the system also includes：
Weight setting module, for the attribute combined according to the process resource, for multiple processing of process resource combination
Resource sets corresponding weight.
16. the system as described in claim 11 or 12, it is characterised in that the task allocating module is additionally operable to：
Based on Hash operation, selection sets the total score of quantity respectively from each total score set.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710333404.6A CN107704320B (en) | 2017-05-12 | 2017-05-12 | A kind of method for allocating tasks and system of distributed system |
| PCT/CN2018/085624 WO2018205890A1 (en) | 2017-05-12 | 2018-05-04 | Task assignment method and system of distributed system, computer readable storage medium and computer device therefor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710333404.6A CN107704320B (en) | 2017-05-12 | 2017-05-12 | A kind of method for allocating tasks and system of distributed system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107704320A true CN107704320A (en) | 2018-02-16 |
| CN107704320B CN107704320B (en) | 2018-08-17 |
Family
ID=61169482
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710333404.6A Active CN107704320B (en) | 2017-05-12 | 2017-05-12 | A kind of method for allocating tasks and system of distributed system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN107704320B (en) |
| WO (1) | WO2018205890A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018205890A1 (en) * | 2017-05-12 | 2018-11-15 | 贵州白山云科技有限公司 | Task assignment method and system of distributed system, computer readable storage medium and computer device therefor |
| CN111858014A (en) * | 2019-04-24 | 2020-10-30 | 中国移动通信集团河北有限公司 | Resource allocation method and device |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101351784A (en) * | 2005-12-30 | 2009-01-21 | 阿西式·A·潘迪亚 | Runtime adaptable search processor |
| CN102902587A (en) * | 2011-07-28 | 2013-01-30 | 中国移动通信集团四川有限公司 | Distribution type task scheduling method, distribution type task scheduling system and distribution type task scheduling device |
| CN105022662A (en) * | 2014-04-22 | 2015-11-04 | 中国银联股份有限公司 | Distributive task allocation method |
| CN105427171A (en) * | 2015-11-30 | 2016-03-23 | 北京口袋财富信息科技有限公司 | Data processing method of Internet lending platform rating |
| US20160226789A1 (en) * | 2015-02-04 | 2016-08-04 | Telefonaktiebolaget L M Ericsson (Publ) | Method and system to rebalance constrained services in a cloud using a genetic algorithm |
| US20160292011A1 (en) * | 2015-03-31 | 2016-10-06 | Stitch Fix, Inc. | Systems and methods for intelligently distributing tasks received from clients among a plurality of worker resources |
| CN106126346A (en) * | 2016-07-05 | 2016-11-16 | 东北大学 | A kind of large-scale distributed data collecting system and method |
| CN106445676A (en) * | 2015-08-05 | 2017-02-22 | 杭州海康威视系统技术有限公司 | Distributed data calculation-based task allocation method and task allocation apparatus |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107704320B (en) * | 2017-05-12 | 2018-08-17 | 贵州白山云科技有限公司 | A kind of method for allocating tasks and system of distributed system |
-
2017
- 2017-05-12 CN CN201710333404.6A patent/CN107704320B/en active Active
-
2018
- 2018-05-04 WO PCT/CN2018/085624 patent/WO2018205890A1/en active Application Filing
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101351784A (en) * | 2005-12-30 | 2009-01-21 | 阿西式·A·潘迪亚 | Runtime adaptable search processor |
| CN102902587A (en) * | 2011-07-28 | 2013-01-30 | 中国移动通信集团四川有限公司 | Distribution type task scheduling method, distribution type task scheduling system and distribution type task scheduling device |
| CN105022662A (en) * | 2014-04-22 | 2015-11-04 | 中国银联股份有限公司 | Distributive task allocation method |
| US20160226789A1 (en) * | 2015-02-04 | 2016-08-04 | Telefonaktiebolaget L M Ericsson (Publ) | Method and system to rebalance constrained services in a cloud using a genetic algorithm |
| US20160292011A1 (en) * | 2015-03-31 | 2016-10-06 | Stitch Fix, Inc. | Systems and methods for intelligently distributing tasks received from clients among a plurality of worker resources |
| CN106445676A (en) * | 2015-08-05 | 2017-02-22 | 杭州海康威视系统技术有限公司 | Distributed data calculation-based task allocation method and task allocation apparatus |
| CN105427171A (en) * | 2015-11-30 | 2016-03-23 | 北京口袋财富信息科技有限公司 | Data processing method of Internet lending platform rating |
| CN106126346A (en) * | 2016-07-05 | 2016-11-16 | 东北大学 | A kind of large-scale distributed data collecting system and method |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018205890A1 (en) * | 2017-05-12 | 2018-11-15 | 贵州白山云科技有限公司 | Task assignment method and system of distributed system, computer readable storage medium and computer device therefor |
| CN111858014A (en) * | 2019-04-24 | 2020-10-30 | 中国移动通信集团河北有限公司 | Resource allocation method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2018205890A1 (en) | 2018-11-15 |
| CN107704320B (en) | 2018-08-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112464784A (en) | Distributed training method based on hybrid parallel | |
| CN102314336B (en) | A kind of data processing method and system | |
| CN104902001B (en) | Web request load-balancing method based on operating system virtualization | |
| CN107741986B (en) | User behavior prediction and corresponding information recommendation method and device | |
| CN106325756B (en) | Data storage method, data calculation method and equipment | |
| CN106156159A (en) | A kind of table connection processing method, device and cloud computing system | |
| CN106815254A (en) | A kind of data processing method and device | |
| CN103677960A (en) | Game resetting method for virtual machines capable of controlling energy consumption | |
| CN108108384A (en) | A kind of date storage method and device | |
| CN107291720B (en) | Method, system and computer cluster for realizing batch data processing | |
| CN106055706B (en) | A kind of cache resources storage method and device | |
| CN110245091A (en) | A kind of method, apparatus and computer storage medium of memory management | |
| CN107704320A (en) | The method for allocating tasks and system of a kind of distributed system | |
| CN107070645A (en) | Compare the method and system of the data of tables of data | |
| CN109285015B (en) | Virtual resource allocation method and system | |
| CN110297959A (en) | Date storage method, device, storage medium and Edge Server | |
| CN104009904B (en) | The virtual network construction method and system of facing cloud platform big data processing | |
| CN104572658B (en) | A kind of very large scale integration domain level compares the unit cutting preprocess method of tool | |
| CN106357418A (en) | Method and device for extracting features on basis of complex networks | |
| CN109788061B (en) | Computing task deployment method and device | |
| CN106775962B (en) | Rule execution method and device | |
| CN107786586A (en) | The load dispatching method and device of business | |
| CN109445863A (en) | A kind of data processing method based on FPGA, device, equipment and medium | |
| CN106067158A (en) | A kind of feature comparison method based on GPU and device | |
| CN105468603B (en) | Data selecting method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 100015 5 floor, block E, 201 IT tower, electronic city, 10 Jiuxianqiao Road, Chaoyang District, Beijing. Patentee after: Guizhou Baishan cloud Polytron Technologies Inc Address before: 100015 5 floor, block E, 201 IT tower, electronic city, 10 Jiuxianqiao Road, Chaoyang District, Beijing. Patentee before: Guizhou white cloud Technology Co., Ltd. |