CN110489224A - A kind of method and apparatus of task schedule - Google Patents

A kind of method and apparatus of task schedule Download PDF

Info

Publication number
CN110489224A
CN110489224A CN201810460659.3A CN201810460659A CN110489224A CN 110489224 A CN110489224 A CN 110489224A CN 201810460659 A CN201810460659 A CN 201810460659A CN 110489224 A CN110489224 A CN 110489224A
Authority
CN
China
Prior art keywords
task
performer
value
task performer
amount
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201810460659.3A
Other languages
Chinese (zh)
Inventor
王宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Jingdong Shangke Information Technology Co Ltd filed Critical Beijing Jingdong Century Trading Co Ltd
Priority to CN201810460659.3A priority Critical patent/CN110489224A/en
Publication of CN110489224A publication Critical patent/CN110489224A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5038Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention discloses a kind of method and apparatus of task schedule, are related to field of computer technology.One specific embodiment of this method comprises determining that task performer, sends workload computing and instructs to each task performer;The end value that each task performer is fed back is received, the time needed for determining each task performer feedback result value, to calculate the working efficiency of each task performer;It determines task amount to be processed, according to the working efficiency of each task performer, carries out task distribution.The embodiment is by way of proof of work, the processing capacity of each task performer of accurate evaluation, and according to its processing capacity and task amount to be processed, realizes task smart allocation, accelerate the processing speed of task.

Description

A kind of method and apparatus of task schedule
Technical field
The present invention relates to field of computer technology more particularly to a kind of method and apparatus of task schedule.
Background technique
In existing distributed task dispatching system, such as TBSchedule, Clover, realize mean allocation task Function, and can detecte currently valid actuator quantity, to guarantee that task can be distributed correctly.For example, TBSchedule Actuator and each actuator task to be dealt with are distinguished by QUEUEID, can guarantee each actuator liberally Obtain task as much.
In the implementation of the present invention, at least there are the following problems for inventor's discovery prior art:
1) actuator for distributing task at present, despite the machine by artificial distributing uniform performance, but since network is trembled The performance of the reasons such as dynamic, task quantity, each machine can not be just the same;
2) state and processing capacity that the prior art can not be current according to each actuator carry out smart allocation, cause For possessing the actuator of higher hardware configuration or free time, the task as many with other actuators can only be obtained, and nothing Method makes full use of its processing capacity, results in waste of resources.
Summary of the invention
In view of this, the embodiment of the present invention provides a kind of method and apparatus of task schedule, it is at least able to solve existing skill The process performance that each actuator is not accounted in art leads to the feelings of " high-performance machine idle, low performance machine processing are endless " Condition.
To achieve the above object, according to an aspect of an embodiment of the present invention, a kind of method of task schedule is provided, is wrapped It includes: determining task performer, send workload computing and instruct to each task performer;Each task performer is received to be fed back End value, the time needed for determining each task performer feedback result value, with calculate each task performer work effect Rate;Wherein, end value is corresponding to workload computing instruction;Task amount to be processed is determined, according to the work of each task performer Make efficiency, carries out task amount distribution.
Optionally, the time needed for determining each task performer feedback result value, to calculate each task performer Time needed for working efficiency comprises determining that each task performer feedback result value counts all task performer feedback knots The total time of fruit value calculate the proof of work value of each task performer;Wherein, proof of work value=all task executions The total time of device/individual task actuator time;According to the proof of work value of each task performer, all tasks are determined The amount of work proof value of actuator, calculates the working efficiency of each task performer;Wherein, working efficiency=individual task is held The proof of work value of row device/all task performers amount of work proof value.
Optionally, the end value that each task performer is fed back is received, determines each task performer feedback result value The required time further include: the end value that each task performer is fed back is received, based on scheduled proof rule to being fed back End value verified, reject end value and do not meet the task performer of proof rule;And determine each task performer Time needed for feedback result value screens the identified time based on scheduled time threshold value, rejects the time beyond predetermined The task performer of time threshold.
Optionally, it determines task amount to be processed, according to the working efficiency of each task performer, carries out task amount distribution Include: to receive newly assigned task amount, obtains each task performer currently untreated task amount, obtain task to be processed Amount;According to task amount to be processed, the working efficiency and current untreated task amount of each task performer, new point is determined It is distributed in the task amount matched to the task amount of each task performer.
Optionally, sending workload computing and instructing to each task performer includes: the hair when reaching the scheduled update period Workload computing is sent to instruct to each task performer.
To achieve the above object, according to another aspect of an embodiment of the present invention, a kind of device of task schedule is provided, is wrapped Include: instruction sending module sends workload computing and instructs to each task performer for determining task performer;Efficiency meter Module is calculated, the end value fed back for receiving each task performer determines needed for each task performer feedback result value Time, to calculate the working efficiency of each task performer;Wherein, end value is corresponding to workload computing instruction;Task point With module, for determining task amount to be processed, according to the working efficiency of each task performer, task amount distribution is carried out.
Optionally, efficiency calculation module is used for: the time needed for determining each task performer feedback result value, statistics The total time of all task performer feedback result values calculate the proof of work value of each task performer;Wherein, workload Proof value=all task performers total time/individual task actuator time;According to the work of each task performer Proof value is measured, the amount of work proof value of all task performers is determined, calculates the working efficiency of each task performer;Its In, working efficiency=individual task actuator proof of work value/all task performers amount of work proof value.
Optionally, efficiency calculation module is used for: the end value that each task performer is fed back is received, based on scheduled Proof rule verifies the end value fed back, and rejects the task performer that end value does not meet proof rule;And really Time needed for fixed each task performer feedback result value screens institute the received time based on scheduled time threshold value, Reject task performer of the time beyond scheduled time threshold value.
Optionally, task allocating module is used for: being received newly assigned task amount, is obtained each task performer currently not The task amount of processing obtains task amount to be processed;According to the working efficiency of task amount to be processed, each task performer with And current untreated task amount, it determines and is distributed in newly assigned task amount to the task amount of each task performer.
Optionally, instruction sending module is used for: when reaching the scheduled update period, being sent workload computing and is instructed to every A task performer.
To achieve the above object, according to an embodiment of the present invention in another aspect, the electronics for providing a kind of task schedule is set It is standby.
The electronic equipment of the embodiment of the present invention includes: one or more processors;Storage device, for storing one or more A program, when one or more of programs are executed by one or more of processors, so that one or more of processing The method that device realizes any of the above-described task schedule.
To achieve the above object, according to an embodiment of the present invention in another aspect, provide a kind of computer-readable medium, On be stored with computer program, the method for any of the above-described task schedule is realized when described program is executed by processor.
The scheme of the offer according to the present invention, one embodiment in foregoing invention have the following advantages that or beneficial to effects Fruit: by way of proof of work, the processing capacity of each task performer of accurate evaluation, and according to its processing capacity and Task amount to be processed realizes that task smart allocation achievees the purpose that task is quickly handled with realization " able people should do more work ".
Further effect possessed by above-mentioned non-usual optional way adds hereinafter in conjunction with specific embodiment With explanation.
Detailed description of the invention
Attached drawing for a better understanding of the present invention, does not constitute an undue limitation on the present invention.Wherein:
Fig. 1 is a kind of main flow schematic diagram of the method for task schedule according to an embodiment of the present invention;
Fig. 2 is the distribution schematic diagram of Master host node and task performer under zookeeper;
A kind of Fig. 3 flow diagram of the method for optional task schedule according to an embodiment of the present invention;
Fig. 4 is the flow diagram of the method for another optional task schedule according to an embodiment of the present invention;
Fig. 5 is the flow diagram of the according to an embodiment of the present invention one specifically method of task schedule;
Fig. 6 is a kind of main modular schematic diagram of the device of task schedule according to an embodiment of the present invention;
Fig. 7 is that the embodiment of the present invention can be applied to exemplary system architecture figure therein;
Fig. 8 is adapted for the structural representation for realizing the mobile device of the embodiment of the present invention or the computer system of server Figure.
Specific embodiment
Below in conjunction with attached drawing, an exemplary embodiment of the present invention will be described, including the various of the embodiment of the present invention Details should think them only exemplary to help understanding.Therefore, those of ordinary skill in the art should recognize It arrives, it can be with various changes and modifications are made to the embodiments described herein, without departing from scope and spirit of the present invention.Together Sample, for clarity and conciseness, descriptions of well-known functions and structures are omitted from the following description.
It should be noted that the embodiment of the present invention is applicable in task allocation scenarios, for example, TBSchedule, Clover。
Distributed system error-prone, especially processing competition with it is synchronous during, existing solution, usually Distributed system is coordinated based on reliable coordination system, for example, Zookeeper, and function provided by Zookeeper is extremely It less include: configuring maintenance, name Service, distributed synchronization, group service etc..
More mature universal in view of zookeeper, present invention is primarily based on zookeeper implementations, and do not do to Zookeeper Any improvement has only used some functions of Zookeeper.But it should be recognized that it is identical that its function also can be used Distributed component.
It lays down a definition below to noun according to the present invention:
Proof of work (Proof-of-Work, PoW) is that a kind of corresponding with service is abused with resource or service is blocked to attack The economic measure hit.User is usually required to carry out some time-consuming complex calculations appropriate, and answer can be quick by service side Checking computations, time, equipment and the energy consumed using this to ensure to service with resource are made by real demand as Cost of Guarantee With.
Used time MS: task performer amount of calculation proves the time used, is counted with millisecond.
Actuator: specific execution task applies role, is made of hardware machine and software systems.In the present invention, it holds Row device is divided into the Master host node actuator of distribution task and handles the task performer of task.
Master host node machine: carrying out job enquiry and that distributes applies role, can merge with actuator code It splits, is made of hardware machine and software systems.
Node (Master node or Executor node): Zookeeper term can be used for storing for distributed system The unit of data.Node can be named as Master or Executor, under node can deposit data or newly-built child node, it is generally tree-like Structure.
Referring to Fig. 1, thus it is shown that a kind of broad flow diagram of the method for task schedule provided in an embodiment of the present invention, including Following steps:
S101: determining task performer, sends workload computing and instructs to each task performer.
S102: receiving the end value that each task performer is fed back, and determines each task performer feedback result value institute The time needed, to calculate the working efficiency of each task performer;Wherein, end value is corresponding to workload computing instruction.
S103: determining task amount to be processed, according to the working efficiency of each task performer, carries out task amount distribution.
The present invention is mainly made of two roles: host node Master and actuator Executor, the two roles pass through It carries out Master election in Zookeeper to determine, to guarantee to only exist a host node in synchronization system.To make system Sound and stable operation, the present invention are illustrated with there is effective Master host node always.
To avoid subsequent title from obscuring, later for Master host node actuator, abbreviation Master host node;For The actuator that task is handled under Executor node, abbreviation task performer.
For step S101, election for host node can be in system starting, be also possible to each child node Detecting that host node is deposited when abnormal, registration request can be the identification information of each actuator, for example, the IP of actuator Location.Specifically:
1) when system starts for the first time, host node can be elected, Zookeeper can receive the main section of each actuator at this time Point registration request determines that the actuator that election is succeeded in registration first is main node M aster, remaining election master node failure is held Row device goes under Executor node to be registered as child node again, so that Master distributes task accordingly;
2) after system starts and elects host node, Zookeeper can be based on predetermined detection period, periodic detection Whether Master node machine survives, if existing state is abnormal (for example, delay machine, network failure), sends to all actuators Notice (including this abnormal Master node machine), to re-elect host node Master.
Extremely is resulted in the need for for host node, the host node redefined continues in undertaking the case where re-registering host node The work of one host node, and its unfinished task amount, can distribute to other task performers and be executed.
In addition, Master host node does not need processing task, therefore for the registration request of host node, wherein without comprising Workload value, but the IP address for having actuator is included at least, appoint convenient for subsequent Master host node to the distribution of actuator child node It is used when business.
Registration for task performer machine, registration content are the IP address and its initialization amount PoW of actuator. And since the child node of Executor is registrable multiple, communicate in normal situation, a possibility that registration failure is lower, if but losing It loses, then the actuator will not be assigned task.In addition, if can periodically be retransmitted when the actuator detects unregistered success Registration request reduces the wasting of resources.
Referring specifically to shown in Fig. 2, system is coordinated for zookeeper, TASK_TYPE_1 is task type node, the section There are Master host node and Executor node under point.A node, content Master can wherein be increased under Master node newly The IP address of machine;Registrable multiple nodes under Executor node, content are the IP address and PoW value of Executor machine.
IP address information is saved in Master node, is elected as Master to distinguish the actuator of which IP address.Its The task performer of minor node can inquire Master host node content, periodically to distinguish one's own duty and to re-elect. Wherein, this re-elect can be detect Master host node exception after notified by Master, be also possible to by actuator Laggard rob of querying node Master host node disappearance accounts for.
After determining Master host node, by the actuator being registered under Master host node, it is registered in by reading The identification information of task performer under Executor node obtains task performer list.Master host node is based on IP Location is communicated with each task performer in actuator list, and notice carries out proof of work calculating or task processing.
Further, for task performer list generated, since the quantity of task performer may increase at any time Or it reduces.To guarantee follow-up work processing effectively, need to detect currently valid task performer quantity, to guarantee that task can Correct distribution.
Master host node is sent to the workload computing instruction of each task performer, can be and is sent out with content-form It send, such as " calculating its hash value top N after increasing variable by random number is 0 ", is a kind of proof of work algorithm, wherein N Can be by actuator child node quantity self-setting, and any data hash value is all different and completely random, such as:
1) after " 0 " is calculated by MD5 (Message Digest Algorithm MD5, Message Digest 5), hash value For " CFCD208495D565EF66E7DFF9F98764DA ";
2) to find the hash value that first 1 is 0, then 27 hash value is obtained after being added to 27 since 0 "02E74F10E0327AD868D138F2B4FDD6F0";
3) to find the hash value that first 2 are 0, then 168 is added to and obtains its hash value “006F52E9102A8D3BE2FE5614F42BA989”。
For step S102, each task performer starts to calculate after receiving workload computing instruction, and will calculate End value returns to Master host node.Here end value, be meet the end value of proof of work algorithm requirement, such as Hash value is " 3636638817772E42B59D74CFF571FBB3 ".
But it should be recognized that task performer, proves algorithm, institute for the different operating amount that Master host node issues The end value of return is different, but all meets proof of work algorithm requirement.Such as above-mentioned example, if it is desired to the Hash since 0 It is worth the number of front two, task performer then returns to 168 and waits numerical value.
It further, is raising task treatment effeciency, before the working efficiency of calculating task actuator, the main section of Master Point can dynamically be detected, for the end value and institute's elapsed time that each task performer is returned to hold to task Row device is screened, specifically:
1) whether correct the end value that each task performer is returned is verified: for example, it is desirable that the front two of hash value is 0, return value is 168 and is verified;But if return value is 169, the hash value that host node calculates 169 is " 3636638817772E42B59D74CFF571FBB3 " is undesirable, then does not pass through verifying;
2) it verifies each task performer and returns the result the time consumed by value: for example, due to degraded performance, network event The reasons such as delay machine, cause certain time not calculate end value in barrier, calculating process, these can not returned the result to appointing for value Business actuator weeds out.
The time that Master host node is returned according to each task performer calculates the work effect of each task performer Rate rate is referred to as being allocation proportion.For example, 752ms --- first actuator used time when 10 actuators share 10ms, second used time 23ms ... ..., the last one used time 10ms, by determining time reciprocal value and each time reciprocal value The sum of ratio, for task performer 1, working efficiency rate are as follows:
Rate1=(1/10)/[(1/10)+(1/23)+...+(1/10)].
It, can be according to each task when Master host node is inquired or receives task to be treated for step S103 The working efficiency of actuator and just in untreated task amount, is intelligently allocated:
1) for system initial start up phase, the proof of work of each task performer is 0, at this time can direct root According to its working efficiency, the task amount accordingly distributed is determined;That is working efficiency * general assignment amount;
2) is had been turned on for system the stage, when receiving new task, there may be partial task actuator, there are also not complete At task, need task quantity in combination processing to carry out task distribution at this time, such as execute task in each task performer After 10s, start the distribution of the second subtask, is still the i.e. rate=(task amount in processing+newly distribute task based on working efficiency Amount)/general assignment amount, therefore newly distribute task amount in task amount=general assignment amount * rate- processing.
Further, some numerical results value may not be integer, can be using the side for rounding up or being rounded downwards Formula determines task sendout.Distribution for remaining task amount (general assignment amount-distributed task amount), can there are many real Apply mode:
1) according to the mode to round up, remaining task amount can be distributed to the task execution minimum to working efficiency Device, to improve overall task treatment effeciency;
2) according to the mode being rounded downwards, remaining task amount can be distributed to the highest task execution of working efficiency Overall task treatment effeciency equally can be improved in device;
3) performance for being not necessarily based on each machine is ranked up, and directly carries out order-assigned according to task performer list, will Remaining task amount distributes into table the last one task performer,.
Method provided by above-described embodiment provides a kind of task process performance based on machine and smart allocation task Thinking realized " able people should do more work " with giving full play to the processing capacity of high performance tasks actuator, and then improve at overall task Manage speed.
Referring to Fig. 3, a kind of method flow schematic diagram of optional task schedule according to an embodiment of the present invention is shown, is wrapped Include following steps:
S301: determining task performer, sends workload computing and instructs to each task performer.
S302: receiving the end value that each task performer is fed back, and determines each task performer feedback result value institute The time needed, the total time of all task performer feedback result values is counted, the proof of work of each task performer is calculated Value;Wherein, proof of work value=all task performers total time/individual task actuator time.
S303: according to the proof of work value of each task performer, the amount of work card of all task performers is determined Bright value calculates the working efficiency of each task performer;Wherein, working efficiency=individual task actuator proof of work Value/all task performers amount of work proof value.
S304: determining task amount to be processed, according to the working efficiency of each task performer, carries out task amount distribution.
In above embodiment, step S301, S304 can distinguish the description of step S101, S103 shown in Figure 1, herein It repeats no more.
In above embodiment, for step S302, Master host node according to each task performer returned when Between, the PoW value of each task performer is calculated, for example, 752ms when 10 actuators share, with 752ms divided by each actuator The time of cost obtains the PoW value of each task performer, it may be assumed that
Wherein, MSNIndicate the used time of task performer N, M is the quantity of effective task performer, PoWNAppoint to calculate gained The proof of work value of business actuator N.
Further, its PoW value can be safeguarded and (be updated and save) to the time returned for each task performer, For example, its PoW value is 75.20 after task performer 1 calculates for the first time, then host node needs to save this value, and in this, as Distribute the foundation of task amount;After second obtains proof of work, PoW value is 5.94, then host node machine needs 5.94 are revised as by 75.20.
Further, proof of work value can be updated according to predetermined period, for example, 10 minutes, 5 minutes, It can be other times voluntarily configured.For example, the PoW value of first time calculating task actuator 1 be 75.20, but 10 minutes it Afterwards, its availability is caused to decline since network jitter or other work occupy CPU, its PoW value obtained by second of calculating is 5.94, then the task quantity of lower sub-distribution will be reduced.
For step S303, it is based on gained proof of work value, can determine corresponding working efficiency, it may be assumed that
Wherein, rateNFor the working efficiency of task performer N, ∑ PoW is the cumulative of all task performer PoW values.
Equally, in machine use process, it is understood that there may be machine loss, using the situations such as abnormal causes its working efficiency to drop It is low.Resulting working efficiency is calculated every time as a result, to be also not quite similar.It, can also be with dimension for the working efficiency being calculated The mode of shield carries out preservation update, is convenient for subsequent query, improves task and distributes accuracy.
For each task performer, resulting PoW value and corresponding working efficiency can be generated table, be stored in It is as shown in table 1 below in node:
1 PoW and rate value example of table
It should be noted that since the quantity of task performer may change at any time, and workload computing instruction is not It can be sent as its quantity changes every time and (avoid server stress excessive), for example, effective task performer number before 10s Amount is 10, and quantity becomes 20 after 10s, but workload computing instruction is 10min primary.
Task performer quantity is changed, and the case where the proof of work update cycle is not up to, it can be with direct basis The proof of work value of each task performer, calculates corresponding working efficiency, without relying on the used time, and is convenient for follow-up management, Reduce server service pressure.
Method provided by above-described embodiment specifically calculates each task performer according to the mode of proof of work Working efficiency.For task performer quantity change and not up to the proof of work update cycle the case where, can be with direct basis The proof of work value of each task performer, determines corresponding working efficiency.
Referring to fig. 4, a kind of method flow schematic diagram of optional task schedule according to an embodiment of the present invention is shown, is wrapped Include following steps:
S401: determining task performer, sends workload computing and instructs to each task performer.
S402: receiving the end value that each task performer is fed back, and determines each task performer feedback result value institute The time needed, to calculate the working efficiency of each task performer;Wherein, end value is corresponding to workload computing instruction.
S403: receiving newly assigned task amount, obtains each task performer currently untreated task amount, obtains wait locate The task amount of reason.
S404: according to task amount to be processed, the working efficiency of each task performer and current untreated task Amount is determined and is distributed in newly assigned task amount to the task amount of each task performer.
In above embodiment, step S401 can be found in the description of step S101 shown in Fig. 1, and step S402 can be found in Fig. 1 The description of shown step S102 also may refer to step S302 shown in Fig. 3 and the description of S303, and details are not described herein.
In above embodiment, for step S403 and S404, for system initial start time, each task execution Task amount in device processing is 0, therefore can distribute task amount according to working efficiency rate.
System has been run, task performer has started to process the task moment, if receiving newly assigned task, each Task performer results in the need for distributing different new task amounts due to the task amount difference in processing.Allocation strategy is still based on work Efficiency carries out, i.e., the task amount ratio after examination distribution is equal to rate value, therefore newly distribution task amount=general assignment amount * rate- does not locate The task amount of reason, untreated task amount+newly assigned task amount of general assignment amount=total.
For example, with reference to shown in table 2, by taking task performer 1 as an example,
1) for the first sub-distribution, initialization amount is 0, and the task amount thus distributed is 0.25*1000=250 Item;
2) after 10s, 34 is there remains and be not disposed, but carried out a collection of new task again, total quantity is still 1000 Item, the working efficiency at this time based on the task performer 1 determine the sum of distributed task amount and unfinished task amount accounting institute The task amount for having task performer total is still consistent, i.e. (34+x)/(1000+236)=0.25, is calculated 309, that is, indicate for This needs to distribute 309-34=275 task.
2 task sendout of table
Method provided by above-described embodiment, provides a kind of mode of specific implementation task distribution, and task performer is worked as It is preceding whether to have untreated task, task distribution can be carried out by working efficiency, to realize that task distributes intelligent processing.
Referring to Fig. 5, the method flow schematic diagram of according to an embodiment of the present invention one task schedule specifically is shown, is wrapped Include following steps:
S501: when system starts for the first time or detect current primary node actuator exception, each actuator transmission is received Host node registration request, electing the actuator that succeeds in registration first is host node actuator, and determines that remaining actuator is to appoint Business actuator;Wherein, registration request includes at least the identification information of actuator.
S502: it when reaching the scheduled update period, sends workload computing and instructs to each task performer.
S503: the end value that each task performer is fed back is received, based on scheduled proof rule to the knot fed back Fruit value is verified, and the task performer that end value does not meet proof rule is rejected.
S503 ': the time needed for determining each task performer feedback result value, based on scheduled time threshold value to determining Time screened, reject the time exceed scheduled time threshold value task performer.
S504: the time needed for determining each task performer feedback result value counts all task performer feedback knots The total time of fruit value calculate the proof of work value of each task performer;Wherein, proof of work value=all task executions The total time of device/individual task actuator time.
S505: according to the proof of work value of each task performer, the amount of work card of all task performers is determined Bright value calculates the working efficiency of each task performer;Wherein, working efficiency=individual task actuator proof of work Value/all task performers amount of work proof value.
S506: receiving newly assigned task amount, obtains each task performer currently untreated task amount, obtains wait locate The task amount of reason.
S507: according to task amount to be processed, the working efficiency of each task performer and current untreated task Amount is determined and is distributed in newly assigned task amount to the task amount of each task performer.
In above embodiment, step S501~S503 ' can be found in the description of step S101 shown in Fig. 1;Step S504, S505 can distinguish the description of step S302, S303 shown in Figure 3, and step S506, S507 can distinguish step shown in Figure 4 The description of S403, S404, details are not described herein.
It should be noted that the present invention is when configuring actuator, without manually calculating the processing capacity of each machine.
For handling the machine of same generic task, it is generally the case that the almost the same machine of hardware configuration can be configured as far as possible. But in practical work process, due to the increase of business demand, machine is often repeatedly increased newly, for example, may 3 machines before 1 year Device may need to increase to regard to enough, after 1 year 10, and due to the missing of machine resources every time newly-increased machine with before Hardware configuration may not be completely the same.
Currently generally required artificial to guarantee consistent, the workload that distribution is balanced of machine performance in actual operation Configuration is calculated, and splitting and reorganizing is carried out to hardware, for example, the CPU of 8 cores is invented 42 core CPU, to guarantee performance one It causes.Method shown in the present invention, the working efficiency based on machine carry out task distribution, are not necessarily to the labour cost when configuring machine and calculate often The processing capacity of a machine realizes the effect of smart allocation.
Method provided by the embodiment of the present invention, by way of proof of work, each task performer of accurate evaluation Processing capacity realize task smart allocation, reach realization task and according to its processing capacity and task amount to be processed Quickly processing.In addition, avoiding task performer by way of regularly updating to proof of work value, working efficiency There is abnormal, the distribution more situation for causing task treatment effeciency low of task amount.
Referring to Fig. 6, a kind of main modular signal of the device 600 of task schedule provided in an embodiment of the present invention is shown Figure, comprising:
Instruction sending module 601 sends workload computing and instructs to each task execution for determining task performer Device;
Efficiency calculation module 602, the end value fed back for receiving each task performer, determines each task execution Time needed for device feedback result value, to calculate the working efficiency of each task performer;Wherein, end value and workload computing Instruction is corresponding;
Task allocating module 603, for determining task amount to be processed, according to the working efficiency of each task performer, Carry out task amount distribution.
In implementing device of the present invention, efficiency calculation module 602 is used for:
Time needed for determining each task performer feedback result value counts all task performer feedback result values Total time calculate the proof of work value of each task performer;Wherein, proof of work value=all task performers are total Time/individual task actuator time;
According to the proof of work value of each task performer, the amount of work proof value of all task performers is determined, Calculate the working efficiency of each task performer;Wherein, working efficiency=individual task actuator proof of work value/all The amount of work proof value of task performer.
In implementing device of the present invention, efficiency calculation module 602 is used for:
Receive the end value that each task performer is fed back, based on scheduled proof rule to the end value fed back into The task performer that end value does not meet proof rule is rejected in row verifying;And
Time needed for determining each task performer feedback result value, based on scheduled time threshold value to the received time It is screened, rejects task performer of the time beyond scheduled time threshold value.
In implementing device of the present invention, task allocating module 603 is used for:
Newly assigned task amount is received, each task performer currently untreated task amount is obtained, obtains to be processed Task amount;
According to task amount to be processed, the working efficiency and current untreated task amount of each task performer, really It is distributed in fixed newly assigned task amount to the task amount of each task performer.
In implementing device of the present invention, instruction sending module 601 is used for: when reaching the scheduled update period, sending workload Computations are to each task performer.
In addition, the specific implementation content of the task scheduling apparatus described in embodiments of the present invention, task described above It has been described in detail in dispatching method, therefore has no longer illustrated in this duplicate contents.
Device provided by the embodiment of the present invention, by way of proof of work, each task performer of accurate evaluation Processing capacity realize task smart allocation, reach realization task and according to its processing capacity and task amount to be processed Quickly processing.In addition, avoiding task performer by way of regularly updating to proof of work value, working efficiency There is abnormal, the distribution more situation for causing task treatment effeciency low of task amount.
The exemplary of the method for scheduling task or task scheduling apparatus that can apply the embodiment of the present invention is shown referring to Fig. 7 System architecture 700.
As shown in fig. 7, system architecture 700 may include terminal device 701,702,703, network 704 and server 705. Network 704 between terminal device 701,702,703 and server 705 to provide the medium of communication link.Network 704 can be with Including various connection types, such as wired, wireless communication link or fiber optic cables etc..
User can be used terminal device 701,702,703 and be interacted by network 704 with server 705, to receive or send out Send message etc..Various telecommunication customer end applications, such as the application of shopping class, net can be installed on terminal device 701,702,703 (merely illustrative) such as the application of page browsing device, searching class application, instant messaging tools, mailbox client, social platform softwares.
Terminal device 701,702,703 can be the various electronic equipments with display screen and supported web page browsing, packet Include but be not limited to smart phone, tablet computer, pocket computer on knee and desktop computer etc..
Server 705 can be to provide the server of various services, such as utilize terminal device 701,702,703 to user The shopping class website browsed provides the back-stage management server (merely illustrative) supported.Back-stage management server can be to reception To the data such as information query request analyze etc. processing, and by processing result (such as target push information, product letter Breath -- merely illustrative) feed back to terminal device.
It should be noted that method for scheduling task provided by the embodiment of the present invention is generally executed by server 705, accordingly Ground, task scheduling apparatus are generally positioned in server 705.
It should be understood that the number of terminal device, network and server in Fig. 7 is only schematical.According to realization need It wants, can have any number of terminal device, network and server.
Referring to Fig. 8, it illustrates the knots of the computer system 800 for the terminal device for being suitable for being used to realize the embodiment of the present invention Structure schematic diagram.Terminal device shown in Fig. 8 is only an example, should not function and use scope band to the embodiment of the present invention Carry out any restrictions.
As shown in figure 8, computer system 800 includes central processing unit (CPU) 801, it can be read-only according to being stored in Program in memory (ROM) 802 or be loaded into the program in random access storage device (RAM) 803 from storage section 808 and Execute various movements appropriate and processing.In RAM 803, also it is stored with system 800 and operates required various programs and data. CPU 801, ROM 802 and RAM 803 are connected with each other by bus 804.Input/output (I/O) interface 805 is also connected to always Line 804.
I/O interface 805 is connected to lower component: the importation 806 including keyboard, mouse etc.;It is penetrated including such as cathode The output par, c 807 of spool (CRT), liquid crystal display (LCD) etc. and loudspeaker etc.;Storage section 808 including hard disk etc.; And the communications portion 809 of the network interface card including LAN card, modem etc..Communications portion 809 via such as because The network of spy's net executes communication process.Driver 810 is also connected to I/O interface 805 as needed.Detachable media 811, such as Disk, CD, magneto-optic disk, semiconductor memory etc. are mounted on as needed on driver 810, in order to read from thereon Computer program be mounted into storage section 808 as needed.
Particularly, disclosed embodiment, the process described above with reference to flow chart may be implemented as counting according to the present invention Calculation machine software program.For example, embodiment disclosed by the invention includes a kind of computer program product comprising be carried on computer Computer program on readable medium, the computer program include the program code for method shown in execution flow chart.In In such embodiment, which can be downloaded and installed from network by communications portion 809, and/or from can Medium 811 is dismantled to be mounted.When the computer program is executed by central processing unit (CPU) 801, system of the invention is executed The above-mentioned function of middle restriction.
It should be noted that computer-readable medium shown in the present invention can be computer-readable signal media or meter Calculation machine readable storage medium storing program for executing either the two any combination.Computer readable storage medium for example can be --- but not Be limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor system, device or device, or any above combination.Meter The more specific example of calculation machine readable storage medium storing program for executing can include but is not limited to: have the electrical connection, just of one or more conducting wires Taking formula computer disk, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type may be programmed read-only storage Device (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory device, Or above-mentioned any appropriate combination.In the present invention, computer readable storage medium can be it is any include or storage journey The tangible medium of sequence, the program can be commanded execution system, device or device use or in connection.And at this In invention, computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal, Wherein carry computer-readable program code.The data-signal of this propagation can take various forms, including but unlimited In electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be that computer can Any computer-readable medium other than storage medium is read, which can send, propagates or transmit and be used for By the use of instruction execution system, device or device or program in connection.Include on computer-readable medium Program code can transmit with any suitable medium, including but not limited to: wireless, electric wire, optical cable, RF etc. are above-mentioned Any appropriate combination.
Flow chart and block diagram in attached drawing are illustrated according to the system of various embodiments of the invention, method and computer journey The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation A part of one module, program segment or code of table, a part of above-mentioned module, program segment or code include one or more Executable instruction for implementing the specified logical function.It should also be noted that in some implementations as replacements, institute in box The function of mark can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are practical On can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it wants It is noted that the combination of each box in block diagram or flow chart and the box in block diagram or flow chart, can use and execute rule The dedicated hardware based systems of fixed functions or operations is realized, or can use the group of specialized hardware and computer instruction It closes to realize.
Being described in module involved in the embodiment of the present invention can be realized by way of software, can also be by hard The mode of part is realized.Described module also can be set in the processor, for example, can be described as: a kind of processor packet Include instruction sending module, efficiency calculation module, task allocating module.Wherein, the title of these modules is not under certain conditions The restriction to the module itself is constituted, for example, efficiency calculation module is also described as " the working efficiency meter of task performer Calculate module ".
As on the other hand, the present invention also provides a kind of computer-readable medium, which be can be Included in equipment described in above-described embodiment;It is also possible to individualism, and without in the supplying equipment.Above-mentioned calculating Machine readable medium carries one or more program, when said one or multiple programs are executed by the equipment, makes Obtaining the equipment includes:
It determines task performer, sends workload computing and instruct to each task performer;
The end value that each task performer is fed back is received, when determining needed for each task performer feedback result value Between, to calculate the working efficiency of each task performer;Wherein, the end value is corresponding to workload computing instruction;
It determines task amount to be processed, according to the working efficiency of each task performer, carries out task amount distribution.
Technical solution according to an embodiment of the present invention, by way of proof of work, each task execution of accurate evaluation The processing capacity of device, and according to its processing capacity and task amount to be processed, it realizes task smart allocation, it is quick to reach task The effect of processing.In addition, it is different to avoid task performer appearance by way of regularly updating to proof of work value Often, the more situation for causing task treatment effeciency low of task amount is distributed.
Above-mentioned specific embodiment, does not constitute a limitation on the scope of protection of the present invention.Those skilled in the art should be bright It is white, design requirement and other factors are depended on, various modifications, combination, sub-portfolio and substitution can occur.It is any Made modifications, equivalent substitutions and improvements etc. within the spirit and principles in the present invention, should be included in the scope of the present invention Within.

Claims (12)

1. a kind of method of task schedule characterized by comprising
It determines task performer, sends workload computing and instruct to each task performer;
The end value that each task performer is fed back is received, the time needed for determining each task performer feedback result value, To calculate the working efficiency of each task performer;Wherein, the end value is corresponding to workload computing instruction;
It determines task amount to be processed, according to the working efficiency of each task performer, carries out task amount distribution.
2. the method according to claim 1, wherein needed for each task performer feedback result value of the determination Time, include: to calculate the working efficiency of each task performer
Time needed for determining each task performer feedback result value, count all task performer feedback result values it is total when Between, calculate the proof of work value of each task performer;Wherein, when proof of work value=all task performers are total Between/time of individual task actuator;
According to the proof of work value of each task performer, the amount of work proof value of all task performers is determined, calculate The working efficiency of each task performer;Wherein, working efficiency=individual task actuator proof of work value/all tasks The amount of work proof value of actuator.
3. the method according to claim 1, wherein the result for receiving each task performer and being fed back Value, time needed for determining each task performer feedback result value further include:
The end value that each task performer is fed back is received, the end value fed back is tested based on scheduled proof rule Card rejects the task performer that end value does not meet the proof rule;And
Time needed for determining each task performer feedback result value carries out the identified time based on scheduled time threshold value Task performer of the time beyond the scheduled time threshold value is rejected in screening.
4. the method according to claim 1, wherein the task amount that the determination is to be processed, according to each task The working efficiency of actuator, carrying out task amount distribution includes:
Newly assigned task amount is received, each task performer currently untreated task amount is obtained, obtains described to be processed Task amount;
According to the task amount to be processed, the working efficiency and current untreated task amount of each task performer, really It is distributed in the fixed newly assigned task amount to the task amount of each task performer.
5. the method according to claim 1, wherein the transmission workload computing is instructed to each task execution Device includes:
When reaching the scheduled update period, sends the workload computing and instruct to each task performer.
6. a kind of device of task schedule characterized by comprising
Instruction sending module sends workload computing and instructs to each task performer for determining task performer;
Efficiency calculation module, the end value fed back for receiving each task performer determine each task performer feedback Time needed for end value, to calculate the working efficiency of each task performer;Wherein, the end value and the work meter It is corresponding to calculate instruction;
Task allocating module, according to the working efficiency of each task performer, carries out task for determining task amount to be processed Amount distribution.
7. device according to claim 6, which is characterized in that the efficiency calculation module is used for:
Time needed for determining each task performer feedback result value, count all task performer feedback result values it is total when Between, calculate the proof of work value of each task performer;Wherein, when proof of work value=all task performers are total Between/time of individual task actuator;
According to the proof of work value of each task performer, the amount of work proof value of all task performers is determined, calculate The working efficiency of each task performer;Wherein, working efficiency=individual task actuator proof of work value/all tasks The amount of work proof value of actuator.
8. device according to claim 6, which is characterized in that the efficiency calculation module is used for:
The end value that each task performer is fed back is received, the end value fed back is tested based on scheduled proof rule Card rejects the task performer that end value does not meet the proof rule;And
Time needed for determining each task performer feedback result value carries out institute the received time based on scheduled time threshold value Task performer of the time beyond the scheduled time threshold value is rejected in screening.
9. device according to claim 6, which is characterized in that the task allocating module is used for:
Newly assigned task amount is received, each task performer currently untreated task amount is obtained, obtains described to be processed Task amount;
According to the task amount to be processed, the working efficiency and current untreated task amount of each task performer, really It is distributed in the fixed newly assigned task amount to the task amount of each task performer.
10. device according to claim 6, which is characterized in that described instruction sending module is used for:
When reaching the scheduled update period, sends the workload computing and instruct to each task performer.
11. a kind of electronic equipment characterized by comprising
One or more processors;
Storage device, for storing one or more programs,
When one or more of programs are executed by one or more of processors, so that one or more of processors are real Now such as method as claimed in any one of claims 1 to 5.
12. a kind of computer-readable medium, is stored thereon with computer program, which is characterized in that described program is held by processor Such as method as claimed in any one of claims 1 to 5 is realized when row.
CN201810460659.3A 2018-05-15 2018-05-15 A kind of method and apparatus of task schedule Pending CN110489224A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810460659.3A CN110489224A (en) 2018-05-15 2018-05-15 A kind of method and apparatus of task schedule

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810460659.3A CN110489224A (en) 2018-05-15 2018-05-15 A kind of method and apparatus of task schedule

Publications (1)

Publication Number Publication Date
CN110489224A true CN110489224A (en) 2019-11-22

Family

ID=68545071

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810460659.3A Pending CN110489224A (en) 2018-05-15 2018-05-15 A kind of method and apparatus of task schedule

Country Status (1)

Country Link
CN (1) CN110489224A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112379986A (en) * 2020-11-16 2021-02-19 北京字跳网络技术有限公司 Task processing method and device and electronic equipment
CN112486658A (en) * 2020-12-17 2021-03-12 华控清交信息科技(北京)有限公司 Task scheduling method and device for task scheduling
CN112948111A (en) * 2021-02-26 2021-06-11 北京奇艺世纪科技有限公司 Task allocation method, device, equipment and computer readable medium

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0862113A2 (en) * 1997-02-27 1998-09-02 Zuno Limited Autonomous agent architecture
CN101916209A (en) * 2010-08-06 2010-12-15 华东交通大学 Cluster task resource allocation method for multi-core processor
CN101957780A (en) * 2010-08-17 2011-01-26 中国电子科技集团公司第二十八研究所 Resource state information-based grid task scheduling processor and grid task scheduling processing method
CN102111337A (en) * 2011-03-14 2011-06-29 浪潮(北京)电子信息产业有限公司 Method and system for task scheduling
CN104239148A (en) * 2013-06-06 2014-12-24 腾讯科技(深圳)有限公司 Distributed task scheduling method and device
CN105049268A (en) * 2015-08-28 2015-11-11 东方网力科技股份有限公司 Distributed computing resource allocation system and task processing method
CN105373428A (en) * 2015-12-09 2016-03-02 北京奇虎科技有限公司 Task scheduling method and system
CN105487930A (en) * 2015-12-01 2016-04-13 中国电子科技集团公司第二十八研究所 Task optimization scheduling method based on Hadoop
CN106126332A (en) * 2016-06-27 2016-11-16 北京京东尚科信息技术有限公司 Distributed timing task scheduling system and method
CN106325984A (en) * 2015-06-30 2017-01-11 上海证大喜马拉雅网络科技有限公司 Big data task scheduling apparatus
CN106528280A (en) * 2015-09-15 2017-03-22 阿里巴巴集团控股有限公司 Task allocation method and system
KR101794696B1 (en) * 2016-08-12 2017-11-07 서울시립대학교 산학협력단 Distributed processing system and task scheduling method considering heterogeneous processing type

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0862113A2 (en) * 1997-02-27 1998-09-02 Zuno Limited Autonomous agent architecture
CN101916209A (en) * 2010-08-06 2010-12-15 华东交通大学 Cluster task resource allocation method for multi-core processor
CN101957780A (en) * 2010-08-17 2011-01-26 中国电子科技集团公司第二十八研究所 Resource state information-based grid task scheduling processor and grid task scheduling processing method
CN102111337A (en) * 2011-03-14 2011-06-29 浪潮(北京)电子信息产业有限公司 Method and system for task scheduling
CN104239148A (en) * 2013-06-06 2014-12-24 腾讯科技(深圳)有限公司 Distributed task scheduling method and device
CN106325984A (en) * 2015-06-30 2017-01-11 上海证大喜马拉雅网络科技有限公司 Big data task scheduling apparatus
CN105049268A (en) * 2015-08-28 2015-11-11 东方网力科技股份有限公司 Distributed computing resource allocation system and task processing method
CN106528280A (en) * 2015-09-15 2017-03-22 阿里巴巴集团控股有限公司 Task allocation method and system
CN105487930A (en) * 2015-12-01 2016-04-13 中国电子科技集团公司第二十八研究所 Task optimization scheduling method based on Hadoop
CN105373428A (en) * 2015-12-09 2016-03-02 北京奇虎科技有限公司 Task scheduling method and system
CN106126332A (en) * 2016-06-27 2016-11-16 北京京东尚科信息技术有限公司 Distributed timing task scheduling system and method
KR101794696B1 (en) * 2016-08-12 2017-11-07 서울시립대학교 산학협력단 Distributed processing system and task scheduling method considering heterogeneous processing type

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
徐爱萍;吴笛;徐武平;陈军;: "实时多任务异构云计算平台负载均衡算法", 中国科学技术大学学报, no. 03, 15 March 2016 (2016-03-15) *
杨杭;张昕;赵建平;: "基于动态反馈的负载均衡方法研究", 长春理工大学学报(自然科学版), no. 06, 15 December 2017 (2017-12-15) *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112379986A (en) * 2020-11-16 2021-02-19 北京字跳网络技术有限公司 Task processing method and device and electronic equipment
CN112379986B (en) * 2020-11-16 2023-12-12 北京字跳网络技术有限公司 Task processing method and device and electronic equipment
CN112486658A (en) * 2020-12-17 2021-03-12 华控清交信息科技(北京)有限公司 Task scheduling method and device for task scheduling
CN112948111A (en) * 2021-02-26 2021-06-11 北京奇艺世纪科技有限公司 Task allocation method, device, equipment and computer readable medium
CN112948111B (en) * 2021-02-26 2023-07-14 北京奇艺世纪科技有限公司 Task allocation method, device, equipment and computer readable medium

Similar Documents

Publication Publication Date Title
Yang et al. Joint computation partitioning and resource allocation for latency sensitive applications in mobile edge clouds
Lu et al. Join-idle-queue: A novel load balancing algorithm for dynamically scalable web services
Li et al. Efficient online coflow routing and scheduling
US9201690B2 (en) Resource aware scheduling in a distributed computing environment
US9417903B2 (en) Storage management for a cluster of integrated computing systems comprising integrated resource infrastructure using storage resource agents and synchronized inter-system storage priority map
JP5664098B2 (en) Composite event distribution apparatus, composite event distribution method, and composite event distribution program
CN103927229A (en) Scheduling Mapreduce Jobs In A Cluster Of Dynamically Available Servers
CN108874640A (en) A kind of appraisal procedure and device of clustering performance
CN107003887A (en) Overloaded cpu setting and cloud computing workload schedules mechanism
CN106598735B (en) A kind of distributed computing method, main controlled node and computing system
CN110162388A (en) A kind of method for scheduling task, system and terminal device
CN110489224A (en) A kind of method and apparatus of task schedule
CN110019339A (en) A kind of data query method and system
CN110166507A (en) More resource regulating methods and device
CN102724103A (en) Proxy server, hierarchical network system and distributed workload management method
Zhang et al. Heterogeneity aware dominant resource assistant heuristics for virtual machine consolidation
CN110377416A (en) Distributed subregion method for scheduling task and device
CN109614227A (en) Task resource concocting method, device, electronic equipment and computer-readable medium
CN109413125A (en) The method and apparatus of dynamic regulation distributed system resource
CN109879126A (en) A kind of method and system of elevator reservation
Nahir et al. Workload factoring with the cloud: A game-theoretic perspective
CN113495779A (en) Task scheduling method and device and task execution system
Convolbo et al. DRASH: A data replication-aware scheduler in geo-distributed data centers
CN110413419A (en) A kind of method and apparatus that rule executes
Li Optimal load distribution for multiple heterogeneous blade servers in a cloud computing environment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination