CN104965754A - Task scheduling method and task scheduling apparatus - Google Patents

Task scheduling method and task scheduling apparatus Download PDF

Info

Publication number
CN104965754A
CN104965754A CN201510147593.9A CN201510147593A CN104965754A CN 104965754 A CN104965754 A CN 104965754A CN 201510147593 A CN201510147593 A CN 201510147593A CN 104965754 A CN104965754 A CN 104965754A
Authority
CN
China
Prior art keywords
task
current task
tasks carrying
current
scheduling
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201510147593.9A
Other languages
Chinese (zh)
Inventor
宫振飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN201510147593.9A priority Critical patent/CN104965754A/en
Publication of CN104965754A publication Critical patent/CN104965754A/en
Pending legal-status Critical Current

Links

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a task scheduling method and a task scheduling apparatus. The method comprises following steps: triggering a current task when triggering conditions of the current task are met; determining whether dependent tasks exist in the current task according to a task dependence graph, and acquiring task execution result states of the dependent tasks if the dependent tasks exist in the current task, wherein the task dependence graph is a directed acyclic graph showing dependence relations among the tasks; determining whether all the dependent tasks are successfully executed according to the task execution result states of the dependent tasks; and sending the current task to a task execution node for execution if all the dependent tasks are successfully executed. According to the embodiment scheme provided in the invention, the reliability of file dependence checking on the tasks having dependence relations is improved, and the task execution efficiency for the tasks during a scheduling process is improved.

Description

Method for scheduling task and task scheduling apparatus
Technical field
The present invention relates to field of information processing, particularly a kind of method for scheduling task and a kind of task scheduling apparatus.
Background technology
Hadoop is one and realizes based on Java, increase income, comprise the project of distributed file system and MapReduce Computational frame, distributed open source system Hadoop is separated by file system and computing system, each computing node shared-nothing, realize the linear expansion of machine, by the normalization using MapReduce computation module to realize computing method, data analysis process is divided into two stages by MapReduce computation module: the map stage of dividing heap in order to realize data acquisition &, in order to realize the reduce stage of data summarization, thus solve the linear calculating of mass data, mainly solve the large problem of data volume.
Hive is Apache SQL analytics engine of increasing income under one's name, it is a Tool for Data Warehouse based on Hadoop, can by SQL (Structured Query Language, Structured Query Language (SQL), for accessing the computerese with the standard of process database) statement is converted to a MapReduce task run, thus achieves simplification and the structuring of programmed method, is familiar with SQL statement, just can realize complicated calculating fast, improve development efficiency.
But based on the solution of Hadoop+Hive, between task during Existence dependency relationship, the calculating of such as data must wait after the warehouse-in of pending data completes, and needs to ensure that the order of task performs by additive method.The mode that current guarantee task can sequentially perform, the warehouse-in of pending data must be waited to complete for the calculating of data, generate a file after data loading mission script completes, first detect this file before the script startup of data calculation task whether to exist, do not exist, need to wait for, until this file detected.Based on this mode, need the shared file configuring each tasks carrying, layoutprocedure more complicated, and in each tasks carrying process, need continuously to this shared region written document and read file, too much read-write number of times can badly influence the efficiency of tasks carrying.
Summary of the invention
Based on this, the object of the embodiment of the present invention is to provide a kind of method for scheduling task and a kind of task scheduling apparatus, tasks carrying efficiency when its each task that can improve Existence dependency relationship is dispatched.
For achieving the above object, the embodiment of the present invention by the following technical solutions:
A kind of method for scheduling task, comprises step:
When reaching current task trigger condition, trigger current task;
Judge whether this current task has the task of dependence according to Task Dependent figure, if having, obtain the task action result state of each task relied on, described Task Dependent figure is the directed acyclic graph of the dependence between each task;
According to the task result executing state of relied on each task, judge whether each task relied on all runs succeeded;
If so, described current task is handed down to tasks carrying node to perform.
A kind of task scheduling apparatus, comprising:
Task trigger module, reaches task trigger condition for judging whether, and when reaching current task trigger condition, triggers current task;
Task dependencies determination module, for judging whether described current task has the task of dependence according to Task Dependent figure, described Task Dependent figure is the directed acyclic graph of the dependence between each task;
Any enforceability determination module, for when the result of determination of described task dependencies determination module is for being, obtain the task action result state of each task relied on, according to the task result executing state of relied on each task, judge whether each task relied on all runs succeeded;
Task processing module, for when the result of determination of described task dependencies determination module is no, or the result of determination of described task enforceability determination module for being time, carry out described current task being handed down to tasks carrying node and perform.
According to the scheme of the embodiment of the present invention as above, its task based access control dependency graph embodies the dependence between the task such as data task stream, calculation task stream, when reaching the trigger condition of current task, first task based access control dependency graph obtains the task that this current task relies on, and when the task action result state of relied on each task be run succeeded, just this task is issued execution.It is without the need to the operation carrying out written document continually and read file, direct task based access control execution result state can determine whether relied on task runs succeeded, overcome file checking frequently and operate the low problem of the tasks carrying efficiency caused reliability that is low and that cause due to file error, tasks carrying efficiency when each task that improve Existence dependency relationship is dispatched.
Accompanying drawing explanation
Fig. 1 is the working environment schematic diagram of the present invention program in an embodiment;
Fig. 2 is the working environment schematic diagram of the present invention program in another embodiment;
Fig. 3 is the composition structural representation of the dispatch server in an embodiment;
Fig. 4 is the schematic flow sheet of the method for scheduling task in an embodiment;
Fig. 5 is the Task Dependent figure in a concrete example;
Fig. 6 is the interaction flow schematic diagram of the method for scheduling task in a concrete example;
Fig. 7 is the structural representation of the task scheduling apparatus in a concrete example.
Embodiment
For making object of the present invention, technical scheme and advantage clearly understand, below in conjunction with drawings and Examples, the present invention is described in further detail.Should be appreciated that embodiment described herein only in order to explain the present invention, do not limit protection scope of the present invention.
Fig. 1 shows the working environment schematic diagram in one embodiment of the invention.Task scheduling node 101 is connected with more than one tasks carrying node 102, task scheduling node 101 is when the trigger condition reaching certain task being detected, analyze this task and whether can issue execution, if passable, this task be handed down to tasks carrying node 102 and perform.Certain dependence may be there is between each task that task scheduling node 101 issues, the warehouse-in of pending data such as must be waited to the calculating of data, so, data calculation task depends on completing of data loading task, need the correct execution of this data calculation task of guarantee after data loading task completes.Dependence between each task can be represented by Task Dependent figure, and this Task Dependent figure can be a directed acyclic graph.This Task Dependent figure can be stored in metadata store node 103.In metadata store node 103, the task result executing state of each task can also be stored, such as, run succeeded, perform failure etc.
Fig. 2 shows the working environment schematic diagram of another embodiment of the present invention, in this embodiment, for the working environment shown in Fig. 1, also include configuration and show node 100, in order to be configured the dependence of each task, obtain above-mentioned dependence graph, the dependence graph obtained can be stored into metadata store node 103.
In the working environment shown in Fig. 1, Fig. 2, each node, comprise configuration and show node 100, task scheduling node 101, tasks carrying node 102 and metadata store node 103, independently machinery and equipment can be adopted respectively to realize, to realize other task scheduling of machine level.
Adopt independently machinery and equipment to be embodied as example respectively with each node, suppose that this task scheduling node 101 is task scheduling equipment server, the structural representation of the task scheduling server 101 in an embodiment has been shown in Fig. 3.As shown in Figure 3, this task scheduling server 101 comprises the processor, storage medium, communication interface, power interface and the internal memory that are connected by system bus.Wherein, the storage medium of task scheduling server 101 stores a kind of task scheduling apparatus, and this device is for realizing a kind of method for scheduling task.The communication interface of task scheduling server 101 is used for showing that node 100, each tasks carrying node 102 and metadata store node 103 are connected and communicate with configuration, the power interface of task scheduling server 101 is used for being connected with external power source, and external power source is powered to task scheduling server 101 by this power interface.
Schematic diagram shown in composition graphs 1, Fig. 2, Fig. 3, has illustrated the schematic flow sheet of the method for scheduling task in an embodiment in Fig. 4, the method is implemented on above-mentioned task scheduling node 101.
As shown in Figure 4, the method for scheduling task in the present embodiment comprises step:
S401: when reaching current task trigger condition, triggers current task;
S402: judge whether this current task has the task of dependence according to Task Dependent figure, if having, enter step S403;
Step S403: the task action result state obtaining each task relied on, described Task Dependent figure is the directed acyclic graph of the dependence between each task;
Step S404: according to the task result executing state of relied on each task, judges whether each task relied on all runs succeeded, and if so, enters step S405;
Step S405: described current task is handed down to tasks carrying node and performs.
According to the scheme of the embodiment of the present invention as above, its task based access control dependency graph embodies the dependence between the task such as data task stream, calculation task stream, when reaching the trigger condition of current task, first task based access control dependency graph obtains the task that this current task relies on, and when the task action result state of relied on each task be run succeeded, just this task is issued execution.It is without the need to the operation carrying out written document continually and read file, direct task based access control execution result state can determine whether relied on task runs succeeded, overcome file checking frequently and operate the problem that the tasks carrying efficiency caused is low and reliability is low, and to avoid in implementation the problem just judging the waste resource whether institute's dependence task runs succeeded caused and inefficiency, tasks carrying efficiency when each task that improve Existence dependency relationship is dispatched.
Wherein, when judging to have at least one not run succeeded in each task that current task relies in above-mentioned steps S404, perform waiting process, illustrate that current task cannot perform, start to perform waiting process, and when the stand-by period is greater than or equal to default time-out time threshold value, judges that described current task performs failure, send unsuccessfully warning information.If each task relied in default time-out time threshold value is all finished, then this current task is handed down to tasks carrying node and performs.
After current task being issued, tasks carrying node can perform this current task, and task based access control execution result returns tasks carrying return code.Task scheduling node can receive the tasks carrying return code that tasks carrying node returns, and task based access control execution return code judges that whether this tasks carrying node execution current task is successful.
If tasks carrying node performs current task success, then think that this current task runs succeeded, and upgrade the task result executing state of current task, so that the follow-up work depending on this current task can perform smoothly.
Wherein, when judging that this tasks carrying node execution current task is unsuccessful, can also judge whether further to reach mission failure retry termination condition.
If do not reach mission failure retry termination condition, another tasks carrying node then described current task is sent to perform, until some tasks carrying nodes perform this current task success or reach mission failure retry termination condition, the tasks carrying failure caused to avoid tasks carrying one malfunctions, realizes other fault tolerant mechanism of machine level.
If reach mission failure retry termination condition, then can judge that described current task performs unsuccessful, and upgrade the task result executing state of current task.
Wherein, above-mentioned mission failure retry termination condition, can adopt various possible mode to arrange, such as, arrange task execution time, task retry number of times etc.In one embodiment of the invention, mission failure retry termination condition is set to predetermined failed number of retries threshold value, and when the number of retries of current task reaches predetermined failed number of retries threshold value, judges to reach mission failure retry termination condition.
After above-mentioned current task runs succeeded, can also judge whether current task is executing the task after fault recovery, if so, then according to the order of Task Dependent figure further, trigger the follow-up work of current task, not arrive working time until follow-up without task or follow-up work.Task chain when improve tasks carrying failure recovers efficiency.
As mentioned above, the basic thought of the embodiment of the present invention, realize data stream, calculate the abstract of the task such as stream, the Task Dependent figure of directed acyclic is used to solve dependence, in configuration, dependence between task can by manually showing that node 100 is configured, configure the Task Dependent figure that obtains and can be stored in metadata store node 103, and each task can at a fixed time in clocked flip.
Below in conjunction with the interaction flow schematic diagram of the method for scheduling task shown in Task Dependent figure, the Fig. 6 shown in Fig. 5, be illustrated with regard to one of them task scheduling process.
As shown in Figure 5, in this Task Dependent figure, always have four task: task A, task B, task C, task D, wherein, task B, task C depend on task A, and task D depends on task B and task C, and each directed edge in this Task Dependent figure can be stored in metadata store node 103.
As mentioned above, in this example illustrates, each task is clocked flip, supposes the current working time reaching task C, and current task is task C.
Shown in composition graphs 6, in step s 601, task scheduling node can detect whether reach current task trigger condition, namely when arriving the working time of task C, judging to reach task C trigger condition, and carry out clocked flip in step S602.
After triggering current task C, in step S603, based on Task Dependent figure, task scheduling node judges whether this task C has the task of dependence, wherein, it is local that this Task Dependent figure can be stored in task scheduling node, the Task Dependent figure that task scheduling node directly stores based on this locality can judge, on the other hand, this Task Dependent figure can obtain (this process is not shown in Figure 6) from metadata store node.
If task C has the task of dependence, then enter step S610, if the task that task C does not rely on, then enter step S620.
In step S610, task scheduling node obtains the task action result state of each task that task C relies on from metadata store node, and judges whether relying on of task all runs succeeded in step s 611.Shown in composition graphs 5, then task scheduling node is the task result executing state obtaining task A from metadata store node, and judges whether task A runs succeeded.
If the result of determination of step S611 is no, namely relied on task has at least an execution unsuccessful, then enter step S612 to execute the task waiting status, and in step S613, whether default time-out time threshold value is arrived to the stand-by period and monitor, if the tasks carrying still having at least one to rely on when arriving and presetting time-out time threshold value is unsuccessful, in step S614, then send task C failure warning information, this failed warning information can inform operation maintenance personnel by the communication mode such as note, micro-letter.Operation maintenance personnel in step S615 can analysis task C failure root problem and it is solved, partition that such as mechanical disorder is recovered, data volume is too large etc., and after failure problems solves, can a manual key recovery tasks C, then can return step S601 and again trigger this task C.
If judge in step S611, each task that task C relies on all runs succeeded, then enter step S620, and before not arriving default time-out time threshold value if monitor in step S613, each task of relying on of task C all runs succeeded, then enter step S620.
In step S620, task C is handed down to tasks carrying node 1 by task scheduling node, and tasks carrying node 1 performs this task C, and in step S621, return the execution return code of task C.
Task scheduling node is after receiving the execution return code that tasks carrying node 1 returns, in step S622, whether successful judge that tasks carrying node 1 performs this task C based on this execution return code, if tasks carrying node 1 is in this task C success of execution, then think that this task C runs succeeded, and enter step S627, based on the execution result more new database of this task C, by the task action result state updating of the task C in metadata store node for running succeeded.
If judge that tasks carrying node 1 C that executes the task is unsuccessful based on execution return code, then enter step S623, task scheduling node judges whether the mission failure retry termination condition reaching this task C, such as, whether the number of retries of this task C reaches predetermined failed number of retries threshold value, if do not reach, then enter step S624, this task C is handed down to another tasks carrying node K, tasks carrying node K performs this task C, and in step S625, return the execution return code of task C.
Task scheduling node is after receiving the execution return code that tasks carrying node K returns, return step S622, adopt the mode same with step S622, based on step S62, whether the execution return code in 5 judges that tasks carrying node K performs this task C successful, if tasks carrying node K performs this task C success, then think that this task C runs succeeded, and enter step S627, if to perform this task C unsuccessful for tasks carrying node K, then enter step S623.
Reach the mission failure retry termination condition of this task C if judge in step S623, then in step S626, judge that this task C performs failure.Based on the setting of predetermined failed number of retries threshold value, only reach in the number of retries of task or exceed predetermined failed number of retries threshold value, task scheduling node just can think that this tasks carrying is failed, record failure information, thus can distributed scheduling be realized, prevent from, because the wherein mission failure that causes of a mechanical disorder, achieving other disaster tolerance of machine level.
Task scheduling node judges that in step S626 task C performs unsuccessfully, enters step S627, based on the execution result more new database of this task C, by the task action result state updating of the task C in metadata store node for perform failure.Can enter step S614, send task C failure warning information, operation maintenance personnel can be removed fault based on this failed warning information simultaneously.
When task C runs succeeded, in step S627 more after new database, can step S628 be entered, judge whether this task C is fault recovery task.This is because, the execution of this task C may be the task based on the further pull-up of fault recovery above, such as task A performs failure, task A is performed to the task that failed reason solves rear task based access control dependency graph pull-up, or task C itself performs performing unsuccessfully, to task C and again triggering after failed reason solves of task.And in the process solved in fault task, other tasks follow-up have reached the triggered time, and due to the existence of Task Dependent relation, what can cause follow-up a series of task performs failure.Now, to again manually recover each task one by one, especially when failure recovery time is longer, great maintenance work amount can be caused.
Now, if determining this task C in step S628 is fault recovery task, then enter step S629, order in task based access control dependency graph, the follow-up work of triggering tasks C, not arrive working time until follow-up without task or follow-up work, thus task chain when can improve tasks carrying failure recovers efficiency.
Based on the thought identical with above-mentioned method for scheduling task, the embodiment of the present invention also provides a kind of task scheduling apparatus.The structural representation of the task scheduling apparatus in an embodiment has been shown in Fig. 7.This task scheduling apparatus is arranged at the task scheduling node 101 shown in Fig. 1,2.
As shown in Figure 7, the task scheduling apparatus in the present embodiment includes:
Task trigger module 701, reaches task trigger condition for judging whether, and when reaching current task trigger condition, triggers current task;
Task dependencies determination module 702, for judging whether described current task has the task of dependence according to Task Dependent figure, described Task Dependent figure is the directed acyclic graph of the dependence between each task;
Any enforceability determination module 703, for when the result of determination of task dependencies determination module 702 is for being, obtain the task action result state of each task relied on, according to the task result executing state of relied on each task, judge whether each task relied on all runs succeeded;
Task processing module 704, for when the result of determination of task dependencies determination module 702 is no, or the result of determination of task enforceability determination module 703 for being time, carry out described current task being handed down to tasks carrying node and perform.
According to the scheme of the embodiment of the present invention as above, its task based access control dependency graph embodies the dependence between the task such as data task stream, calculation task stream, when reaching the trigger condition of current task, first task based access control dependency graph obtains the task that this current task relies on, and when the task action result state of relied on each task be run succeeded, just this task is issued execution.It is without the need to the operation carrying out written document continually and read file, direct task based access control execution result state can determine whether relied on task runs succeeded, overcome file checking frequently and operate the problem that the tasks carrying efficiency caused is low and reliability is low, and to avoid in implementation the problem just judging the waste resource whether institute's dependence task runs succeeded caused and inefficiency, tasks carrying efficiency when each task that improve Existence dependency relationship is dispatched.
Wherein, when above-mentioned task enforceability determination module 703 has at least one not run succeeded in each task judging that current task relies on, waiting process can also be performed, and when the stand-by period is greater than or equal to default time-out time threshold value, judge that described current task performs failure, send unsuccessfully warning information, and upgrade the task result executing state of described current task.If task enforceability determination module 703 monitors each task relied in default time-out time threshold value and is all finished, then still this current task is handed down to tasks carrying node and performs.
After current task being issued, tasks carrying node can perform this current task, and task based access control execution result returns tasks carrying return code.Task scheduling node can receive the tasks carrying return code that tasks carrying node returns, and task based access control execution return code judges that whether this tasks carrying node execution current task is successful.
Based on this, in a concrete example,
Above-mentioned task processing module 704, also for receiving the tasks carrying return code that described tasks carrying node returns, and whether judge that described tasks carrying node performs current task based on described tasks carrying return code successful, if, judge that described current task runs succeeded, and upgrade the task result executing state of described current task.
If based on described tasks carrying return code, task processing module 704 judges that described tasks carrying node performs current task unsuccessful, then can judge whether further to reach mission failure retry termination condition, if not, another tasks carrying node described current task is sent to perform.Thus the tasks carrying that tasks carrying one malfunctions can be avoided to cause failure, realize other fault tolerant mechanism of machine level.
If task processing module 704 judges to reach mission failure retry termination condition, then directly can judge that described current task performs unsuccessful, and upgrade the task result executing state of described current task.
Wherein, above-mentioned mission failure retry termination condition, can adopt various possible mode to arrange, such as, arrange task execution time, task retry number of times etc.In one embodiment of the invention, mission failure retry termination condition can be set to predetermined failed number of retries threshold value, and when the number of retries of current task reaches predetermined failed number of retries threshold value, judge to reach mission failure retry termination condition.
Execution due to this current task may be the task based on the further pull-up of fault recovery above, such as relied on tasks carrying failure, the reason of relied on tasks carrying failure is solved after the task of task based access control dependency graph pull-up, or current task itself performs unsuccessfully rear, to again triggering after the failed reason solution of current task execution of task.And in the process solved in fault task, because other tasks follow-up have reached the triggered time, and due to the existence of Task Dependent relation, what can cause follow-up a series of task performs failure.Now, to again manually recover each task one by one, especially when failure recovery time is longer, great maintenance work amount can be caused.
Accordingly, in a concrete example, above-mentioned task processing module 704, after current task runs succeeded, can also judge whether described current task is executing the task after fault recovery, now, above-mentioned task trigger module 707, time also for judging current task at task module 704 as executing the task after fault recovery, according to the order of Task Dependent figure, trigger the follow-up work of described current task, not reach trigger condition until follow-up without task or follow-up work.
One of ordinary skill in the art will appreciate that all or part of flow process realized in above-described embodiment method, that the hardware that can carry out instruction relevant by computer program has come, described program can be stored in a computer read/write memory medium, this program, when performing, can comprise the flow process of the embodiment as above-mentioned each side method.Wherein, described storage medium can be magnetic disc, CD, read-only store-memory body (Read-Only Memory, ROM) or random store-memory body (Random Access Memory, RAM) etc.
Each technical characteristic of the above embodiment can combine arbitrarily, for making description succinct, the all possible combination of each technical characteristic in above-described embodiment is not all described, but, as long as the combination of these technical characteristics does not exist contradiction, be all considered to be the scope that this instructions is recorded.
The above embodiment only have expressed several embodiment of the present invention, and it describes comparatively concrete and detailed, but can not therefore be construed as limiting the scope of the patent.It should be pointed out that for the person of ordinary skill of the art, without departing from the inventive concept of the premise, can also make some distortion and improvement, these all belong to protection scope of the present invention.Therefore, the protection domain of patent of the present invention should be as the criterion with claims.

Claims (16)

1. a method for scheduling task, is characterized in that, comprises step:
When reaching current task trigger condition, trigger current task;
Judge whether this current task has the task of dependence according to Task Dependent figure, if having, obtain the task action result state of each task relied on, described Task Dependent figure is the directed acyclic graph of the dependence between each task;
According to the task result executing state of relied on each task, judge whether each task relied on all runs succeeded;
If so, described current task is handed down to tasks carrying node to perform.
2. method for scheduling task according to claim 1, is characterized in that, also comprises step:
When judging to have at least one not run succeeded in each task that current task relies on, perform waiting process;
When the stand-by period is greater than or equal to default time-out time threshold value, judges that described current task performs failure, send unsuccessfully warning information.
3. method for scheduling task according to claim 1, is characterized in that, after current task being issued, also comprises step:
Receive the tasks carrying return code that described tasks carrying node returns;
Whether successfully judge that described tasks carrying node performs current task based on described tasks carrying return code;
If so, judge that described current task runs succeeded, and upgrade the task result executing state of described current task.
4. method for scheduling task according to claim 3, is characterized in that, when judging that described tasks carrying node execution current task is unsuccessful, also comprises step:
Judge whether to reach mission failure retry termination condition, if not, described current task sent to another tasks carrying node to perform.
5. method for scheduling task according to claim 4, is characterized in that, when judging to reach mission failure retry termination condition, judging that described current task performs unsuccessful, and upgrading the task result executing state of described current task.
6. method for scheduling task according to claim 4, is characterized in that, when the number of retries of described current task reaches predetermined failed number of retries threshold value, judges to reach mission failure retry termination condition.
7. method for scheduling task according to claim 3, it is characterized in that, after described current task runs succeeded, also comprise step: judge whether described current task is executing the task after fault recovery, if, according to the order of Task Dependent figure, trigger the follow-up work of described current task, not arrive working time until follow-up without task or follow-up work.
8. method for scheduling task according to claim 1, is characterized in that, when arriving the execution time of current task, judges to reach current task trigger condition.
9. a task scheduling apparatus, is characterized in that, comprising:
Task trigger module, reaches task trigger condition for judging whether, and when reaching current task trigger condition, triggers current task;
Task dependencies determination module, for judging whether described current task has the task of dependence according to Task Dependent figure, described Task Dependent figure is the directed acyclic graph of the dependence between each task;
Any enforceability determination module, for when the result of determination of described task dependencies determination module is for being, obtain the task action result state of each task relied on, according to the task result executing state of relied on each task, judge whether each task relied on all runs succeeded;
Task processing module, for when the result of determination of described task dependencies determination module is no, or the result of determination of described task enforceability determination module for being time, carry out described current task being handed down to tasks carrying node and perform.
10. task scheduling apparatus according to claim 9, is characterized in that:
Described task enforceability determination module, time also for judging that each task that current task relies on has at least one not run succeeded, perform waiting process, and when the stand-by period is greater than or equal to default time-out time threshold value, judge that described current task performs failure, send unsuccessfully warning information, and upgrade the task result executing state of described current task.
11. task scheduling apparatus according to claim 9, is characterized in that:
Described task processing module, also for receiving the tasks carrying return code that described tasks carrying node returns, and whether judge that described tasks carrying node performs current task based on described tasks carrying return code successful, if, judge that described current task runs succeeded, and upgrade the task result executing state of described current task.
12. task scheduling apparatus according to claim 11, is characterized in that:
Described task processing module, also for when judging that described tasks carrying node execution current task is unsuccessful, judging whether to reach mission failure retry termination condition, if so, described current task being sent to another tasks carrying node to perform.
13. task scheduling apparatus according to claim 12, is characterized in that, described task processing module, when the number of retries of current task reaches predetermined failed number of retries threshold value, judges to reach mission failure retry termination condition.
14. task scheduling apparatus according to claim 12, it is characterized in that, described task processing module, also for when judging to reach mission failure retry termination condition, judge that described current task performs unsuccessful, and upgrade the task result executing state of described current task.
15. task scheduling apparatus according to claim 11, is characterized in that:
Described task processing module, also for after described current task runs succeeded, judges whether described current task is executing the task after fault recovery;
Described task trigger module, time also for judging described current task at described task module as executing the task after fault recovery, according to the order of Task Dependent figure, trigger the follow-up work of described current task, not reach trigger condition until follow-up without task or follow-up work.
16. task scheduling apparatus according to claim 9, is characterized in that, described task trigger module, when arriving the execution time of current task, judges to reach current task trigger condition.
CN201510147593.9A 2015-03-31 2015-03-31 Task scheduling method and task scheduling apparatus Pending CN104965754A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510147593.9A CN104965754A (en) 2015-03-31 2015-03-31 Task scheduling method and task scheduling apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510147593.9A CN104965754A (en) 2015-03-31 2015-03-31 Task scheduling method and task scheduling apparatus

Publications (1)

Publication Number Publication Date
CN104965754A true CN104965754A (en) 2015-10-07

Family

ID=54219789

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510147593.9A Pending CN104965754A (en) 2015-03-31 2015-03-31 Task scheduling method and task scheduling apparatus

Country Status (1)

Country Link
CN (1) CN104965754A (en)

Cited By (51)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106130929A (en) * 2016-06-17 2016-11-16 众安在线财产保险股份有限公司 The service message automatic processing method of the Internet based on graph-theoretical algorithm insurance field and system
CN106293971A (en) * 2016-08-15 2017-01-04 张家林 A kind of method and apparatus of distributed task dispatching
CN106371918A (en) * 2016-08-23 2017-02-01 北京云纵信息技术有限公司 Task cluster scheduling management method and apparatus
CN106484520A (en) * 2016-10-17 2017-03-08 北京集奥聚合科技有限公司 A kind of intelligent dispatching method based on data blood relationship and system
CN106547613A (en) * 2016-10-20 2017-03-29 西北工业大学 The scheduling of task handling process and fault recovery method
CN106970834A (en) * 2017-01-20 2017-07-21 阿里巴巴集团控股有限公司 The method and device of task processing
CN107016479A (en) * 2016-01-28 2017-08-04 五八同城信息技术有限公司 Task scheduling and managing method, apparatus and system
CN107220111A (en) * 2017-04-28 2017-09-29 华中科技大学 Method for scheduling task and system that a kind of task based access control is stolen
CN107315581A (en) * 2017-05-23 2017-11-03 努比亚技术有限公司 Mission script generating means and method, task scheduling system and method
CN107368359A (en) * 2017-05-31 2017-11-21 杭州大搜车汽车服务有限公司 A kind of asynchronous task performs method and its storage medium, device
CN107608774A (en) * 2017-09-08 2018-01-19 新智云数据服务有限公司 A kind of method for scheduling task, device, equipment and storage medium
CN107679817A (en) * 2016-08-02 2018-02-09 阿里巴巴集团控股有限公司 Workflow executing method and relevant device
CN107766402A (en) * 2017-06-27 2018-03-06 深圳市云房网络科技有限公司 A kind of building dictionary cloud source of houses big data platform
CN108037991A (en) * 2017-12-26 2018-05-15 中山大学 A kind of timing operation dispatching method and system for supporting job dependence relation
CN108268494A (en) * 2016-12-30 2018-07-10 中移(苏州)软件技术有限公司 The ETL dispatching methods and device that a kind of data processing is detached with control logic
CN108287756A (en) * 2018-01-25 2018-07-17 联动优势科技有限公司 A kind of method and device of processing task
CN108345977A (en) * 2017-01-25 2018-07-31 阿里巴巴集团控股有限公司 A kind of method for processing business and device
CN108519915A (en) * 2018-04-12 2018-09-11 北京邮电大学 Traffic assignment dispatching method and device
CN109375996A (en) * 2018-09-27 2019-02-22 安徽省鼎众金融信息咨询服务有限公司 A kind of support dependence managerial role scheduling system
CN109561148A (en) * 2018-11-30 2019-04-02 湘潭大学 Distributed task dispatching method in edge calculations network based on directed acyclic graph
CN109684229A (en) * 2019-01-08 2019-04-26 杭州雅拓信息技术有限公司 The chain type dependency analysis method and system of directed acyclic graph
CN109918180A (en) * 2018-12-14 2019-06-21 深圳壹账通智能科技有限公司 A kind of method for scheduling task, device, computer system and readable storage medium storing program for executing
CN109933414A (en) * 2017-12-18 2019-06-25 飞狐信息技术(天津)有限公司 A kind of method for scheduling task, apparatus and system
CN109992390A (en) * 2019-04-23 2019-07-09 广州裕睿信息科技有限公司 Timed task dispatching method and timed task processing method
CN110275768A (en) * 2019-06-28 2019-09-24 北京字节跳动网络技术有限公司 Data processing method, device and electronic equipment
CN110287052A (en) * 2019-06-25 2019-09-27 深圳前海微众银行股份有限公司 A kind of root of abnormal task determines method and device because of task
CN110489261A (en) * 2019-07-31 2019-11-22 上海艾融软件股份有限公司 Task handles alarm method, device and electronic equipment, storage medium
CN110543148A (en) * 2018-05-28 2019-12-06 华为技术有限公司 task scheduling method and device
CN110780981A (en) * 2018-07-31 2020-02-11 北京科杰信息技术有限公司 Task scheduling management system and method for hadoop big data platform
CN110895486A (en) * 2018-09-12 2020-03-20 北京奇虎科技有限公司 Distributed task scheduling system
CN110895484A (en) * 2018-09-12 2020-03-20 北京奇虎科技有限公司 Task scheduling method and device
CN110908821A (en) * 2019-11-08 2020-03-24 腾讯音乐娱乐科技(深圳)有限公司 Method, device, equipment and storage medium for task failure management
CN111147541A (en) * 2019-11-18 2020-05-12 广州文远知行科技有限公司 Node processing method, device and equipment based on parameter server and storage medium
CN111190725A (en) * 2019-08-09 2020-05-22 腾讯科技(深圳)有限公司 Task processing method and device, storage medium and server
CN111290911A (en) * 2020-01-21 2020-06-16 上海悦易网络信息技术有限公司 Method and equipment for detecting mobile terminal task
CN111309464A (en) * 2020-02-12 2020-06-19 杭州涂鸦信息技术有限公司 Task scheduling method and system
CN111817903A (en) * 2020-09-02 2020-10-23 湖南双菱电子科技有限公司 Link fault analysis and alarm method for digital signal transmission processing equipment
CN111950988A (en) * 2020-08-18 2020-11-17 北京字节跳动网络技术有限公司 Distributed workflow scheduling method and device, storage medium and electronic equipment
CN112346842A (en) * 2020-11-19 2021-02-09 广州嘉为科技有限公司 Workflow task scheduling method based on directed acyclic graph
CN112507171A (en) * 2020-12-03 2021-03-16 深圳市易平方网络科技有限公司 Task scheduling method, intelligent terminal and storage medium
CN112764898A (en) * 2021-01-18 2021-05-07 北京思特奇信息技术股份有限公司 Method and system for scheduling tasks among containers
CN112860389A (en) * 2019-11-27 2021-05-28 上海哔哩哔哩科技有限公司 Task scheduling method based on state machine
CN112965793A (en) * 2021-01-21 2021-06-15 中国互联网络信息中心 Data warehouse task scheduling method and system oriented to identification analysis data
CN113010531A (en) * 2021-02-05 2021-06-22 成都库珀区块链科技有限公司 Block chain BAAS system task scheduling framework based on directed acyclic graph
CN113050949A (en) * 2021-03-23 2021-06-29 上海万向区块链股份公司 JAVA-based method and system for preventing data confusion after execution of parent-child dependence JOB
CN113268318A (en) * 2021-04-07 2021-08-17 北京思特奇信息技术股份有限公司 Task scheduling method and distributed system
CN113656468A (en) * 2020-05-12 2021-11-16 北京市天元网络技术股份有限公司 Task flow triggering method and device based on NIFI
CN114612286A (en) * 2022-02-28 2022-06-10 上海深至信息科技有限公司 Image processing scheduling system and method
WO2022151668A1 (en) * 2021-01-15 2022-07-21 长鑫存储技术有限公司 Data task scheduling method and apparatus, storage medium, and scheduling tool
CN114936113A (en) * 2022-04-26 2022-08-23 北京科杰科技有限公司 Task avalanche recovery method and device, electronic equipment and storage medium
CN115878272A (en) * 2021-09-22 2023-03-31 中科寒武纪科技股份有限公司 Graph task scheduling method, execution end device, storage medium and program product

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101692208A (en) * 2009-10-15 2010-04-07 北京交通大学 Task scheduling method and task scheduling system for processing real-time traffic information
CN101963921A (en) * 2010-09-21 2011-02-02 卓望数码技术(深圳)有限公司 Operation scheduling method and system
CN103377075A (en) * 2012-04-28 2013-10-30 腾讯科技(深圳)有限公司 Task management method, device and system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101692208A (en) * 2009-10-15 2010-04-07 北京交通大学 Task scheduling method and task scheduling system for processing real-time traffic information
CN101963921A (en) * 2010-09-21 2011-02-02 卓望数码技术(深圳)有限公司 Operation scheduling method and system
CN103377075A (en) * 2012-04-28 2013-10-30 腾讯科技(深圳)有限公司 Task management method, device and system

Cited By (69)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107016479A (en) * 2016-01-28 2017-08-04 五八同城信息技术有限公司 Task scheduling and managing method, apparatus and system
CN106130929B (en) * 2016-06-17 2019-07-09 众安在线财产保险股份有限公司 The service message automatic processing method and system of internet insurance field based on graph-theoretical algorithm
CN106130929A (en) * 2016-06-17 2016-11-16 众安在线财产保险股份有限公司 The service message automatic processing method of the Internet based on graph-theoretical algorithm insurance field and system
CN107679817A (en) * 2016-08-02 2018-02-09 阿里巴巴集团控股有限公司 Workflow executing method and relevant device
CN106293971A (en) * 2016-08-15 2017-01-04 张家林 A kind of method and apparatus of distributed task dispatching
CN106371918B (en) * 2016-08-23 2019-10-18 北京云纵信息技术有限公司 Task cluster schedule management method and device
CN106371918A (en) * 2016-08-23 2017-02-01 北京云纵信息技术有限公司 Task cluster scheduling management method and apparatus
CN106484520A (en) * 2016-10-17 2017-03-08 北京集奥聚合科技有限公司 A kind of intelligent dispatching method based on data blood relationship and system
CN106547613A (en) * 2016-10-20 2017-03-29 西北工业大学 The scheduling of task handling process and fault recovery method
CN106547613B (en) * 2016-10-20 2020-04-03 西北工业大学 Scheduling and fault recovery method for task processing flow
CN108268494B (en) * 2016-12-30 2021-11-02 中移(苏州)软件技术有限公司 ETL scheduling method and device with data processing and control logic separated
CN108268494A (en) * 2016-12-30 2018-07-10 中移(苏州)软件技术有限公司 The ETL dispatching methods and device that a kind of data processing is detached with control logic
CN106970834A (en) * 2017-01-20 2017-07-21 阿里巴巴集团控股有限公司 The method and device of task processing
CN108345977A (en) * 2017-01-25 2018-07-31 阿里巴巴集团控股有限公司 A kind of method for processing business and device
CN108345977B (en) * 2017-01-25 2021-09-21 阿里巴巴集团控股有限公司 Service processing method and device
CN107220111A (en) * 2017-04-28 2017-09-29 华中科技大学 Method for scheduling task and system that a kind of task based access control is stolen
CN107220111B (en) * 2017-04-28 2019-08-09 华中科技大学 A kind of method for scheduling task that task based access control is stolen and system
CN107315581A (en) * 2017-05-23 2017-11-03 努比亚技术有限公司 Mission script generating means and method, task scheduling system and method
CN107315581B (en) * 2017-05-23 2020-09-11 努比亚技术有限公司 Task script generation device and method, and task scheduling system and method
CN107368359A (en) * 2017-05-31 2017-11-21 杭州大搜车汽车服务有限公司 A kind of asynchronous task performs method and its storage medium, device
CN107766402A (en) * 2017-06-27 2018-03-06 深圳市云房网络科技有限公司 A kind of building dictionary cloud source of houses big data platform
CN107608774A (en) * 2017-09-08 2018-01-19 新智云数据服务有限公司 A kind of method for scheduling task, device, equipment and storage medium
CN109933414A (en) * 2017-12-18 2019-06-25 飞狐信息技术(天津)有限公司 A kind of method for scheduling task, apparatus and system
CN108037991A (en) * 2017-12-26 2018-05-15 中山大学 A kind of timing operation dispatching method and system for supporting job dependence relation
CN108287756A (en) * 2018-01-25 2018-07-17 联动优势科技有限公司 A kind of method and device of processing task
CN108519915A (en) * 2018-04-12 2018-09-11 北京邮电大学 Traffic assignment dispatching method and device
CN110543148A (en) * 2018-05-28 2019-12-06 华为技术有限公司 task scheduling method and device
CN110780981B (en) * 2018-07-31 2022-03-25 北京科杰科技有限公司 Task scheduling management system and method for hadoop big data platform
CN110780981A (en) * 2018-07-31 2020-02-11 北京科杰信息技术有限公司 Task scheduling management system and method for hadoop big data platform
CN110895484A (en) * 2018-09-12 2020-03-20 北京奇虎科技有限公司 Task scheduling method and device
CN110895486A (en) * 2018-09-12 2020-03-20 北京奇虎科技有限公司 Distributed task scheduling system
CN110895486B (en) * 2018-09-12 2022-08-12 北京奇虎科技有限公司 Distributed task scheduling system
CN109375996A (en) * 2018-09-27 2019-02-22 安徽省鼎众金融信息咨询服务有限公司 A kind of support dependence managerial role scheduling system
CN109561148A (en) * 2018-11-30 2019-04-02 湘潭大学 Distributed task dispatching method in edge calculations network based on directed acyclic graph
CN109561148B (en) * 2018-11-30 2021-03-23 湘潭大学 Distributed task scheduling method based on directed acyclic graph in edge computing network
CN109918180A (en) * 2018-12-14 2019-06-21 深圳壹账通智能科技有限公司 A kind of method for scheduling task, device, computer system and readable storage medium storing program for executing
CN109684229A (en) * 2019-01-08 2019-04-26 杭州雅拓信息技术有限公司 The chain type dependency analysis method and system of directed acyclic graph
CN109992390A (en) * 2019-04-23 2019-07-09 广州裕睿信息科技有限公司 Timed task dispatching method and timed task processing method
CN110287052A (en) * 2019-06-25 2019-09-27 深圳前海微众银行股份有限公司 A kind of root of abnormal task determines method and device because of task
CN110287052B (en) * 2019-06-25 2022-01-28 深圳前海微众银行股份有限公司 Root cause task determination method and device for abnormal task
CN110275768A (en) * 2019-06-28 2019-09-24 北京字节跳动网络技术有限公司 Data processing method, device and electronic equipment
CN110489261A (en) * 2019-07-31 2019-11-22 上海艾融软件股份有限公司 Task handles alarm method, device and electronic equipment, storage medium
CN111190725A (en) * 2019-08-09 2020-05-22 腾讯科技(深圳)有限公司 Task processing method and device, storage medium and server
CN110908821A (en) * 2019-11-08 2020-03-24 腾讯音乐娱乐科技(深圳)有限公司 Method, device, equipment and storage medium for task failure management
CN110908821B (en) * 2019-11-08 2024-01-02 腾讯音乐娱乐科技(深圳)有限公司 Method, device, equipment and storage medium for task failure management
CN111147541A (en) * 2019-11-18 2020-05-12 广州文远知行科技有限公司 Node processing method, device and equipment based on parameter server and storage medium
CN112860389A (en) * 2019-11-27 2021-05-28 上海哔哩哔哩科技有限公司 Task scheduling method based on state machine
CN111290911A (en) * 2020-01-21 2020-06-16 上海悦易网络信息技术有限公司 Method and equipment for detecting mobile terminal task
CN111309464A (en) * 2020-02-12 2020-06-19 杭州涂鸦信息技术有限公司 Task scheduling method and system
CN113656468A (en) * 2020-05-12 2021-11-16 北京市天元网络技术股份有限公司 Task flow triggering method and device based on NIFI
CN113656468B (en) * 2020-05-12 2024-02-27 北京市天元网络技术股份有限公司 Task flow triggering method and device based on NIFI
CN111950988B (en) * 2020-08-18 2022-03-29 北京字节跳动网络技术有限公司 Distributed workflow scheduling method and device, storage medium and electronic equipment
CN111950988A (en) * 2020-08-18 2020-11-17 北京字节跳动网络技术有限公司 Distributed workflow scheduling method and device, storage medium and electronic equipment
CN111817903A (en) * 2020-09-02 2020-10-23 湖南双菱电子科技有限公司 Link fault analysis and alarm method for digital signal transmission processing equipment
CN111817903B (en) * 2020-09-02 2020-12-01 湖南双菱电子科技有限公司 Link fault analysis and alarm method for digital signal transmission processing equipment
CN112346842B (en) * 2020-11-19 2024-03-26 广州嘉为科技有限公司 Workflow task scheduling method based on directed acyclic graph
CN112346842A (en) * 2020-11-19 2021-02-09 广州嘉为科技有限公司 Workflow task scheduling method based on directed acyclic graph
CN112507171A (en) * 2020-12-03 2021-03-16 深圳市易平方网络科技有限公司 Task scheduling method, intelligent terminal and storage medium
WO2022151668A1 (en) * 2021-01-15 2022-07-21 长鑫存储技术有限公司 Data task scheduling method and apparatus, storage medium, and scheduling tool
CN112764898A (en) * 2021-01-18 2021-05-07 北京思特奇信息技术股份有限公司 Method and system for scheduling tasks among containers
CN112965793B (en) * 2021-01-21 2023-11-21 中国互联网络信息中心 Identification analysis data-oriented data warehouse task scheduling method and system
CN112965793A (en) * 2021-01-21 2021-06-15 中国互联网络信息中心 Data warehouse task scheduling method and system oriented to identification analysis data
CN113010531A (en) * 2021-02-05 2021-06-22 成都库珀区块链科技有限公司 Block chain BAAS system task scheduling framework based on directed acyclic graph
CN113010531B (en) * 2021-02-05 2022-11-01 成都库珀创新科技有限公司 Block chain BAAS system task scheduling framework based on directed acyclic graph
CN113050949A (en) * 2021-03-23 2021-06-29 上海万向区块链股份公司 JAVA-based method and system for preventing data confusion after execution of parent-child dependence JOB
CN113268318A (en) * 2021-04-07 2021-08-17 北京思特奇信息技术股份有限公司 Task scheduling method and distributed system
CN115878272A (en) * 2021-09-22 2023-03-31 中科寒武纪科技股份有限公司 Graph task scheduling method, execution end device, storage medium and program product
CN114612286A (en) * 2022-02-28 2022-06-10 上海深至信息科技有限公司 Image processing scheduling system and method
CN114936113A (en) * 2022-04-26 2022-08-23 北京科杰科技有限公司 Task avalanche recovery method and device, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
CN104965754A (en) Task scheduling method and task scheduling apparatus
US11048688B2 (en) Deleting configuration items in a configuration management database
CN110069572B (en) HIVE task scheduling method, device, equipment and storage medium based on big data platform
CN107016480B (en) Task scheduling method, device and system
CN109358947B (en) Method and system for realizing virtual machine snapshot
US8782643B2 (en) Device and method for controlling communication between BIOS and BMC
CN109325016B (en) Data migration method, device, medium and electronic equipment
CN112527879B (en) Kafka-based real-time data extraction method and related equipment
EP3147797B1 (en) Data management method, node and system for database cluster
CN110673936B (en) Breakpoint continuous operation method and device for arrangement service, storage medium and electronic equipment
US20140195861A1 (en) Implementing rate controls to limit timeout-based faults
US10855750B2 (en) Centralized management of webservice resources in an enterprise
CN110740145A (en) Message consumption method, device, storage medium and electronic equipment
US9990231B2 (en) Resource pre-configuration
US9864649B2 (en) Technologies for root cause identification of use-after-free memory corruption bugs
US11409704B2 (en) Method, device and computer program product for managing storage system
EP3696658A1 (en) Log management method, server and database system
CN107908370B (en) Data storage method and device
US20220035695A1 (en) Computer unit, computer system and event management method
US8984336B1 (en) Systems and methods for performing first failure data captures
CN106648985A (en) Disaster-tolerant repair method and device of text database
US11874821B2 (en) Block aggregation for shared streams
US9218256B1 (en) Systems and methods for shipping I/O operations to prevent replication failure
CN110658989B (en) System and method for backup storage garbage collection
US20220164219A1 (en) Processing system, processing method, higher-level system, lower-level system, higher-level program, and lower-level program

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20151007

RJ01 Rejection of invention patent application after publication