CN104239148A - Distributed task scheduling method and device - Google Patents
Distributed task scheduling method and device Download PDFInfo
- Publication number
- CN104239148A CN104239148A CN201310223395.7A CN201310223395A CN104239148A CN 104239148 A CN104239148 A CN 104239148A CN 201310223395 A CN201310223395 A CN 201310223395A CN 104239148 A CN104239148 A CN 104239148A
- Authority
- CN
- China
- Prior art keywords
- task
- subtask
- verification
- execution device
- scheduling request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a distributed task scheduling method and device. The method comprises receiving a task scheduling request sent by a task executing device, wherein the task scheduling request contains the task type and the identity information of the task executing device; according to the task scheduling request, verifying at least one subtask according to issuing verification logic to obtain a verified subtask set; selecting subtasks from the subtask set and issuing the subtasks to the task executing device; receiving executing results fed back from the task executing device, and correcting the attributes of the issued subtasks according to the executing results. The distributed task scheduling method can achieve stateless communication between a task scheduling device and the task executing device, conveniently expand the task executing device linearly and improve the task executing capacity, and through centralized management and scheduling of different tasks applicable to different task executing devices, achieves overall correlation among the different tasks and accordingly support dependency among the different tasks.
Description
Technical field
The present invention relates to task scheduling technique field, particularly relate to a kind of distributed task dispatching method and device.
Background technology
In general, scheduling is exactly according to certain strategy, task is reasonably assigned on processing unit and performs, to reach deadline the shortest, object such as raising system availability etc.Scheduling is one of gordian technique realizing high-performance calculation.
Along with the expanding day of computer utility scope, further developing of distributed system and cloud computing technology and network technology, has and more and more multiple distributed system is integrated the demand of carrying out unified management.At present, distributed system in network is mostly that all departments will run constructed by one or more operating information systems, thus these distributed systems are all separate mostly, there is no the incidence relation what is mutual, because do not get through between each distributed system, cannot be compatible, cannot integrate, thus also cannot carry out unified management.
Therefore, no matter the unified management from system and resource sharing, or need to carry out communicating etc. between each distributed system, whether the different task global association that different machines runs can be got through, the support that the polytype realizing the multiple-task cycle relies on and overall scheduling are very important problems.But, existing distributed system, that scheduling by overlapping dispersion is managed independently more to Hadoop, Linux and/or various dissimilar database, cannot get through, one-stop task management can not be realized, understanding difficulty and the work complexity of data analyst and data maintenance personnel can be increased undoubtedly, and then the operation and maintenance cost of system can be increased.
Summary of the invention
In view of this, the embodiment of the present invention provides a kind of distributed task dispatching method and device, solves above problem.Described technical scheme is as follows:
A kind of distributed task dispatching method, comprising:
Receive the task scheduling request that task execution device sends, wherein said task scheduling request comprises the identification information of task type and described task execution device;
According to described task scheduling request, according to issuing check logic, at least one subtask is verified, obtain the subtask set by verification, the attribute of wherein said subtask comprises: subtask identifier, the identifier of subtask type and affiliated task, and the attribute of task comprises: task identifier, task type, the subtask type of subtask is identical with the task type of the task belonging to described subtask;
From the set of described subtask, chooser task is issued to described task execution device;
Receive the execution result that described task execution device returns, according to the attribute of the subtask issued described in execution result amendment.
According to same design of the present invention, present invention also offers a kind of distributed task dispatching device, comprising:
Dispatch request receiving element, for receiving the task scheduling request that task execution device sends, wherein said task scheduling request comprises the identification information of task type and described task execution device;
Issue verification unit, for according to described task scheduling request, according to issuing check logic, at least one subtask is verified, obtain the subtask set by verification, the attribute of wherein said subtask comprises: subtask identifier, the identifier of subtask type and affiliated task, and the attribute of task comprises: task identifier, task type, the subtask type of subtask is identical with the task type of the task belonging to described subtask;
Issue unit, be issued to described task execution device for chooser task from the set of described subtask;
Result feedback unit, for receiving the execution result that described task execution device returns, according to the attribute of the subtask issued described in execution result amendment.
According to same design of the present invention, present invention also offers a kind of server, comprising: display, one or more than one processor; Storer; And one or more than one program, wherein said more than one or one program is stored in described storer, and be configured to be performed by described more than one or one processor, described more than one or one routine package is containing the instruction for carrying out following operation:
Receive the task scheduling request that task execution device sends, wherein said task scheduling request comprises the identification information of task type and described task execution device;
According to described task scheduling request, according to issuing check logic, at least one subtask is verified, obtain the subtask set by verification, the attribute of wherein said subtask comprises: subtask identifier, the identifier of subtask type and affiliated task, and the attribute of task comprises: task identifier, task type, the subtask type of subtask is identical with the task type of the task belonging to described subtask;
From the set of described subtask, chooser task is issued to described task execution device;
Receive the execution result that described task execution device returns, according to the attribute of the subtask issued described in execution result amendment.
The beneficial effect that the technical scheme that the embodiment of the present invention provides is brought is:
Task scheduling apparatus is after the task scheduling request receiving task execution device transmission, according to issuing check logic, subtask is verified, subtask by verification is issued to task execution device, achieve the stateless communication between task scheduling apparatus and task execution device, conveniently can carry out linear expansion to task execution device, the executive capability of task can be promoted; By managing concentratedly the different task being applicable to different task actuating unit and dispatching, the global association between different task can be realized, the dependence be applicable between the different task of different task actuating unit can be supported.
Accompanying drawing explanation
Fig. 1 is the distributed task dispatching method flow diagram described in the embodiment of the present invention one;
Fig. 2 is the task analytic approach process flow diagram described in the embodiment of the present invention one;
Fig. 3 is the distributed task dispatching method flow diagram described in the embodiment of the present invention two;
Fig. 4 is the distributed task dispatching device schematic diagram described in the embodiment of the present invention three;
Fig. 5 is the distributed task dispatching apparatus structure block diagram described in the embodiment of the present invention three;
Fig. 6 is the task execution device schematic layout pattern described in the embodiment of the present invention three;
Fig. 7 is the distributed task dispatching apparatus structure block diagram described in the embodiment of the present invention four.
Embodiment
In order to make the object of the invention, technical scheme and advantage clearly, further illustrate technical scheme of the present invention below in conjunction with accompanying drawing by specific embodiment.
Embodiment one
Fig. 1 is the distributed task dispatching method flow diagram described in the present embodiment, and the present embodiment mainly carries out the method for task scheduling and execution in a distributed manner between task scheduling apparatus (being generally the director server in distributed task dispatching system) and task execution device in task scheduling system (such as Oracle puts functional module in storage, Linux data upload functional module, Hadoop check account module etc.).As shown in Figure 1, the distributed task dispatching method described in the present embodiment comprises:
The task scheduling request that S101, reception task execution device send.
Task scheduling apparatus receives the task scheduling request that task execution device sends, and wherein said task scheduling request comprises the identification information of task type (such as Oracle puts task, Linux data upload task, Hadoop MapReduce task dispatching in storage) and described task execution device.The identification information of wherein said task execution device is used for each task execution device difference to come.In the present embodiment, the identification information of described task execution device can be No. ID or title of task execution device.
S102, according to task scheduling request, subtask to be verified, obtain the subtask set by verification.
Wherein, the number of verification viewing system scale and subtask carried out to how many subtasks and determine.If number school, subtask is few or system scale school is little, can realize verifying all subtasks, from all subtasks, obtain the subtask set by verification; If subtask number is more or system scale school large, can realize verifying parton task, such as, stop comparatively testing other subtasks when obtaining and reaching preset value by the subtask number of verification.
The subtask that the task scheduling request that selection and task execution device send from subtask is suitable, obtains the subtask set by verification.
Wherein, subtask is by carrying out Task-decomposing acquisition to task, and Fig. 2 is the task analytic approach process flow diagram of the present embodiment, and as shown in Figure 2, the task analytic approach described in the present embodiment comprises:
S201, reception also store tasks.
Task comprises different task types, and the attribute of various types of task comprises: task identifier (such as task names or task ID number), and task type, also can comprise entry-into-force time, duty cycle, task execution device person liable etc.And the task of different task type, its attribute is different, if the task of certain type relates to carry out data transmission (such as by the Data Migration in HDFS in MYSQL) between two machines, then the task of the type also comprises the information such as source server identifier (as IP address), destination server identifier (as IP address).
The mode receiving also store tasks can have multiple, such as, by providing special subtask to submit to software or subtask to submit to webpage to realize.In a preferred implementation of the present embodiment, webpage is submitted to supply user to submit subtask to by providing special subtask.Due to different task types, the mission bit stream needing user to provide is different, on webpage, can list all registered task types by the form of drop-down list, when user selects the type wanting submitting to of task, the page dynamically provides the list of the type task.
Such as, user creates the data loading task that a type is " HDFS warehouse-in is to MYSQL ", the task names that user fills in is " APP timesharing overview model warehouse-in MySQL ", this task is used for the suffix under the B file path of A server being called E, F, G, the H in the D table under the file of * .C ... Deng the data of row, respectively according to f1, f2, f3, f4 ... e, f, g, the h in the d table of a service is moved to etc. mapping relations ... Deng row.Duty cycle is once per minute, and the entry-into-force time is " 2013-04-0116:00:00 ", when the time that task can be activated every day is the earliest 00 00 point, performs 100000 data at every turn.
And for example, user creates the task that a type is " Hadoop MapReduce calculates ", the task names that user fills in is " APP timesharing overview daily calculates ", for the suffix under HDFS cluster A upper pathway B being called the data file D of .C, calculate according to the Map class F performed inside Jar bag E and Reduce class G, and under result being outputted to the path H on cluster A.Duty cycle is once a day, and the entry-into-force time is " 2013-04-0100:00:00 ".
S202, according to preset task decomposition rule, Task-decomposing is become subtask.
According to task attribute, according to the Task-decomposing rule of the type task preset, Task-decomposing is become subtask.Such as, according to the entry-into-force time of task as initial time Task-decomposing being become subtask, subtask can be generated successively according to duty cycle, by the subtask write subtask list of generation.
If first time carries out Task-decomposing to certain task, then the initial time of first subtask generated is set as the entry-into-force time of task, and closing time of first subtask is the time adding entry-into-force time of task a duty cycle.
If not first time, Task-decomposing is carried out to certain task, the initial time of the subtask then generated is set as that the generation ordinal number of this subtask subtracts the time of being multiplied by a duty cycle after one, the subtask of generation be set as that the generation ordinal number of this subtask is multiplied by the time of a duty cycle closing time.
When the system time of distributed task dispatching system reaches the closing time of certain subtask, then by this subtask write subtask list.In the subtask of certain task, there is the system time being later than distributed task dispatching system closing time of a subtask or be later than the task out-of-service time, then suspending and Task-decomposing is carried out to this task.
The subtask that S203, storage generate.
Subtask comprises with properties: the identifier (such as task ID number), subtask type etc. of task belonging to subtask identifier (such as No. ID, subtask), subtask, and belonging to wherein said subtask type and this subtask, the task type of task is identical.
Behind each generation subtask, subtask is stored, for task scheduling apparatus scheduling, such as, in the form of a list subtask is stored as subtask list.
S103, from subtask set chooser task be issued to task execution device.
The subtask selecting the task scheduling request that sends with task execution device suitable from the subtask set that previous step obtains is issued to this task execution device.
Such as, if receive the task scheduling request that a task execution device sends, the identification information of its actuating unit is IP address (such as 172.23.1.100), the task type of request is that Hadoop MapReduce calculates, request time is 2013-04-0116:01:15, subtask then in antithetical phrase set of tasks verifies according to the check logic that issues preset, filter out the subtask suitable with described task scheduling request, such as to select task identifier be the initial time of A is the subtask of 2013-04-0100:00:00, is handed down to this task execution device.
The execution result that S104, reception task execution device return, the attribute of amendment subtask.
The success of group tasks carrying, can remove subtask list and store separately by this subtask, or the attribute of amendment subtask, identifies the state of this subtask.If the latter, then the attribute of subtask also can comprise: the information such as subtask state and number of retries.When receiving the execution result that task execution device returns, according to the status information returning results amendment subtask.When running succeeded, sending the information that runs succeeded of subtask, when task scheduling apparatus receives the information that subtask runs succeeded, parsing subtask identifier, changing the state of this subtask in the list of subtask into success status (SUCCESSFUL).When performing unsuccessfully, sending subtask and performing failed information, when task scheduling apparatus receives the failed information of subtask execution, parse subtask and identifier, the number of retries of this subtask in the list of subtask is added 1.
In order to control the inefficiency that the too much subtask of the execution frequency of failure is directed at, also can limit the number of retries of subtask, when the number of retries of subtask reaches default threshold value, then no longer issuing this subtask.
In the technical scheme of the distributed task dispatching method described in the present embodiment, task scheduling apparatus is after the task scheduling request receiving task execution device transmission, according to issuing check logic, subtask is verified, subtask by verification is issued to task execution device, achieve the stateless communication between task scheduling apparatus and task execution device, conveniently can carry out linear expansion to task execution device, the executive capability of task can be promoted; By managing concentratedly the different task being applicable to different task actuating unit and dispatching, the global association between different task can be realized, the dependence be applicable between the different task of different task actuating unit can be supported.
Embodiment two
Fig. 3 is the distributed task dispatching method flow diagram described in the present embodiment, and the present embodiment mainly carries out the method for task scheduling and execution in a distributed manner between task scheduling apparatus (being generally the director server in distributed task dispatching system) and task execution device in task scheduling system (such as Oracle puts functional module in storage, Linux data upload functional module, Hadoop check account module etc.).As shown in Figure 3, the distributed task dispatching method described in the present embodiment comprises:
The task scheduling request that S301, reception task execution device send.
Task scheduling apparatus receives the task scheduling request that task execution device sends, and wherein said task scheduling request comprises the identification information of task type (such as Oracle puts task, Linux data upload task, Hadoop MapReduce task dispatching in storage) and described task execution device.The identification information of wherein said task execution device is used for each task execution device difference to come.In the present embodiment, the identification information of described task execution device can be No. ID or title of task execution device.
S302, according to issuing check logic, subtask to be verified according to task scheduling request, obtaining the subtask set by verification.
Wherein, the number of verification viewing system scale and subtask carried out to how many subtasks and determine.If number school, subtask is few or system scale school is little, can realize verifying all subtasks, from all subtasks, obtain the subtask set by verification; If subtask number is more or system scale school large, can realize verifying parton task, such as, stop comparatively testing other subtasks when obtaining and reaching preset value by the subtask number of verification.
The subtask that the task scheduling request that selection and task execution device send from subtask is suitable, obtains the subtask set by verification.
Wherein, subtask is by carrying out Task-decomposing acquisition to task, and its concrete mode is as shown in embodiment one, and therefore not to repeat here.
In a preferred implementation of the present embodiment, described in issue check logic and comprise: according to preset order, task verification, subtask verification, actuating unit verification and/or dependence verification are carried out to subtask.
Task verification comprises, and the attribute of task belonging to subtask, judges whether subtask meets task scheduling request.Particularly, task verification can comprise:
(1) whether the task type of task belonging to syndrome task is with consistent with the task type of task execution device request.
(2) belonging to syndrome task, whether task is restricted to task execution device, such as, whether limit the identification information (such as IP address) of task execution device whether within the scope of restriction.The attribute of such as subtask also comprises task execution device information, comprising: allow all task execution devices, only allow some or more than one identification information to be the task execution device of particular value.
(3) whether be effective, namely whether verification can be in effective status by task belonging to the subtask that issues if verifying the state of task.Whether the attribute information of such as task also comprises effective, when subtask is verified,
Obtain the affiliated task of this subtask, judge whether effectively whether the attribute of described affiliated task be effective, if not effectively then show that this subtask is not by verification.
(4) whether the priority of task belonging to syndrome task belongs within the priority limit of the task execution device registration sending task requests.The attribute information of such as task also comprises priority, the log-on message of task execution device in task scheduling apparatus comprises identification information and the priority limit of task execution device, the priority limit of this task execution device is obtained by the identification information of the task execution device in task scheduling request, when subtask is verified, obtain the affiliated task of this subtask, judge described belonging to the priority of task whether within the priority limit of described task execution device.
Such as, the task execution device X laid respectively on machine A, B, C type of all executing the task is the task of Y, only be absorbed in make the task execution device on machine A and do the high task of priority, the priority limit that then can limit the task execution device X on machine A is 8-10, then when the task execution device X on machine A asks subtask, the task of priority limit to be the task type of 1-7 be Y can not issue.
(5) dependence that whether task belonging to syndrome task meets between this task and other tasks limits.
Subtask verification comprises, and judges whether each subtask meets issue condition according to the information of subtask.Particularly, subtask verification can comprise:
(1) whether syndrome task is the subtask can forcing to issue.
Such as, a kind of subtask type can be set, for the different subtasks belonging to same task, the subtask of this subtask type be had and the highest issues priority.
(2) whether the state of syndrome task is can by the state issued.Such as, the subtask of ready state (READY) can be issued, and the subtask that task status is success status (SUCCESSFUL) can not be issued.
(3) whether syndrome task is later than the request time of task scheduling apparatus closing time.If just can be issued.
(4) whether syndrome task meets time delay.
Such as, require that the closing time that current time is later than subtask adds time delay, just can be issued.
(5) whether syndrome task meets start-up time.
Such as, require that current time is later than the start-up time on the same day of subtask, just can be issued.
(6) whether syndrome task meets the dependence between this subtask and other subtasks.
(7) whether syndrome task meets the number of attempt restriction preset.
In order to control the inefficiency that the too much subtask of the execution frequency of failure is directed at, also can limit the number of retries of subtask, when the number of retries of subtask reaches default threshold value,
(8) whether the subtask verifying retry reaches the retry stand-by period.The attribute of such as subtask also comprises the retry stand-by period, and the described retry stand-by period is for limiting the minimum time also needing when subtask issues unsuccessfully to wait for.
Actuating unit verification comprises, and according to the log-on message of the task execution device of transmission task requests, obtains the trustship machine information of described task execution device, judges whether described trustship machine information meets the qualifications of subtask to trustship machine.Particularly, actuating unit verification comprises:
(1) the restriction whether identification information sending the task execution device of task requests meets subtask is verified.The attribute of such as subtask also comprises the identification information of task execution device, for limiting the task execution device performing this subtask.
(2) verification sends whether the current state of the task execution device of task requests is effective.
(3) verification sends the restriction whether subtask that the task execution device of task requests runs simultaneously meets subtask.
(4) priority of syndrome task whether meet send task requests task execution device to the qualifications of the priority of subtask.
(5) verify the number sending the subtask that the task execution device of task requests is performing whether to meet by the restriction of the subtask verified.
In a preferred implementation of the present embodiment, the preposition subtask of dependence verification for obtaining subtask, according to the dependence between described subtask and described preposition subtask, judges whether described subtask meets the condition of issuing.
It should be noted that, concrete issues in check logic, the order of various verification the order of the unalterable of non-critical, within all spirit in present embodiment and principle, the increasing of any step, delete and/or exchange, be all included within the method described in the present embodiment.
S303, from subtask set chooser task be issued to task execution device.
Through above-mentioned issue check logic verification after, sorted according to preset algorithm in the subtask met, first subtask be issued to the task execution device of described transmission task requests.
Particularly, sequence comprises: to each subtask in the set of described subtask according to number of attempt Bit-reversed, carry out order according to priority and sort, sort according to period type number sequence or sort according to order working time.
The execution result that S304, reception task execution device return, the attribute of amendment subtask.
The success of group tasks carrying, can remove subtask list and store separately by this subtask, or the attribute of amendment subtask, identifies the state of this subtask.If the latter, then the attribute of subtask also can comprise: the information such as subtask state and number of retries.When receiving the execution result that task execution device returns, according to the status information returning results amendment subtask.When running succeeded, sending the information that runs succeeded of subtask, when task scheduling apparatus receives the information that subtask runs succeeded, parsing subtask identifier, changing the state of this subtask in the list of subtask into success status (SUCCESSFUL).When performing unsuccessfully, sending subtask and performing failed information, when task scheduling apparatus receives the failed information of subtask execution, parse subtask and identifier, the number of retries of this subtask in the list of subtask is added 1.
In order to control the inefficiency that the too much subtask of the execution frequency of failure is directed at, also can limit the number of retries of subtask, when the number of retries of subtask reaches default threshold value, then no longer issuing this subtask.
In the technical scheme of the distributed task dispatching method described in the present embodiment, collect by issuing verification sieve subtask is verified, obtain the subtask set by verification, meticulousr centralized management and scheduling can be carried out to the different task being applicable to different task actuating unit, the global association between different task can be realized, the dependence be applicable between the different task of different task actuating unit can be supported.
Embodiment three
The present embodiment proposes a kind of distributed task dispatching device, Fig. 4 is the distributed task dispatching device schematic diagram described in the present embodiment, as described in Figure 4, distributed task dispatching device described in the present embodiment can cover all data analysis systems, comprises Postgre, TDW, Linux, Hadoop, Oracle, Mysql etc.
Fig. 5 is the distributed task dispatching apparatus structure block diagram described in the present embodiment, as shown in Figure 5, the distributed task dispatching device 501 described in the present embodiment is for receiving and respond the task scheduling request of task execution device 1, task execution device 2, task execution device N etc.
Described distributed task dispatching device 501 comprises: dispatch request receiving element 5011, issue verification unit 5012, issue unit 5013 and result feedback unit 5014.
Dispatch request receiving element 5011, for receiving the task scheduling request that task execution device sends, wherein said task scheduling request comprises the identification information of task type and described task execution device.
Wherein said task scheduling request comprises the identification information of task type (such as Oracle puts task, Linux data upload task, Hadoop MapReduce task dispatching in storage) and described task execution device.The identification information of wherein said task execution device is used for each task execution device difference to come.In the present embodiment, the identification information of described task execution device can be No. ID or title of task execution device.
Issue verification unit 5012, for according to described task scheduling request, according to issuing check logic, at least one subtask is verified, obtain the subtask set by verification, the attribute of wherein said subtask comprises: subtask identifier, the identifier of subtask type and affiliated task, and the attribute of task comprises: task identifier, task type, the subtask type of subtask is identical with the task type of the task belonging to described subtask.
Wherein, the number of verification viewing system scale and subtask carried out to how many subtasks and determine.If number school, subtask is few or system scale school is little, can realize verifying all subtasks, from all subtasks, obtain the subtask set by verification; If subtask number is more or system scale school large, can realize verifying parton task, such as, stop comparatively testing other subtasks when obtaining and reaching preset value by the subtask number of verification.
The subtask that the task scheduling request that selection and task execution device send from subtask is suitable, obtains the subtask set by verification.
Wherein, subtask is by carrying out Task-decomposing acquisition to task, and its concrete mode is as described in embodiment one, and therefore not to repeat here.
Issue unit 5013, be issued to for chooser task from the set of subtask the task execution device sending task scheduling request.
Issue after verification unit 5012 carries out issuing check logic verification through above-mentioned, sorted according to preset algorithm in the subtask met, first subtask is issued to the task execution device of described transmission task requests.
Particularly, sequence comprises: to each subtask in the set of described subtask according to number of attempt Bit-reversed, carry out order according to priority and sort, sort according to period type number sequence or sort according to order working time.
Result feedback unit 5014, for receiving the execution result that described task execution device returns, according to the attribute of the subtask issued described in execution result amendment.
The success of group tasks carrying, can remove subtask list and store separately by this subtask, or the attribute of amendment subtask, identifies the state of this subtask.If the latter, then the attribute of subtask also can comprise: the information such as subtask state and number of retries.When receiving the execution result that task execution device returns, according to the status information returning results amendment subtask.When running succeeded, sending the information that runs succeeded of subtask, when task scheduling apparatus receives the information that subtask runs succeeded, parsing subtask identifier, changing the state of this subtask in the list of subtask into success status (SUCCESSFUL).When performing unsuccessfully, sending subtask and performing failed information, when task scheduling apparatus receives the failed information of subtask execution, parse subtask and identifier, the number of retries of this subtask in the list of subtask is added 1.
In order to control the inefficiency that the too much subtask of the execution frequency of failure is directed at, also can limit the number of retries of subtask, when the number of retries of subtask reaches default threshold value, then no longer issuing this subtask.
The technical scheme of the distributed task dispatching device described in the present embodiment adopts distributed architectural framework to communicate with stateless, achieve the stateless communication between task scheduling apparatus and task execution device, conveniently can carry out linear expansion to task execution device, the executive capability of task can be promoted; By managing concentratedly the different task being applicable to different task actuating unit and dispatching, the global association between different task can be realized, the dependence be applicable between the different task of different task actuating unit can be supported.
Embodiment four
Fig. 7 is the distributed task dispatching apparatus structure block diagram described in the present embodiment, as shown in Figure 7, the distributed task dispatching device 701 described in the present embodiment is for receiving and responding the task scheduling requests such as task execution device 1, task execution device 2, task execution device N.
Described distributed task dispatching device 701 comprises: dispatch request receiving element 7011, issue verification unit 7012, issue unit 7013 and result feedback unit 7014.
Dispatch request receiving element 7011, for receiving the task scheduling request that task execution device sends, wherein said task scheduling request comprises the identification information of task type and described task execution device.
Wherein said task scheduling request comprises the identification information of task type (such as Oracle puts task, Linux data upload task, Hadoop MapReduce task dispatching in storage) and described task execution device.The identification information of wherein said task execution device is used for each task execution device difference to come.In the present embodiment, the identification information of described task execution device can be No. ID or title of task execution device.
Issuing verification unit 7012, for according to described task scheduling request, according to issuing check logic, at least one subtask being verified, obtain the subtask set by verification.
Wherein, the number of verification viewing system scale and subtask carried out to how many subtasks and determine.If number school, subtask is few or system scale school is little, can realize verifying all subtasks, from all subtasks, obtain the subtask set by verification; If subtask number is more or system scale school large, can realize verifying parton task, such as, stop comparatively testing other subtasks when obtaining and reaching preset value by the subtask number of verification.
The subtask that the task scheduling request that selection and task execution device send from subtask is suitable, obtains the subtask set by verification.
Wherein, subtask is by carrying out Task-decomposing acquisition to task, and its concrete mode is as shown in embodiment one, and therefore not to repeat here.
In a preferred implementation of the present embodiment, described verification unit 7012 comprises task syndrome unit 70121, subtask syndrome unit 70122, actuating unit syndrome unit 70123 and dependence syndrome unit 70124:
According to preset order by described task syndrome unit 70121, subtask syndrome unit 70122, actuating unit syndrome unit 70123 and dependence syndrome unit 70124, task verification, subtask verification, actuating unit verification and/or dependence verification are carried out to subtask.
Task syndrome unit 70121 for: the attribute of task belonging to subtask, judges whether subtask meets task scheduling request.Particularly, task syndrome unit 70121 can be used for:
(1) whether the task type of task belonging to syndrome task is with consistent with the task type of task execution device request.
(2) belonging to syndrome task, whether task is restricted to task execution device, such as, whether limit the identification information (such as IP address) of task execution device whether within the scope of restriction.The attribute of such as subtask also comprises task execution device information, comprising: allow all task execution devices, only allow some or more than one identification information to be the task execution device of particular value.
(3) whether be effective, namely whether verification can be in effective status by task belonging to the subtask that issues if verifying the state of task.Whether the attribute information of such as task also comprises effective, when verifying subtask, obtains the affiliated task of this subtask, judges whether effectively whether the attribute of described affiliated task be effective, if not effectively then show that this subtask is not by verification.
(4) whether the priority of task belonging to syndrome task belongs within the priority limit of the task execution device registration sending task requests.The attribute information of such as task also comprises priority, the log-on message of task execution device in task scheduling apparatus comprises identification information and the priority limit of task execution device, the priority limit of this task execution device is obtained by the identification information of the task execution device in task scheduling request, when subtask is verified, obtain the affiliated task of this subtask, judge described belonging to the priority of task whether within the priority limit of described task execution device.
Such as, the task execution device X laid respectively on machine A, B, C type of all executing the task is the task of Y, only be absorbed in make the task execution device on machine A and do the high task of priority, the priority limit that then can limit the task execution device X on machine A is 8-10, then when the task execution device X on machine A asks subtask, the task of priority limit to be the task type of 1-7 be Y can not issue.
(5) dependence that whether task belonging to syndrome task meets between this task and other tasks limits.
Subtask syndrome unit 70122 for: judge that whether each subtask meets according to the information of subtask and issue condition.Particularly, subtask syndrome unit 70122 for:
(1) whether syndrome task is the subtask can forcing to issue.
Such as, a kind of subtask type can be set, for the different subtasks belonging to same task, the subtask of this subtask type be had and the highest issues priority.
(2) whether the state of syndrome task is can by the state issued.Such as, the subtask of ready state (READY) can be issued, and the subtask that task status is success status (SUCCESSFUL) can not be issued.
(3) whether syndrome task is later than the request time of task scheduling apparatus closing time.If just can be issued.
(4) whether syndrome task meets time delay.
Such as, require that the closing time that current time is later than subtask adds time delay, just can be issued.
(5) whether syndrome task meets start-up time.
Such as, require that current time is later than the start-up time on the same day of subtask, just can be issued.
(6) whether syndrome task meets the dependence between this subtask and other subtasks.
(7) whether syndrome task meets the number of attempt restriction preset.
In order to control the inefficiency that the too much subtask of the execution frequency of failure is directed at, also can limit the number of retries of subtask, when the number of retries of subtask reaches default threshold value,
(8) whether the subtask verifying retry reaches the retry stand-by period.The attribute of such as subtask also comprises the retry stand-by period, and the described retry stand-by period is for limiting the minimum time also needing when subtask issues unsuccessfully to wait for.
Actuating unit syndrome unit 70123 for: according to the log-on message of task execution device sending task requests, obtain the trustship machine information of described task execution device, judge whether described trustship machine information meets the qualifications of subtask to trustship machine.Particularly, actuating unit syndrome unit 70123 can be used for:
(1) the restriction whether identification information sending the task execution device of task requests meets subtask is verified.The attribute of such as subtask also comprises the identification information of task execution device, for limiting the task execution device performing this subtask.
(2) verification sends whether the current state of the task execution device of task requests is effective.
(3) verification sends the restriction whether subtask that the task execution device of task requests runs simultaneously meets subtask.
(4) priority of syndrome task whether meet send task requests task execution device to the qualifications of the priority of subtask.
(5) verify the number sending the subtask that the task execution device of task requests is performing whether to meet by the restriction of the subtask verified.
In a preferred implementation of the present embodiment, the preposition subtask of dependence verification for obtaining subtask, according to the dependence between described subtask and described preposition subtask, judges whether described subtask meets the condition of issuing.
It should be noted that, concrete issues in check logic, the order of various verification the order of the unalterable of non-critical, within all spirit in present embodiment and principle, the increasing of any step, delete and/or exchange, be all included within the method described in the present embodiment.
Issue unit 7013, be issued to described task execution device for chooser task from the set of subtask.
Through above-mentioned issue check logic verification after, sorted according to preset algorithm in the subtask met, first subtask be issued to the task execution device of described transmission task requests.
Particularly, sequence comprises: to each subtask in the set of described subtask according to number of attempt Bit-reversed, carry out order according to priority and sort, sort according to period type number sequence or sort according to order working time.
Result feedback unit 7014, for receiving the execution result that described task execution device returns, according to the attribute of the subtask issued described in execution result amendment.
The success of group tasks carrying, can remove subtask list and store separately by this subtask, or the attribute of amendment subtask, identifies the state of this subtask.If the latter, then the attribute of subtask also can comprise: the information such as subtask state and number of retries.When receiving the execution result that task execution device returns, according to the status information returning results amendment subtask.When running succeeded, sending the information that runs succeeded of subtask, when task scheduling apparatus receives the information that subtask runs succeeded, parsing subtask identifier, changing the state of this subtask in the list of subtask into success status (SUCCESSFUL).When performing unsuccessfully, sending subtask and performing failed information, when task scheduling apparatus receives the failed information of subtask execution, parse subtask and identifier, the number of retries of this subtask in the list of subtask is added 1.
In order to control the inefficiency that the too much subtask of the execution frequency of failure is directed at, also can limit the number of retries of subtask, when the number of retries of subtask reaches default threshold value, then no longer issuing this subtask.
The present embodiment also proposed a kind of server, comprising: display, one or more than one processor; Storer; And one or more than one program, wherein said more than one or one program is stored in described storer, and be configured to be performed by described more than one or one processor, described more than one or one routine package is containing the instruction for carrying out following operation:
Receive the task scheduling request that task execution device sends, wherein said task scheduling request comprises the identification information of task type and described task execution device;
According to described task scheduling request, according to issuing check logic, at least one subtask is verified, obtain the subtask set by verification, the attribute of wherein said subtask comprises: subtask identifier, the identifier of subtask type and affiliated task, and the attribute of task comprises: task identifier, task type, the subtask type of subtask is identical with the task type of the task belonging to described subtask;
From the set of described subtask, chooser task is issued to described task execution device;
Receive the execution result that described task execution device returns, according to the attribute of the subtask issued described in execution result amendment.
The present embodiment also proposed a kind of computer-readable recording medium, described computer-readable recording medium stores more than one or one program, described more than one or one program is used for execution distributed task dispatching method by one or more than one processor, and described method comprises:
Receive the task scheduling request that task execution device sends, wherein said task scheduling request comprises the identification information of task type and described task execution device;
According to described task scheduling request, according to issuing check logic, at least one subtask is verified, obtain the subtask set by verification, the attribute of wherein said subtask comprises: subtask identifier, the identifier of subtask type and affiliated task, and the attribute of task comprises: task identifier, task type, the subtask type of subtask is identical with the task type of the task belonging to described subtask;
From the set of described subtask, chooser task is issued to described task execution device;
Receive the execution result that described task execution device returns, according to the attribute of the subtask issued described in execution result amendment.
In the technical scheme of the distributed task dispatching device described in the present embodiment, collect by issuing verification sieve subtask is verified, obtain the subtask set by verification, meticulousr centralized management and scheduling can be carried out to the different task being applicable to different task actuating unit, the global association between different task can be realized, the dependence be applicable between the different task of different task actuating unit can be supported.
All or part of content in the technical scheme that above embodiment provides can be realized by software programming, and its software program is stored in the storage medium that can read, storage medium such as: the hard disk in computing machine, CD or floppy disk.
The foregoing is only preferred embodiment of the present invention, not in order to limit the present invention, within the spirit and principles in the present invention all, any amendment done, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (12)
1. a distributed task dispatching method, is characterized in that, comprising:
Receive the task scheduling request that task execution device sends, wherein said task scheduling request comprises the identification information of task type and described task execution device;
According to described task scheduling request, according to issuing check logic, at least one subtask is verified, obtain the subtask set by verification, the attribute of wherein said subtask comprises: subtask identifier, the identifier of subtask type and affiliated task, and the attribute of task comprises: task identifier, task type, the subtask type of subtask is identical with the task type of the task belonging to described subtask;
From the set of described subtask, chooser task is issued to described task execution device;
Receive the execution result that described task execution device returns, according to the attribute of the subtask issued described in execution result amendment.
2. distributed task dispatching method as claimed in claim 1, it is characterized in that, describedly verification is carried out at least one subtask comprise according to issuing check logic: according to preset order, task verification, subtask verification, actuating unit verification and/or dependence verification are carried out at least one subtask;
Wherein, described task verification comprises, and the attribute of task belonging to subtask, judges whether subtask meets task scheduling request;
The verification of described subtask comprises, and whether the determined property subtask according to subtask meets task scheduling request;
Described actuating unit verification comprises, and according to the log-on message of the task execution device of described transmission task requests, judges whether subtask meets task scheduling request;
Described dependence verification comprises, and according to the dependence between subtask and other subtasks, judges whether subtask meets task scheduling request.
3. the distributed task dispatching method described in claim 1 or 2, it is characterized in that, described from the set of described subtask chooser task be issued to described task execution device and comprise: sorted according to predetermined order rule in the subtask in the set of described subtask, the subtask of ranking the first be issued to described task execution device.
4. distributed task dispatching method as claimed in claim 3, it is characterized in that, describedly according to predetermined order rule, sequence carried out to the subtask in the set of described subtask comprise, to each subtask in the set of described subtask according to number of attempt Bit-reversed, carry out order according to priority and sort, sort according to period type number sequence or sort according to order working time.
5. distributed task dispatching method as claimed in claim 1 or 2, it is characterized in that, the attribute of described task also comprises entry-into-force time and duty cycle, at least one subtask described be by task to be scheduled according to described entry-into-force time and described duty cycle, according to preset task decomposition rule carry out decomposition obtain.
6. distributed task dispatching method as claimed in claim 2, it is characterized in that, the attribute of described subtask also comprises priority number, described task scheduling request also comprises priority limit, and described actuating unit verification also comprises the priority limit whether priority number judging subtask meets described task scheduling request.
7. a distributed task dispatching device, is characterized in that, comprising:
Dispatch request receiving element, for receiving the task scheduling request that task execution device sends, wherein said task scheduling request comprises the identification information of task type and described task execution device;
Issue verification unit, for according to described task scheduling request, according to issuing check logic, at least one subtask is verified, obtain the subtask set by verification, the attribute of wherein said subtask comprises: subtask identifier, the identifier of subtask type and affiliated task, and the attribute of task comprises: task identifier, task type, the subtask type of subtask is identical with the task type of the task belonging to described subtask;
Issue unit, be issued to described task execution device for chooser task from the set of described subtask;
Result feedback unit, for receiving the execution result that described task execution device returns, according to the attribute of the subtask issued described in execution result amendment.
8. distributed task dispatching device as claimed in claim 7, it is characterized in that, describedly verification is carried out at least one subtask comprise according to issuing check logic: according to preset order, task verification, subtask verification, actuating unit verification and/or dependence verification are carried out at least one subtask;
Wherein, described task verification comprises, and the attribute of task belonging to subtask, judges whether subtask meets task scheduling request;
The verification of described subtask comprises, and whether the determined property subtask according to subtask meets task scheduling request;
Described actuating unit verification comprises, and according to the log-on message of the task execution device of described transmission task requests, judges whether subtask meets task scheduling request;
Described dependence verification comprises, and according to the dependence between subtask and other subtasks, judges whether subtask meets task scheduling request.
9. the distributed task dispatching device described in claim 7 or 8, it is characterized in that, described from the set of described subtask chooser task be issued to described task execution device and comprise: sorted according to predetermined order rule in the subtask in the set of described subtask, the subtask of ranking the first be issued to described task execution device.
10. distributed task dispatching device as claimed in claim 9, it is characterized in that, describedly according to predetermined order rule, sequence is carried out to the subtask in the set of described subtask comprise: to each subtask in the set of described subtask according to number of attempt Bit-reversed, carry out order according to priority and sort, sort according to period type number sequence or sort according to order working time.
11. distributed task dispatching devices as claimed in claim 7 or 8, it is characterized in that, the attribute of described task also comprises entry-into-force time and duty cycle, at least one subtask described be by task to be scheduled according to described entry-into-force time and described duty cycle, according to preset task decomposition rule carry out decomposition obtain.
12. distributed task dispatching devices as claimed in claim 8, it is characterized in that, the attribute of described subtask also comprises priority number, described task scheduling request also comprises priority limit, and described actuating unit verification also comprises the priority limit whether priority number judging subtask meets described task scheduling request.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310223395.7A CN104239148B (en) | 2013-06-06 | 2013-06-06 | A kind of distributed task dispatching method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310223395.7A CN104239148B (en) | 2013-06-06 | 2013-06-06 | A kind of distributed task dispatching method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104239148A true CN104239148A (en) | 2014-12-24 |
| CN104239148B CN104239148B (en) | 2018-05-18 |
Family
ID=52227279
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310223395.7A Active CN104239148B (en) | 2013-06-06 | 2013-06-06 | A kind of distributed task dispatching method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104239148B (en) |
Cited By (43)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104615486A (en) * | 2014-12-26 | 2015-05-13 | 北京京东尚科信息技术有限公司 | Multi-task scheduling and executing method, device and system for search promotion platform |
| CN104750549A (en) * | 2015-04-13 | 2015-07-01 | 飞狐信息技术（天津）有限公司 | Computational task processing device, method and system |
| CN105045832A (en) * | 2015-06-30 | 2015-11-11 | 北京奇艺世纪科技有限公司 | Data acquisition method and apparatus |
| CN105045602A (en) * | 2015-08-24 | 2015-11-11 | 北京金山安全软件有限公司 | Method and device for constructing Hadoop application development framework and electronic device |
| CN105335236A (en) * | 2015-12-10 | 2016-02-17 | 厦门市美亚柏科信息股份有限公司 | Distributed evidence obtaining dynamic load balanced scheduling method and device |
| CN105608531A (en) * | 2015-12-18 | 2016-05-25 | 北京奇虎科技有限公司 | Task dispatching method and device |
| CN105991737A (en) * | 2015-02-26 | 2016-10-05 | 阿里巴巴集团控股有限公司 | Distributed task scheduling method and system |
| WO2016184341A1 (en) * | 2015-05-19 | 2016-11-24 | 阿里巴巴集团控股有限公司 | Method and apparatus for processing service |
| CN106201671A (en) * | 2015-04-29 | 2016-12-07 | 北大方正集团有限公司 | Method for scheduling task and device in one cluster deployed environment |
| CN106406983A (en) * | 2015-07-27 | 2017-02-15 | 阿里巴巴集团控股有限公司 | Task scheduling method and device in cluster |
| CN106484524A (en) * | 2015-08-28 | 2017-03-08 | 阿里巴巴集团控股有限公司 | A kind of task processing method and device |
| CN106598705A (en) * | 2015-10-15 | 2017-04-26 | 阿里巴巴集团控股有限公司 | Asynchronous task scheduling method, device and system and electronic equipment |
| CN106777180A (en) * | 2016-12-22 | 2017-05-31 | 北京京东金融科技控股有限公司 | The method of high-performance distributed data conversion, apparatus and system |
| CN106844018A (en) * | 2015-12-07 | 2017-06-13 | 阿里巴巴集团控股有限公司 | A kind of task processing method, apparatus and system |
| CN107015853A (en) * | 2016-10-10 | 2017-08-04 | 阿里巴巴集团控股有限公司 | The implementation method and device of phased mission system |
| CN107016479A (en) * | 2016-01-28 | 2017-08-04 | 五八同城信息技术有限公司 | Task scheduling and managing method, apparatus and system |
| CN107105009A (en) * | 2017-03-22 | 2017-08-29 | 北京荣之联科技股份有限公司 | Job scheduling method and device based on Kubernetes system docking workflow engines |
| CN107168789A (en) * | 2016-03-08 | 2017-09-15 | 阿里巴巴集团控股有限公司 | The serial scheduling method and device of multitask |
| CN107479963A (en) * | 2016-06-08 | 2017-12-15 | 国家计算机网络与信息安全管理中心 | A kind of method for allocating tasks and system |
| CN107784479A (en) * | 2017-02-16 | 2018-03-09 | 平安科技（深圳）有限公司 | A kind of business flow processing method and apparatus |
| CN107832901A (en) * | 2017-07-28 | 2018-03-23 | 平安科技（深圳）有限公司 | Method for allocating tasks, device, terminal device and storage medium |
| CN107861811A (en) * | 2017-10-24 | 2018-03-30 | 平安科技（深圳）有限公司 | Mission bit stream transmission method, device and computer equipment in Workflow system |
| CN107870948A (en) * | 2016-09-28 | 2018-04-03 | 平安科技（深圳）有限公司 | Method for scheduling task and device |
| CN107870807A (en) * | 2016-09-26 | 2018-04-03 | 平安科技（深圳）有限公司 | A kind of cross-platform method for scheduling task and device |
| CN108021431A (en) * | 2016-11-04 | 2018-05-11 | 广东亿迅科技有限公司 | Method and its system based on web data interactive maintenance Hive |
| CN108021430A (en) * | 2016-10-31 | 2018-05-11 | 杭州海康威视数字技术股份有限公司 | A kind of distributed task scheduling processing method and processing device |
| CN108256989A (en) * | 2017-04-06 | 2018-07-06 | 平安科技（深圳）有限公司 | The data display method and system of Reserve Fund system |
| CN108572863A (en) * | 2017-03-13 | 2018-09-25 | 国家新闻出版广电总局广播电视卫星直播管理中心 | Distributed task dispatching system and method |
| CN108733464A (en) * | 2017-04-18 | 2018-11-02 | 华为软件技术有限公司 | A kind of method and device of the scheduling scheme of determining calculating task |
| CN109002353A (en) * | 2018-08-16 | 2018-12-14 | 北京中科梧桐网络科技有限公司 | A kind of JAVA unifies task central dispatching method |
| CN109213586A (en) * | 2018-08-23 | 2019-01-15 | 北京奇虎科技有限公司 | A kind of dispatching method and device of task |
| CN110109756A (en) * | 2019-04-28 | 2019-08-09 | 北京永信至诚科技股份有限公司 | A kind of network target range construction method, system and storage medium |
| CN110489224A (en) * | 2018-05-15 | 2019-11-22 | 北京京东尚科信息技术有限公司 | A kind of method and apparatus of task schedule |
| CN110532077A (en) * | 2019-08-22 | 2019-12-03 | 腾讯科技（深圳）有限公司 | Task processing method, device and storage medium |
| CN110659151A (en) * | 2018-06-28 | 2020-01-07 | 阿里巴巴集团控股有限公司 | Data verification method and device and storage medium |
| CN110764998A (en) * | 2019-09-06 | 2020-02-07 | 平安健康保险股份有限公司 | Data comparison method, device and equipment based on Django framework and storage medium |
| CN112214020A (en) * | 2020-09-23 | 2021-01-12 | 北京特种机械研究所 | Method and device for establishing task framework and processing tasks of AGV (automatic guided vehicle) scheduling system |
| CN112367205A (en) * | 2020-11-12 | 2021-02-12 | 深圳前海微众银行股份有限公司 | Processing method and scheduling system for HTTP scheduling request |
| CN112488579A (en) * | 2020-12-17 | 2021-03-12 | 上海亿保健康管理有限公司 | Task assignment method and device, electronic equipment and storage medium |
| CN112632566A (en) * | 2021-03-05 | 2021-04-09 | 腾讯科技（深圳）有限公司 | Vulnerability scanning method and device, storage medium and electronic equipment |
| CN113050949A (en) * | 2021-03-23 | 2021-06-29 | 上海万向区块链股份公司 | JAVA-based method and system for preventing data confusion after execution of parent-child dependence JOB |
| CN113485801A (en) * | 2021-06-25 | 2021-10-08 | 中国科学技术大学苏州高等研究院 | Real-time DNN scheduling system and method based on neural network similarity modeling |
| CN114003384A (en) * | 2021-11-01 | 2022-02-01 | 北京深演智能科技股份有限公司 | Task management method, device and equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101739292A (en) * | 2009-12-04 | 2010-06-16 | 曙光信息产业(北京)有限公司 | Application characteristic-based isomeric group operation self-adapting dispatching method and system |
| CN103092698A (en) * | 2012-12-24 | 2013-05-08 | 中国科学院深圳先进技术研究院 | System and method of cloud computing application automatic deployment |
| US8832173B2 (en) * | 2009-01-20 | 2014-09-09 | Sap Ag | System and method of multithreaded processing across multiple servers |
-
2013
- 2013-06-06 CN CN201310223395.7A patent/CN104239148B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8832173B2 (en) * | 2009-01-20 | 2014-09-09 | Sap Ag | System and method of multithreaded processing across multiple servers |
| CN101739292A (en) * | 2009-12-04 | 2010-06-16 | 曙光信息产业(北京)有限公司 | Application characteristic-based isomeric group operation self-adapting dispatching method and system |
| CN103092698A (en) * | 2012-12-24 | 2013-05-08 | 中国科学院深圳先进技术研究院 | System and method of cloud computing application automatic deployment |
Cited By (63)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104615486B (en) * | 2014-12-26 | 2019-07-02 | 北京京东尚科信息技术有限公司 | For searching for the multi-task scheduling of Extension Software Platform and executing methods, devices and systems |
| CN104615486A (en) * | 2014-12-26 | 2015-05-13 | 北京京东尚科信息技术有限公司 | Multi-task scheduling and executing method, device and system for search promotion platform |
| CN105991737A (en) * | 2015-02-26 | 2016-10-05 | 阿里巴巴集团控股有限公司 | Distributed task scheduling method and system |
| CN104750549A (en) * | 2015-04-13 | 2015-07-01 | 飞狐信息技术（天津）有限公司 | Computational task processing device, method and system |
| CN106201671A (en) * | 2015-04-29 | 2016-12-07 | 北大方正集团有限公司 | Method for scheduling task and device in one cluster deployed environment |
| CN106201671B (en) * | 2015-04-29 | 2019-10-15 | 北大方正集团有限公司 | Method for scheduling task and device in one cluster deployed environment |
| WO2016184341A1 (en) * | 2015-05-19 | 2016-11-24 | 阿里巴巴集团控股有限公司 | Method and apparatus for processing service |
| CN106169979A (en) * | 2015-05-19 | 2016-11-30 | 阿里巴巴集团控股有限公司 | A kind of method for processing business and equipment |
| CN105045832B (en) * | 2015-06-30 | 2018-08-07 | 北京奇艺世纪科技有限公司 | A kind of collecting method and device |
| CN105045832A (en) * | 2015-06-30 | 2015-11-11 | 北京奇艺世纪科技有限公司 | Data acquisition method and apparatus |
| CN106406983A (en) * | 2015-07-27 | 2017-02-15 | 阿里巴巴集团控股有限公司 | Task scheduling method and device in cluster |
| CN106406983B (en) * | 2015-07-27 | 2021-01-08 | 创新先进技术有限公司 | Task scheduling method and device in cluster |
| CN105045602B (en) * | 2015-08-24 | 2019-07-23 | 北京金山安全软件有限公司 | Method and device for constructing Hadoop application development framework and electronic device |
| CN105045602A (en) * | 2015-08-24 | 2015-11-11 | 北京金山安全软件有限公司 | Method and device for constructing Hadoop application development framework and electronic device |
| CN106484524A (en) * | 2015-08-28 | 2017-03-08 | 阿里巴巴集团控股有限公司 | A kind of task processing method and device |
| CN106598705B (en) * | 2015-10-15 | 2020-08-11 | 菜鸟智能物流控股有限公司 | Asynchronous task scheduling method, device and system and electronic equipment |
| CN106598705A (en) * | 2015-10-15 | 2017-04-26 | 阿里巴巴集团控股有限公司 | Asynchronous task scheduling method, device and system and electronic equipment |
| CN106844018A (en) * | 2015-12-07 | 2017-06-13 | 阿里巴巴集团控股有限公司 | A kind of task processing method, apparatus and system |
| CN105335236A (en) * | 2015-12-10 | 2016-02-17 | 厦门市美亚柏科信息股份有限公司 | Distributed evidence obtaining dynamic load balanced scheduling method and device |
| CN105608531A (en) * | 2015-12-18 | 2016-05-25 | 北京奇虎科技有限公司 | Task dispatching method and device |
| CN107016479A (en) * | 2016-01-28 | 2017-08-04 | 五八同城信息技术有限公司 | Task scheduling and managing method, apparatus and system |
| CN107168789A (en) * | 2016-03-08 | 2017-09-15 | 阿里巴巴集团控股有限公司 | The serial scheduling method and device of multitask |
| CN107479963A (en) * | 2016-06-08 | 2017-12-15 | 国家计算机网络与信息安全管理中心 | A kind of method for allocating tasks and system |
| CN107870807A (en) * | 2016-09-26 | 2018-04-03 | 平安科技（深圳）有限公司 | A kind of cross-platform method for scheduling task and device |
| CN107870948A (en) * | 2016-09-28 | 2018-04-03 | 平安科技（深圳）有限公司 | Method for scheduling task and device |
| CN107015853A (en) * | 2016-10-10 | 2017-08-04 | 阿里巴巴集团控股有限公司 | The implementation method and device of phased mission system |
| CN108021430B (en) * | 2016-10-31 | 2021-11-05 | 杭州海康威视数字技术股份有限公司 | Distributed task processing method and device |
| CN108021430A (en) * | 2016-10-31 | 2018-05-11 | 杭州海康威视数字技术股份有限公司 | A kind of distributed task scheduling processing method and processing device |
| CN108021431A (en) * | 2016-11-04 | 2018-05-11 | 广东亿迅科技有限公司 | Method and its system based on web data interactive maintenance Hive |
| CN106777180A (en) * | 2016-12-22 | 2017-05-31 | 北京京东金融科技控股有限公司 | The method of high-performance distributed data conversion, apparatus and system |
| WO2018149396A1 (en) * | 2017-02-16 | 2018-08-23 | 平安科技（深圳）有限公司 | Service flow processing method and apparatus, readable storage medium and terminal device |
| CN107784479A (en) * | 2017-02-16 | 2018-03-09 | 平安科技（深圳）有限公司 | A kind of business flow processing method and apparatus |
| CN108572863A (en) * | 2017-03-13 | 2018-09-25 | 国家新闻出版广电总局广播电视卫星直播管理中心 | Distributed task dispatching system and method |
| CN108572863B (en) * | 2017-03-13 | 2022-07-12 | 国家新闻出版广电总局广播电视卫星直播管理中心 | Distributed task scheduling system and method |
| CN107105009B (en) * | 2017-03-22 | 2020-03-10 | 北京荣之联科技股份有限公司 | Job scheduling method and device for butting workflow engine based on Kubernetes system |
| CN107105009A (en) * | 2017-03-22 | 2017-08-29 | 北京荣之联科技股份有限公司 | Job scheduling method and device based on Kubernetes system docking workflow engines |
| WO2018184552A1 (en) * | 2017-04-06 | 2018-10-11 | 平安科技（深圳）有限公司 | Data display method, system, and terminal device for reserve system, and storage medium |
| CN108256989A (en) * | 2017-04-06 | 2018-07-06 | 平安科技（深圳）有限公司 | The data display method and system of Reserve Fund system |
| CN108733464A (en) * | 2017-04-18 | 2018-11-02 | 华为软件技术有限公司 | A kind of method and device of the scheduling scheme of determining calculating task |
| CN108733464B (en) * | 2017-04-18 | 2021-09-14 | 华为技术有限公司 | Method and device for determining scheduling scheme of computing task |
| CN107832901B (en) * | 2017-07-28 | 2021-04-23 | 平安科技（深圳）有限公司 | Task allocation method and device, terminal equipment and storage medium |
| CN107832901A (en) * | 2017-07-28 | 2018-03-23 | 平安科技（深圳）有限公司 | Method for allocating tasks, device, terminal device and storage medium |
| CN107861811B (en) * | 2017-10-24 | 2020-07-31 | 平安科技（深圳）有限公司 | Task information transmission method and device in workflow system and computer equipment |
| CN107861811A (en) * | 2017-10-24 | 2018-03-30 | 平安科技（深圳）有限公司 | Mission bit stream transmission method, device and computer equipment in Workflow system |
| CN110489224A (en) * | 2018-05-15 | 2019-11-22 | 北京京东尚科信息技术有限公司 | A kind of method and apparatus of task schedule |
| CN110659151B (en) * | 2018-06-28 | 2023-05-02 | 阿里巴巴集团控股有限公司 | Data verification method and device and storage medium |
| US11537304B2 (en) | 2018-06-28 | 2022-12-27 | Alibaba Group Holding Limited | Data verification method and apparatus, and storage medium |
| CN110659151A (en) * | 2018-06-28 | 2020-01-07 | 阿里巴巴集团控股有限公司 | Data verification method and device and storage medium |
| CN109002353A (en) * | 2018-08-16 | 2018-12-14 | 北京中科梧桐网络科技有限公司 | A kind of JAVA unifies task central dispatching method |
| CN109213586A (en) * | 2018-08-23 | 2019-01-15 | 北京奇虎科技有限公司 | A kind of dispatching method and device of task |
| CN110109756A (en) * | 2019-04-28 | 2019-08-09 | 北京永信至诚科技股份有限公司 | A kind of network target range construction method, system and storage medium |
| CN110532077B (en) * | 2019-08-22 | 2021-12-07 | 腾讯科技（深圳）有限公司 | Task processing method and device and storage medium |
| CN110532077A (en) * | 2019-08-22 | 2019-12-03 | 腾讯科技（深圳）有限公司 | Task processing method, device and storage medium |
| CN110764998A (en) * | 2019-09-06 | 2020-02-07 | 平安健康保险股份有限公司 | Data comparison method, device and equipment based on Django framework and storage medium |
| CN110764998B (en) * | 2019-09-06 | 2024-04-02 | 平安健康保险股份有限公司 | Data comparison method, device, equipment and storage medium based on Django framework |
| CN112214020A (en) * | 2020-09-23 | 2021-01-12 | 北京特种机械研究所 | Method and device for establishing task framework and processing tasks of AGV (automatic guided vehicle) scheduling system |
| CN112367205A (en) * | 2020-11-12 | 2021-02-12 | 深圳前海微众银行股份有限公司 | Processing method and scheduling system for HTTP scheduling request |
| CN112367205B (en) * | 2020-11-12 | 2023-04-18 | 深圳前海微众银行股份有限公司 | Processing method and scheduling system for HTTP scheduling request |
| CN112488579A (en) * | 2020-12-17 | 2021-03-12 | 上海亿保健康管理有限公司 | Task assignment method and device, electronic equipment and storage medium |
| CN112632566A (en) * | 2021-03-05 | 2021-04-09 | 腾讯科技（深圳）有限公司 | Vulnerability scanning method and device, storage medium and electronic equipment |
| CN113050949A (en) * | 2021-03-23 | 2021-06-29 | 上海万向区块链股份公司 | JAVA-based method and system for preventing data confusion after execution of parent-child dependence JOB |
| CN113485801A (en) * | 2021-06-25 | 2021-10-08 | 中国科学技术大学苏州高等研究院 | Real-time DNN scheduling system and method based on neural network similarity modeling |
| CN114003384A (en) * | 2021-11-01 | 2022-02-01 | 北京深演智能科技股份有限公司 | Task management method, device and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104239148B (en) | 2018-05-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104239148A (en) | Distributed task scheduling method and device | |
| Dong et al. | Greedy scheduling of tasks with time constraints for energy-efficient cloud-computing data centers | |
| US9916183B2 (en) | Scheduling mapreduce jobs in a cluster of dynamically available servers | |
| US8112659B2 (en) | Reducing recovery time for business organizations in case of disasters | |
| CN102937918B (en) | A kind of HDFS runtime data block balance method | |
| US20230342191A1 (en) | Task Scheduling Method and System | |
| KR20150030332A (en) | Distributed and parallel processing system on data and method of operating the same | |
| US10154091B1 (en) | Deploying infrastructure units according to resource hosting constraints | |
| CN102880510A (en) | Parallel programming method oriented to data intensive application based on multiple data architecture centers | |
| US10102490B2 (en) | Optimization of workload placement | |
| Maeno et al. | Evolution of the ATLAS PanDA production and distributed analysis system | |
| dos Anjos et al. | Smart: An application framework for real time big data analysis on heterogeneous cloud environments | |
| US10944814B1 (en) | Independent resource scheduling for distributed data processing programs | |
| Wang et al. | Improving utilization through dynamic VM resource allocation in hybrid cloud environment | |
| Vasiliu et al. | A hybrid scheduler for many task computing in big data systems | |
| Suresh et al. | User demand aware grid scheduling model with hierarchical load balancing | |
| Lin et al. | Analyzing job completion reliability and job energy consumption for a heterogeneous MapReduce cluster under different intermediate-data replication policies | |
| Kumaresan et al. | Aegeus: An online partition skew mitigation algorithm for mapreduce | |
| Sanad et al. | Survey: Utilizing excess cloud resources to reduce the cost and time of distributed processing | |
| Khan et al. | Efficient virtual machine scheduling in cloud computing | |
| Pulamolu et al. | Intra-Tenant resource sharing in yarn based on weighted arithmetic mean | |
| Xie et al. | A resource scheduling algorithm based on trust degree in cloud computing | |
| Ghorpade et al. | A Review on Big Data Processing using Green Hadoop | |
| US20240202051A1 (en) | Routing for data grid in a computing cluster | |
| Hemamalini et al. | State of the art: task scheduling algorithms in a heterogeneous grid computing environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| EXSB | Decision made by sipo to initiate substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20190805 Address after: 518000 Nanshan District science and technology zone, Guangdong, Zhejiang Province, science and technology in the Tencent Building on the 1st floor of the 35 layer Co-patentee after: Tencent cloud computing (Beijing) limited liability company Patentee after: Tencent Technology (Shenzhen) Co., Ltd. Address before: Shenzhen Futian District City, Guangdong province 518044 Zhenxing Road, SEG Science Park 2 East Room 403 Patentee before: Tencent Technology (Shenzhen) Co., Ltd. |
|
| TR01 | Transfer of patent right |