CN110908780B - Task combing method, device, equipment and storage medium of dispatching platform - Google Patents

Task combing method, device, equipment and storage medium of dispatching platform Download PDF

Info

Publication number
CN110908780B
CN110908780B CN201910967574.9A CN201910967574A CN110908780B CN 110908780 B CN110908780 B CN 110908780B CN 201910967574 A CN201910967574 A CN 201910967574A CN 110908780 B CN110908780 B CN 110908780B
Authority
CN
China
Prior art keywords
task
combed
target
link
link level
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910967574.9A
Other languages
Chinese (zh)
Other versions
CN110908780A (en
Inventor
任世民
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Property and Casualty Insurance Company of China Ltd
Original Assignee
Ping An Property and Casualty Insurance Company of China Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Property and Casualty Insurance Company of China Ltd filed Critical Ping An Property and Casualty Insurance Company of China Ltd
Priority to CN201910967574.9A priority Critical patent/CN110908780B/en
Publication of CN110908780A publication Critical patent/CN110908780A/en
Application granted granted Critical
Publication of CN110908780B publication Critical patent/CN110908780B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention relates to the field of research and development management, and discloses a task combing method, a device, equipment and a storage medium of a dispatching platform. The method comprises the following steps: task information of all tasks of a scheduling platform is obtained; extracting tasks to be combed according to the acquired task information to obtain a queue to be combed; randomly selecting a task to be combed from the queue to be combed as a target task to be combed and setting a link level; setting a link level of a target dependent task based on the link level of the target task to be combed; deleting the task to be combed with the link level from the queue to be combed; setting link levels of other tasks to be combed until no task to be combed exists in the queue to be combed; and outputting the link level of each task to be combed to a preset task relation table.

Description

Task combing method, device, equipment and storage medium of dispatching platform
Technical Field
The present invention relates to the field of development aids, and in particular, to a task grooming method, device, apparatus, and storage medium for a dispatch platform.
Background
A scheduling platform is a platform that provides an asynchronous task scheduling solution. The method mainly supports the relation control and the dispatch execution of asynchronous tasks in data processing such as big data platform projects, data mining projects, financial settlement, security report settlement and the like.
Task inquiry on the existing dispatching platform can only be repeated inquiry for each task in the whole task combing, along with the proliferation of development tasks, developers and maintainers can carry out the combing of front-end and rear-end relations of online tasks, key node tasks, data paths and the like, a great deal of manpower is required, and repeated combing or missing of the tasks can occur due to the participation of multiple persons. With personnel change, missing of the handover document may cause that the scheduling task cannot find the responsible person, and no person performs daily maintenance work, so that hidden danger is left. In the task combing process of the scheduling platform, the task combing efficiency is low, and the overall development efficiency of the project is difficult to improve.
Disclosure of Invention
The invention provides a task combing method, a task combing device, task combing equipment and a storage medium of a dispatching platform, which are used for combing a dependent link of a single task and a dispatching link of an overall task through a task combing script and outputting hierarchical structure data of the task, so that the task combing efficiency is improved, and accurate data support is provided for project development.
A first aspect of an embodiment of the present invention provides a task combing method of a scheduling platform, including: task information of all tasks of a scheduling platform is obtained, wherein the task information comprises task names, task states, front task information and rear task information; extracting tasks to be combed according to the acquired task information to obtain a queue to be combed; randomly selecting one task to be combed from the queue to be combed as a target task to be combed, setting a link level of the target task to be combed as K, wherein K is any integer; setting a link level of a target dependent task based on a link level K of the target task to be combed according to task information of the task to be combed in the queue to be combed, wherein the target dependent task is the task to be combed with a dependency relationship with the target task to be combed; deleting the task to be combed with the link level from the queue to be combed; selecting other target to-be-combed tasks randomly, setting link levels of the other target to-be-combed tasks, combing the to-be-combed tasks with dependency relation with the other target to-be-combed tasks, setting the link levels, deleting the to-be-combed tasks with the link levels from the to-be-combed queue until no to-be-combed task exists in the to-be-combed queue; and outputting the link level of each task to be combed and storing the link level into a preset task relation table.
Optionally, in a first implementation manner of the first aspect of the embodiment of the present invention, the setting, based on the link level K of the target task to be combed, a link level of a target dependent task according to task information of the task to be combed in the queue to be combed, where the target dependent task is a task to be combed having a dependency relationship with the target task to be combed includes: setting a link level of a front link task in a target dependent task according to task information of the task to be combed in the queue to be combed and a link level K of the target task to be combed, wherein the target dependent task is the task to be combed with a dependency relationship with the target task to be combed, and the front link task is the task to be combed with the execution sequence before the target task to be combed or with the same execution sequence as the target task to be combed; setting a link level of a post link task in the target dependent task according to task information of the task to be combed in the queue to be combed and the link level K of the target task to be combed, wherein the post link task is a task to be combed with the execution sequence after the target task to be combed or with the same execution sequence as the target task to be combed.
Optionally, in a second implementation manner of the first aspect of the embodiment of the present invention, the setting a link level of a front link task in the target dependent task according to task information of the task to be combed in the queue to be combed and the link level K of the target task to be combed includes: searching the preposed tasks of the target task to be combed according to the preposed task information of the target task to be combed to obtain a first preposed task, wherein the number of the preposed tasks is at least one; setting the link level of the first pre-task as the link level of the target task to be combed minus 1, and obtaining the link level as K-1; searching the preposed task of the first preposed task according to the preposed task information of the first preposed task to obtain a second preposed task; setting the link level of the second pre-task as the link level of the first pre-task minus 1, and obtaining a link level of K-2; circularly searching the P-th pre-task and setting a link level as K-P until the pre-task cannot be searched; searching the post-tasks of the P pre-tasks as first forward tasks according to the post-task information of the P pre-tasks, wherein the number of the post-tasks is at least one; setting the link level of the first forward task as the link level of the P-th pre-task plus 1 to obtain a link level K-P+1; and circularly searching the Q forward task and setting the link level as K-P+Q until the link level K-P+Q of the Q forward task is equal to K.
Optionally, in a third implementation manner of the first aspect of the embodiment of the present invention, the setting a link level of a post link task in the target dependent task according to task information of the task to be combed in the queue to be combed and a link level K of the target task to be combed includes: searching the post-tasks of the target task to be combed according to the post-task information of the target task to be combed to obtain a first post-task, wherein the number of the post-tasks is at least one; setting the link level of the first post task as the link level of the target task to be combed plus 1 to obtain a link level of K+1; searching for the post-task of the first post-task according to the post-task information of the first post-task to obtain a second post-task; setting the link level of the second post task as the link level of the first post task plus 1 to obtain a link level of K+2; circularly searching the L-th post task and setting the link level as K+L until the post task cannot be searched; searching the preposed tasks of the L-th posttask according to the preposed task information of the L-th posttask to obtain a first backward task, wherein the number of the preposed tasks is at least one; setting the link level of the first backward task as the link level of the L-th backward task minus 1 to obtain a link level K+L-1; and circularly searching the S-th backward task and setting the link level as K+L-S until the link level K+L-S of the S-th backward task is equal to K.
Optionally, in a fourth implementation manner of the first aspect of the embodiment of the present invention, extracting a task to be combed according to the obtained task information to obtain a queue to be combed includes: judging whether the target task is stored in a to-be-combed queue according to the task state of the target task, wherein the target task is a task of any one of the dispatching platforms, the task state comprises an enabling state and a stopping state, and the to-be-combed queue is a preset list; if the task state of the target task is determined to be the starting state, determining the target task to be a task to be combed, and storing the task name of the target task into the queue to be combed in a list element mode; and if the task state of the target task is determined to be the stop state, not storing the task name of the target task into the queue to be combed.
Optionally, in a fifth implementation manner of the first aspect of the embodiment of the present invention, the selecting, at random, one task to be combed from the queue to be combed as a target task to be combed, setting a link level of the target task to be combed to be K, where K is any integer, includes: extracting an element in the queue to be combed through a random sampling function to obtain a task name of a target task to be combed; and setting a link level of the target task to be combed to K according to the task name of the target task to be combed, wherein K is any integer, and the link level is a preset dynamic statement variable.
Optionally, in a sixth implementation manner of the first aspect of the embodiment of the present invention, the outputting the link level of each task to be combed and storing the link level in a preset task relationship table includes: acquiring a link level, a task name and a front task of the task to be combed; and storing the task names, the front tasks and the link levels of the tasks to be combed into a preset task relation table in a record form according to the order from small link levels to large link levels, wherein each task to be combed has at least one record in the preset task relation table by taking the task names and the front tasks as main keys, and one record only comprises one front task.
A second aspect of an embodiment of the present invention provides a task grooming device for a dispatch platform, including: the task information comprises task names, task states, front task information and rear task information; the extraction unit is used for extracting tasks to be combed according to the acquired task information to obtain a queue to be combed; the first setting unit is used for randomly selecting one task to be combed from the queue to be combed as a target task to be combed, and setting the link level of the target task to be combed as K, wherein K is any integer; the second setting unit is used for setting a link level of a target dependent task based on a link level K of the target task to be combed according to task information of the task to be combed in the queue, wherein the target dependent task is the task to be combed with a dependency relationship with the target task to be combed; the deleting unit is used for deleting the task to be combed, which is already provided with the link level, from the queue to be combed; the third setting unit is used for randomly selecting other target tasks to be combed and setting link levels of the other target tasks to be combed, combing the tasks to be combed with dependency relation with the other target tasks to be combed and setting the link levels, and deleting the tasks to be combed with the link levels from the queue to be combed until no task to be combed exists in the queue to be combed; and the output unit outputs the link level of each task to be combed and stores the link level into a preset task relation table.
Optionally, in a first implementation manner of the second aspect of the embodiment of the present invention, the second setting unit specifically includes: the first setting module is used for setting a link level of a front link task in a target dependent task according to task information of the task to be combed in the queue to be combed and a link level K of the target task to be combed, wherein the front link task is a task to be combed, the execution sequence of which is before the target task to be combed, or the execution sequence of which is the same as that of the target task to be combed; the second setting module is configured to set a link level of a post-link task in the target dependent task according to task information of the task to be combed in the queue to be combed and the link level K of the target task to be combed, where the post-link task is a task to be combed whose execution order is after the target task to be combed or whose execution order is the same as that of the target task to be combed.
Optionally, in a second implementation manner of the second aspect of the embodiment of the present invention, the first setting module is specifically configured to: searching the preposed tasks of the target task to be combed according to the preposed task information of the target task to be combed to obtain a first preposed task, wherein the number of the preposed tasks is at least one. Setting the link level of the first pre-task as the link level of the target task to be combed minus 1, and obtaining the link level as K-1. And searching the preposed task of the first preposed task according to the preposed task information of the first preposed task to obtain a second preposed task. Setting the link level of the second pre-task as the link level of the first pre-task minus 1, and obtaining the link level as K-2. And circularly searching the P-th pre-task and setting the link level as K-P until the pre-task cannot be searched. Searching the post-tasks of the P pre-tasks as first forward tasks according to the post-task information of the P pre-tasks, wherein the number of the post-tasks is at least one. Setting the link level of the first forward task as the link level of the P-th pre-task plus 1 to obtain a link level K-P+1. And circularly searching the Q forward task and setting the link level as K-P+Q until the link level K-P+Q of the Q forward task is equal to K.
Optionally, in a third implementation manner of the second aspect of the embodiment of the present invention, the second setting module is specifically configured to: searching the post-tasks of the target task to be combed according to the post-task information of the target task to be combed to obtain a first post-task, wherein the number of the post-tasks is at least one; setting the link level of the first post task as the link level of the target task to be combed plus 1 to obtain a link level of K+1; searching for the post-task of the first post-task according to the post-task information of the first post-task to obtain a second post-task; setting the link level of the second post task as the link level of the first post task plus 1 to obtain a link level of K+2; circularly searching the L-th post task and setting the link level as K+L until the post task cannot be searched; searching the preposed tasks of the L-th posttask according to the preposed task information of the L-th posttask to obtain a first backward task, wherein the number of the preposed tasks is at least one; setting the link level of the first backward task as the link level of the L-th backward task minus 1 to obtain a link level K+L-1; and circularly searching the S-th backward task and setting the link level as K+L-S until the link level K+L-S of the S-th backward task is equal to K.
Optionally, in a fourth implementation manner of the second aspect of the embodiment of the present invention, the extracting unit is specifically configured to: judging whether the target task is stored in a to-be-combed queue according to the task state of the target task, wherein the target task is a task of any one of the dispatching platforms, the task state comprises an enabling state and a stopping state, and the to-be-combed queue is a preset list; if the task state of the target task is determined to be the starting state, determining the target task to be a task to be combed, and storing the task name of the target task into the queue to be combed in a list element mode; and if the task state of the target task is determined to be the stop state, not storing the task name of the target task into the queue to be combed.
Optionally, in a fifth implementation manner of the second aspect of the embodiment of the present invention, the first setting unit is specifically configured to: extracting an element in the queue to be combed through a random sampling function to obtain a task name of a target task to be combed; and setting a link level of the target task to be combed to K according to the task name of the target task to be combed, wherein K is any integer, and the link level is a preset dynamic statement variable.
Optionally, in a sixth implementation manner of the second aspect of the embodiment of the present invention, the output unit is specifically configured to: acquiring a link level, a task name and a front task of the task to be combed; and storing the task names, the front tasks and the link levels of the tasks to be combed into a preset task relation table in a record form according to the order from small link levels to large link levels, wherein each task to be combed has at least one record in the preset task relation table by taking the task names and the front tasks as main keys, and one record only comprises one front task.
A third aspect of the embodiment of the present invention provides a task grooming device for a dispatch platform, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the task grooming method for a dispatch platform according to any one of the foregoing embodiments is implemented when the processor executes the computer program.
A fourth aspect of the embodiments of the present invention provides a computer readable storage medium, including instructions that when executed on a computer cause the computer to perform the steps of the task orchestration method of the scheduling platform according to any one of the embodiments described above.
In the technical scheme provided by the embodiment of the invention, task information of all tasks of a scheduling platform is acquired, wherein the task information comprises task names, task states, front task information and rear task information; extracting tasks to be combed according to the acquired task information to obtain a queue to be combed; randomly selecting one task to be combed from the queue to be combed as a target task to be combed, setting a link level of the target task to be combed as K, wherein K is any integer; setting a link level of a target dependent task based on a link level K of the target task to be combed according to task information of the task to be combed in the queue to be combed, wherein the target dependent task is the task to be combed with a dependency relationship with the target task to be combed; deleting the task to be combed with the link level from the queue to be combed; selecting other target to-be-combed tasks randomly, setting link levels of the other target to-be-combed tasks, combing the to-be-combed tasks with dependency relation with the other target to-be-combed tasks, setting the link levels, deleting the to-be-combed tasks with the link levels from the to-be-combed queue until no to-be-combed task exists in the to-be-combed queue; and outputting the link level of each task to be combed and storing the link level into a preset task relation table. According to the embodiment of the invention, the task combing script is used for combing the dependent links of the single task and the scheduling links of the whole task and outputting the hierarchical structure data of the task, so that the task combing efficiency is improved, and accurate data support is provided for project development.
Drawings
FIG. 1 is a schematic diagram of an embodiment of a task grooming method for a dispatch platform according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of another embodiment of a task grooming method for a dispatch platform according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an embodiment of a task orchestration device of a dispatch platform according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of another embodiment of a task orchestration device of a dispatch platform according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of an embodiment of a task orchestration device of a dispatch platform according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a task combing method, device, equipment and storage medium of a dispatching platform, which are used for combing a dependent link of a single task and a dispatching link of an overall task through a task combing script and outputting hierarchical structure data of the task, so that the task combing efficiency is improved, and accurate data support is provided for project development.
In order to enable those skilled in the art to better understand the present invention, embodiments of the present invention will be described below with reference to the accompanying drawings.
The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.
Referring to fig. 1, a flowchart of a task combing method of a scheduling platform according to an embodiment of the present invention specifically includes:
101. task information of all tasks of the scheduling platform is obtained, wherein the task information comprises task names, task states, front task information and rear task information.
The server acquires task information of all tasks of the scheduling platform, wherein the task information comprises task names, task states, front task information and rear task information. Specifically, the server acquires task information of all tasks of the scheduling platform from task operation records of the scheduling platform within a preset time period. For example, a task a of a scheduling platform, the task name is "financial settlement service task a", the task state is enabled, the pre-task information of the task a includes a pre-task B of the task a, and the pre-task C, and the post-task information of the task a includes a post-task D of the task a.
The task states include an enabled state and a stopped state. When the task state of the task is an enabling state, the task is a task which is normally enabled by the project and needs to be combed; when the task state of the task is a stop state, the task is a task which does not need to be combed.
102. And extracting tasks to be combed according to the acquired task information to obtain a queue to be combed.
And the server extracts tasks to be combed according to the acquired task information to obtain a queue to be combed. Specifically, the server judges whether to store the target task into a queue to be combed according to the task state of the target task, wherein the target task is any one scheduling platform task, and the task state comprises an enabling state and a stopping state; if the server determines that the task state of the target task is an enabling state, storing the task name of the target task into a queue to be combed in a list element mode; if the server determines that the task state of the target task is the stop state, the task name of the target task is not stored in the queue to be combed.
It should be noted that, the queue to be combed is a preset Python list, and when the task is in an enabled state, the server stores the task name of the task into the queue to be combed in the form of an element list. For example, if the server determines that the task states of the task a, the task B, the task C, and the task D are the start states, the queue to be combed is [ ' task a ', ' task B ', ' task C ', ' task D ].
103. Randomly selecting a task to be combed from the queue to be combed as a target task to be combed, and setting the link level of the target task to be combed as K, wherein K is any integer.
The server randomly selects a task to be combed from the queue to be combed as a target task to be combed, and sets the link level of the target task to be combed as K, wherein K is any integer. Specifically, the server extracts an element in the queue to be combed through a random sampling function, and obtains the task name of the target task to be combed. And setting a link level of the target task to be combed as K according to the task name of the target task to be combed, wherein the link level is a preset dynamic statement variable.
For example, when the server extracts the task a through the random sampling function as the target task to be combed, the server sets the link level of the task a to 0, the link level is a preset dynamic declaration variable, the variable name of the preset dynamic declaration variable includes the task name of the task a, and the variable value is link level 0.
104. Setting a link level of a target dependent task based on a link level K of the target task to be combed according to task information of the task to be combed in the queue to be combed, wherein the target dependent task is the task to be combed with a dependent relation with the target task to be combed.
It should be noted that, according to the task information of the task to be combed, the server searches the task to be combed whose execution sequence is not after the target task to be combed in the task link by circularly searching the task to be combed whose execution sequence is not before the target task to be combed in the task link backward, so as to combe out a task link including the target task to be combed, link levels of the tasks, and determine execution sequences of the tasks according to the link levels, thereby being beneficial to improving execution efficiency of the task link.
105. And deleting the task to be combed, which is provided with the link level, from the queue to be combed.
The server deletes the task to be combed, which has been set with the link hierarchy, from the queue to be combed. Specifically, the server deletes the task to be combed with the link layer from the queue to be combed in a mode of removing the preset elements from the list, so that other tasks to be combed are conveniently combed, and the combing efficiency is improved.
For example, the queue to be combed is [ 'task a', 'task B', 'task C', 'task D' ], after the server sets the link level of task a to K, the server deletes task a from the queue to be combed, and the queue to be combed is changed to [ 'task B', 'task C', 'task D' ].
106. And randomly selecting other target tasks to be combed, setting link levels of the other target tasks to be combed, combing the tasks to be combed which have a dependency relationship with the other target tasks to be combed, setting the link levels, deleting the tasks to be combed with the link levels from the queue to be combed until no task to be combed exists in the queue to be combed.
The server randomly selects other target tasks to be combed and sets link levels of the other target tasks to be combed, the tasks to be combed which have a dependency relationship with the other target tasks to be combed are combed, the link levels are set, the tasks to be combed with the link levels are deleted from the queue to be combed, and until no task to be combed exists in the queue to be combed, so that the link levels of all the tasks to be combed are ensured to be determined, and the task combing efficiency is improved.
107. And outputting the link level of each task to be combed and storing the link level into a preset task relation table.
The server outputs the link level of each task to be combed and stores the link level in a preset database. Specifically, the server stores the dependency relationship of the task of the scheduling platform by storing the link levels, the task names and the pre-tasks of all the tasks to be combed into a preset task relationship table. Specifically, a server acquires a link level, a task name and a front-end task of a task to be combed; the server stores the task names, the front tasks and the link levels of the tasks to be combed into a preset task relation table in a record form according to the order of the link levels from small to large, the preset task relation table takes the task names and the front tasks as main keys, and each task to be combed has at least one record in the preset task relation table, wherein one record only comprises one front task.
Task name Front-end tasks Link hierarchy
A B -1
A BB -1
A BBB -1
B C -2
BB CC -2
BB CCC -2
BBB D -2
BBB DD -2
TABLE 1
The link level storage form of the task to be combed is shown in table 1, the link level of the task a is-1, the task a has three records in a preset task relation table, each record comprises a front-end task of the task a, the front-end tasks of the task a are task B, task BB and task BBB, the link levels of the front-end tasks of the task a are all-2, wherein the task B has a front-end task C, and the task BB has two front-end tasks: task CC and task CCC, task BBB has two pre-tasks: task D and task DD.
According to the embodiment of the invention, the task combing script is used for combing the dependent links of the single task and the scheduling links of the whole task and outputting the hierarchical structure data of the task, so that the task combing efficiency is improved, and accurate data support is provided for project development.
Referring to fig. 2, another embodiment of a task grooming method of a dispatch platform according to an embodiment of the present invention includes:
201. task information of all tasks of the scheduling platform is obtained, wherein the task information comprises task names, task states, front task information and rear task information.
The server acquires task information of all tasks of the scheduling platform, wherein the task information comprises task names, task states, front task information and rear task information. Specifically, the server acquires task information of all tasks of the scheduling platform from task operation records of the scheduling platform within a preset time period. For example, a task a of a scheduling platform, the task name is "financial settlement service task a", the task state is enabled, the pre-task information of the task a includes a pre-task B of the task a, and the pre-task C, and the post-task information of the task a includes a post-task D of the task a.
The task states include an enabled state and a stopped state. When the task state of the task is an enabling state, the task is a task which is normally enabled by the project and needs to be combed; when the task state of the task is a stop state, the task is a task which does not need to be combed.
202. And extracting tasks to be combed according to the acquired task information to obtain a queue to be combed.
And the server extracts tasks to be combed according to the acquired task information to obtain a queue to be combed. Specifically, the server judges whether to store the target task into a queue to be combed according to the task state of the target task, wherein the target task is any one scheduling platform task, and the task state comprises an enabling state and a stopping state; if the server determines that the task state of the target task is an enabling state, storing the task name of the target task into a queue to be combed in a list element mode; if the server determines that the task state of the target task is the stop state, the task name of the target task is not stored in the queue to be combed.
It should be noted that, the queue to be combed is a preset Python list, and when the task is in an enabled state, the server stores the task name of the task into the queue to be combed in the form of an element list. For example, if the server determines that the task states of the task a, the task B, the task C, and the task D are the start states, the queue to be combed is [ ' task a ', ' task B ', ' task C ', ' task D ].
203. Randomly selecting a task to be combed from the queue to be combed as a target task to be combed, and setting the link level of the target task to be combed as K, wherein K is any integer.
The server randomly selects a task to be combed from the queue to be combed as a target task to be combed, and sets the link level of the target task to be combed as K, wherein K is any integer. Specifically, the server extracts an element in the queue to be combed through a random sampling function, and obtains the task name of the target task to be combed. And setting a link level of the target task to be combed as K according to the task name of the target task to be combed, wherein the link level is a preset dynamic statement variable.
For example, when the server extracts the task a through the random sampling function as the target task to be combed, the server sets the link level of the task a to 0, the link level is a preset dynamic declaration variable, the variable name of the preset dynamic declaration variable includes the task name of the task a, and the variable value is link level 0.
204. According to task information of tasks to be combed in a queue to be combed and a link level K of a target task to be combed, setting a link level of a front link task in a target dependent task, wherein the target dependent task is the task to be combed which has a dependency relationship with the target task to be combed, and the front link task is the task to be combed which is executed in front of the target task to be combed or is executed in the same order as the target task to be combed.
The server sets a link level of a front link task in the target dependent task according to task information of the task to be combed in the queue to be combed and a link level K of the target task to be combed, wherein the front link task is the task to be combed with the execution sequence before the target task to be combed or with the execution sequence identical to that of the target task to be combed. Specifically, the server searches for the preposed tasks of the target task to be combed according to the preposed task information of the target task to be combed to obtain a first preposed task, wherein the number of the preposed tasks is at least one; the server sets the link level of the first pre-task as the link level of the target task to be combed minus 1, and the link level is K-1; the server searches the preposed task of the first preposed task according to the preposed task information of the first preposed task to obtain a second preposed task; the server sets the link level of the second pre-task as the link level of the first pre-task minus 1, and the link level is K-2; the server circularly searches the P-th pre-task and sets the link level as K-P until the pre-task cannot be searched; the server searches the post-tasks of the P pre-tasks according to the post-task information of the P pre-tasks, and takes the post-tasks as a first forward task, wherein the number of the post-tasks is at least one; the server sets the link level of the first forward task as the link level of the P-th pre-task plus 1 to obtain a link level K-P+1; the server circularly searches the Q forward task and sets the link level as K-P+Q until the link level K-P+Q of the Q forward task is equal to K.
For example, the link hierarchy of task a is 0, according to the pre-task information of task a, the server determines that the pre-task of task a is task B and task BB, task B and task BB are first pre-tasks, and the server sets the link hierarchy of the first pre-tasks to-1; according to the preposed task information of the first preposed task, the server determines that a task B has a preposed task C, a task BB has a preposed task CC and a preposed task CCC, and the task C, the task CC and the task CCC are second preposed tasks, and the server sets the link level of the second preposed task to be-2; according to the preposed task information of the second preposed task, the server determines that only the task C has a preposed task E, wherein the task E is a third preposed task, and the server sets a link level of the third preposed task to be-3; after the server determines that the task E does not have a front task according to the front task information of the task E, the server determines that the rear task of the task E comprises a task C and a task D according to the rear task information of the task E, the task C and the task D are first front tasks, the link level of the first front tasks is the link level of the task E plus 1, the link level of the first front tasks is obtained to be-2, the server sets the link level of the second front tasks to be-1 according to the rear task information, and sets the link level of the third front tasks to be 0, when the link level of the third front tasks is the same as the link level of the task A, the third front tasks comprise the task A, and the server finishes combing the front tasks in the task links.
When the selected target task to be combed does not have a front-end task, the server determines that the front-end link task in the task link is the target task to be combed, the value of the link level of the target task to be combed is the minimum in the task link, and the execution sequence of the target task to be combed is the first bit of the task link; when the first pre-task does not have a pre-task, the server determines that the pre-link task in the link task is the first pre-task and the first forward task, wherein the first forward task comprises a target task to be combed, the value of the link level of the first pre-task is minimum in the task link, and the execution sequence of the first pre-task is the first bit of the task link.
205. According to task information of tasks to be combed in the queue to be combed and a link level K of the target tasks to be combed, setting a link level of a post-link task in the target dependent tasks, wherein the post-link task is a task to be combed, the execution sequence of which is behind the target tasks to be combed, or the execution sequence of which is the same as that of the target tasks to be combed.
The server sets a link level of a post link task in the target dependent task according to task information of the task to be combed in the queue to be combed and a link level K of the target task to be combed, wherein the post link task is a task to be combed with the execution sequence after the target task to be combed or with the execution sequence identical to that of the target task to be combed. Specifically, the server searches for the post-tasks of the target task to be combed according to the post-task information of the target task to be combed to obtain a first post-task, wherein the number of the post-tasks is at least one; the server sets the link level of the first post task as the link level of the target task to be combed plus 1 to obtain a link level of K+1; the server searches the post-task of the first post-task according to the post-task information of the first post-task to obtain a second post-task; the server sets the link level of the second post task as the link level of the first post task plus 1 to obtain a link level of K+2; the server circularly searches the L-th post task and sets the link level as K+L until the post task cannot be searched; the server searches the prepositive tasks of the L-th posttask according to the prepositive task information of the L-th posttask to obtain a first backward task, wherein the number of the prepositive tasks is at least one; the server sets the link level of the first backward task as the link level of the L-th backward task minus 1, and a link level K+L-1 is obtained; the server circularly searches the S-th backward task and sets the link level as K+L-S until the link level K+L-S of the S-th backward task is equal to K.
When the selected target task to be combed does not have a post-task, the server determines that the post-link task in the task link is the target task to be combed, the value of the link level of the target task to be combed is the maximum in the task link, and the execution sequence of the target task to be combed is the last bit of the task link; when the first post task does not have the post task, the server determines that the post link task in the link task is the first post task and the first post task, wherein the first post task comprises a target task to be combed, the value of the link level of the first post task is the largest in the task link, and the execution sequence of the first post task is the last bit of the task link.
206. And deleting the task to be combed, which is provided with the link level, from the queue to be combed.
The server deletes the task to be combed, which has been set with the link hierarchy, from the queue to be combed. Specifically, the server deletes the task to be combed with the link layer from the queue to be combed in a mode of removing the preset elements from the list, so that other tasks to be combed are conveniently combed, and the combing efficiency is improved.
For example, the queue to be combed is [ 'task a', 'task B', 'task C', 'task D' ], after the server sets the link level of task a to K, the server deletes task a from the queue to be combed, and the queue to be combed is changed to [ 'task B', 'task C', 'task D' ].
207. And randomly selecting other target tasks to be combed, setting link levels of the other target tasks to be combed, combing the tasks to be combed which have a dependency relationship with the other target tasks to be combed, setting the link levels, deleting the tasks to be combed with the link levels from the queue to be combed until no task to be combed exists in the queue to be combed.
The server randomly selects other target tasks to be combed and sets link levels of the other target tasks to be combed, the tasks to be combed which have a dependency relationship with the other target tasks to be combed are combed, the link levels are set, the tasks to be combed with the link levels are deleted from the queue to be combed, and until no task to be combed exists in the queue to be combed, so that the link levels of all the tasks to be combed are ensured to be determined, and the task combing efficiency is improved.
208. And outputting the link level of each task to be combed and storing the link level into a preset task relation table.
The server outputs the link level of each task to be combed and stores the link level in a preset database. Specifically, the server stores the dependency relationship of the task of the scheduling platform by storing the link levels, the task names and the pre-tasks of all the tasks to be combed into a preset task relationship table. Specifically, a server acquires a link level, a task name and a front-end task of a task to be combed; the server stores the task names, the front tasks and the link levels of the tasks to be combed into a preset task relation table in a record form according to the order of the link levels from small to large, the preset task relation table takes the task names and the front tasks as main keys, and each task to be combed has at least one record in the preset task relation table, wherein one record only comprises one front task.
According to the embodiment of the invention, the task combing script is used for combing the dependent links of the single task and the scheduling links of the whole task and outputting the hierarchical structure data of the task, so that the task combing efficiency is improved, and accurate data support is provided for project development.
The task orchestration method of the scheduling platform in the embodiment of the present invention is described above, and the task orchestration device of the scheduling platform in the embodiment of the present invention is described below, referring to fig. 3, where an embodiment of the task orchestration device of the scheduling platform in the embodiment of the present invention includes:
an obtaining unit 301, configured to obtain task information of all tasks of the scheduling platform, where the task information includes a task name, a task state, front task information, and rear task information;
the extracting unit 302 is configured to extract a task to be combed according to the obtained task information, so as to obtain a queue to be combed;
a first setting unit 303, configured to randomly select one task to be combed from the queue to be combed as a target task to be combed, and set a link level of the target task to be combed to be K, where K is any integer;
the second setting unit 304 is configured to set a link level of a target dependent task based on a link level K of the target task to be combed according to task information of the task to be combed in the queue, where the target dependent task is a task to be combed having a dependency relationship with the target task to be combed;
A deleting unit 305, configured to delete a task to be combed, for which a link hierarchy has been set, from the queue to be combed;
a third setting unit 306, configured to randomly select other target tasks to be combed and set link levels of the other target tasks to be combed, comb the tasks to be combed that have a dependency relationship with the other target tasks to be combed, set link levels, and delete the tasks to be combed that have been set link levels from the queue to be combed until no task to be combed exists in the queue to be combed;
the output unit 307 outputs the link hierarchy of each task to be combed and stores it in a preset task relationship table.
According to the embodiment of the invention, the task combing script is used for combing the dependent links of the single task and the scheduling links of the whole task and outputting the hierarchical structure data of the task, so that the task combing efficiency is improved, and accurate data support is provided for project development.
Referring to fig. 4, an embodiment of a task grooming device of a dispatch platform according to an embodiment of the present invention includes:
an obtaining unit 301, configured to obtain task information of all tasks of the scheduling platform, where the task information includes a task name, a task state, front task information, and rear task information;
The extracting unit 302 is configured to extract a task to be combed according to the obtained task information, so as to obtain a queue to be combed;
a first setting unit 303, configured to randomly select one task to be combed from the queue to be combed as a target task to be combed, and set a link level of the target task to be combed to be K, where K is any integer;
the second setting unit 304 is configured to set a link level of a target dependent task based on a link level K of the target task to be combed according to task information of the task to be combed in the queue, where the target dependent task is a task to be combed having a dependency relationship with the target task to be combed;
a deleting unit 305, configured to delete a task to be combed, for which a link hierarchy has been set, from the queue to be combed;
a third setting unit 306, configured to randomly select other target tasks to be combed and set link levels of the other target tasks to be combed, comb the tasks to be combed that have a dependency relationship with the other target tasks to be combed, set link levels, and delete the tasks to be combed that have been set link levels from the queue to be combed until no task to be combed exists in the queue to be combed;
The output unit 307 outputs the link hierarchy of each task to be combed and stores it in a preset task relationship table.
Optionally, the second setting unit 304 specifically includes:
the first setting module 3041 is configured to set a link level of a front link task in a target dependent task according to task information of a task to be combed in the queue to be combed and a link level K of the target task to be combed, where the front link task is a task to be combed whose execution order is before the target task to be combed or whose execution order is the same as that of the target task to be combed;
the second setting module 3042 is configured to set, according to the task information of the task to be combed in the queue to be combed and the link level K of the target task to be combed, a link level of a post link task in the target dependent task, where the post link task is a task to be combed whose execution order is after the target task to be combed or whose execution order is the same as that of the target task to be combed.
Optionally, the first setting module 3041 is specifically configured to:
searching the preposed tasks of the target task to be combed according to the preposed task information of the target task to be combed to obtain a first preposed task, wherein the number of the preposed tasks is at least one. Setting the link level of the first pre-task as the link level of the target task to be combed minus 1, and obtaining the link level as K-1. And searching the preposed task of the first preposed task according to the preposed task information of the first preposed task to obtain a second preposed task. Setting the link level of the second pre-task as the link level of the first pre-task minus 1, and obtaining the link level as K-2. And circularly searching the P-th pre-task and setting the link level as K-P until the pre-task cannot be searched. Searching the post-tasks of the P pre-tasks as first forward tasks according to the post-task information of the P pre-tasks, wherein the number of the post-tasks is at least one. Setting the link level of the first forward task as the link level of the P-th pre-task plus 1 to obtain a link level K-P+1. And circularly searching the Q forward task and setting the link level as K-P+Q until the link level K-P+Q of the Q forward task is equal to K.
Optionally, the second setting module 3042 is specifically configured to:
searching the post-tasks of the target task to be combed according to the post-task information of the target task to be combed to obtain a first post-task, wherein the number of the post-tasks is at least one; setting the link level of the first post task as the link level of the target task to be combed plus 1 to obtain a link level of K+1; searching for the post-task of the first post-task according to the post-task information of the first post-task to obtain a second post-task; setting the link level of the second post task as the link level of the first post task plus 1 to obtain a link level of K+2; circularly searching the L-th post task and setting the link level as K+L until the post task cannot be searched; searching the preposed tasks of the L-th posttask according to the preposed task information of the L-th posttask to obtain a first backward task, wherein the number of the preposed tasks is at least one; setting the link level of the first backward task as the link level of the L-th backward task minus 1 to obtain a link level K+L-1; and circularly searching the S-th backward task and setting the link level as K+L-S until the link level K+L-S of the S-th backward task is equal to K.
Optionally, the extracting unit 302 is specifically configured to:
judging whether the target task is stored in a to-be-combed queue according to the task state of the target task, wherein the target task is a task of any one of the dispatching platforms, the task state comprises an enabling state and a stopping state, and the to-be-combed queue is a preset list; if the task state of the target task is determined to be the starting state, determining the target task to be a task to be combed, and storing the task name of the target task into the queue to be combed in a list element mode; and if the task state of the target task is determined to be the stop state, not storing the task name of the target task into the queue to be combed.
Optionally, the first setting unit 303 is specifically configured to: extracting an element in the queue to be combed through a random sampling function to obtain a task name of a target task to be combed; and setting a link level of the target task to be combed to K according to the task name of the target task to be combed, wherein K is any integer, and the link level is a preset dynamic statement variable.
Optionally, the output unit 307 is specifically configured to:
Acquiring a link level, a task name and a front task of the task to be combed; and storing the task names, the front tasks and the link levels of the tasks to be combed into a preset task relation table in a record form according to the order from small link levels to large link levels, wherein each task to be combed has at least one record in the preset task relation table by taking the task names and the front tasks as main keys, and one record only comprises one front task.
According to the embodiment of the invention, the task combing script is used for combing the dependent links of the single task and the scheduling links of the whole task and outputting the hierarchical structure data of the task, so that the task combing efficiency is improved, and accurate data support is provided for project development.
The task grooming device of the dispatch platform in the embodiment of the present invention is described in detail above in fig. 3 to 4 from the point of view of modularized functional entities, and the task grooming device of the dispatch platform in the embodiment of the present invention is described in detail below from the point of view of hardware processing.
Fig. 5 is a schematic structural diagram of a task grooming device of a scheduling platform according to an embodiment of the present invention, where a task grooming device 500 of the scheduling platform may have a relatively large difference due to configuration or performance, and may include one or more processors (central processing units, CPU) 501 (e.g., one or more processors) and a memory 509, and one or more storage media 508 (e.g., one or more mass storage devices) storing application programs 507 or data 506. Wherein the memory 509 and storage medium 508 may be transitory or persistent storage. The program stored on the storage medium 508 may include one or more modules (not shown), each of which may include a series of instruction operations in the task orchestration device of the scheduling platform. Still further, the processor 501 may be configured to communicate with the storage medium 508 and execute a series of instruction operations in the storage medium 508 on the task orchestration device 500 of the dispatch platform.
Task orchestration device 500 of the dispatch platform may also include one or more power supplies 502, one or more wired or wireless network interfaces 503, one or more input/output interfaces 504, and/or one or more operating systems 505, such as Windows Serve, mac OS X, unix, linux, freeBSD, and the like. It will be appreciated by those skilled in the art that the task orchestration device structure of the scheduling platform shown in fig. 5 does not constitute a limitation of the task orchestration device of the scheduling platform, and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components. The processor 501 may perform the functions of the acquisition unit 301, the extraction unit 302, the first setting unit 303, the second setting unit 304, the deletion unit 305, and the third setting unit 306, and the output unit 307 in the above-described embodiments.
The following describes the various components of the task grooming device of the dispatching platform in detail in connection with fig. 5:
the processor 501 is a control center of the task orchestration device of the dispatch platform, and may perform processing according to a set task orchestration method. The processor 501 connects the various parts of the task orchestration device of the overall dispatch platform with various interfaces and lines to perform various functions and processes of the task orchestration device of the dispatch platform by running or executing software programs and/or modules stored in the memory 509 and invoking data stored in the memory 509, thereby enabling task orchestration of the dispatch platform. Both the storage medium 508 and the memory 509 are carriers for storing data, the storage medium 508 may refer to an internal memory having a small storage capacity but a high speed, and the memory 509 may be an external memory having a large storage capacity but a low storage speed.
The memory 509 may be used to store software programs and modules, and the processor 501 performs various functional applications and data processing of the task orchestration device 500 of the dispatch platform by running the software programs and modules stored in the memory 509. The memory 509 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function (such as extracting a task to be combed, etc.), and the like; the storage data area may store data created from the use of task orchestration devices of the dispatch platform (such as link levels, etc.), and so on. In addition, the memory 509 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. The task orchestration method programs and received data streams of the dispatch platform provided in embodiments of the present invention are stored in memory and when needed, the processor 501 is invoked from the memory 509.
When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, twisted pair), or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer readable storage medium may be any available medium that can be stored by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., an optical disk), or a semiconductor medium (e.g., a Solid State Disk (SSD)), or the like.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.
In the several embodiments provided in the present invention, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiment of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. A task orchestration method for a dispatch platform, comprising:
task information of all tasks of a scheduling platform is obtained, wherein the task information comprises task names, task states, front task information and rear task information;
extracting tasks to be combed according to the acquired task information to obtain a queue to be combed; the server judges whether the target task is stored in the queue to be combed according to the task state of the target task; the server stores the task names of the tasks into the queue to be combed in the form of an element list;
randomly selecting one task to be combed from the queue to be combed as a target task to be combed, setting a link level of the target task to be combed as K, wherein K is any integer;
Setting a link level of a target dependent task based on a link level K of the target task to be combed according to task information of the task to be combed in the queue to be combed, wherein the target dependent task is the task to be combed with a dependency relationship with the target task to be combed; the server obtains link levels of all tasks in a task link comprising the target task to be combed by circularly searching the task to be combed, which is not after the target task to be combed, in a task link and circularly searching the task to be combed, which is not before the target task to be combed, in the task link according to the task information of the task to be combed;
deleting the task to be combed with the link level from the queue to be combed;
selecting other target to-be-combed tasks randomly, setting link levels of the other target to-be-combed tasks, combing the to-be-combed tasks with dependency relation with the other target to-be-combed tasks, setting the link levels, deleting the to-be-combed tasks with the link levels from the to-be-combed queue until no to-be-combed task exists in the to-be-combed queue;
Outputting the link level of each task to be combed and storing the link level into a preset task relation table;
the deleting the task to be combed with the link layer from the queue to be combed comprises the following steps: the server deletes the task to be combed with the link level set from the queue to be combed by removing the preset element from the list.
2. The task grooming method of the dispatch platform according to claim 1, wherein the setting a link level of a target dependent task based on a link level K of the target task to be groomed according to task information of the task to be groomed in the queue, the target dependent task being a task to be groomed having a dependency relationship with the target task to be groomed, includes:
setting a link level of a front link task in a target dependent task according to task information of the task to be combed in the queue to be combed and a link level K of the target task to be combed, wherein the target dependent task is the task to be combed with a dependency relationship with the target task to be combed, and the front link task is the task to be combed with the execution sequence before the target task to be combed or with the same execution sequence as the target task to be combed;
Setting a link level of a post link task in the target dependent task according to task information of the task to be combed in the queue to be combed and the link level K of the target task to be combed, wherein the post link task is a task to be combed with the execution sequence after the target task to be combed or with the same execution sequence as the target task to be combed.
3. The task grooming method of the dispatch platform according to claim 2, wherein the setting a link hierarchy of a front link task in a target dependent task according to task information of the task to be groomed in the queue to be groomed and the link hierarchy K of the target task to be groomed, includes:
searching the preposed tasks of the target task to be combed according to the preposed task information of the target task to be combed to obtain a first preposed task, wherein the number of the preposed tasks is at least one;
setting the link level of the first pre-task as the link level of the target task to be combed minus 1, and obtaining the link level as K-1;
searching the preposed task of the first preposed task according to the preposed task information of the first preposed task to obtain a second preposed task;
Setting the link level of the second pre-task as the link level of the first pre-task minus 1, and obtaining a link level of K-2;
circularly searching the P-th pre-task and setting a link level as K-P until the pre-task cannot be searched;
searching the post-tasks of the P pre-tasks as first forward tasks according to the post-task information of the P pre-tasks, wherein the number of the post-tasks is at least one;
setting the link level of the first forward task as the link level of the P-th pre-task plus 1 to obtain a link level K-P+1;
and circularly searching the Q forward task and setting the link level as K-P+Q until the link level K-P+Q of the Q forward task is equal to K.
4. The task grooming method of the dispatch platform according to claim 2, wherein the setting the link level of the post-link task in the target dependent task according to the task information of the task to be groomed in the queue to be groomed and the link level K of the target task to be groomed includes:
searching the post-tasks of the target task to be combed according to the post-task information of the target task to be combed to obtain a first post-task, wherein the number of the post-tasks is at least one;
Setting the link level of the first post task as the link level of the target task to be combed plus 1 to obtain a link level of K+1;
searching for the post-task of the first post-task according to the post-task information of the first post-task to obtain a second post-task;
setting the link level of the second post task as the link level of the first post task plus 1 to obtain a link level of K+2;
circularly searching the L-th post task and setting the link level as K+L until the post task cannot be searched;
searching the preposed tasks of the L-th posttask according to the preposed task information of the L-th posttask to obtain a first backward task, wherein the number of the preposed tasks is at least one;
setting the link level of the first backward task as the link level of the L-th backward task minus 1 to obtain a link level K+L-1;
and circularly searching the S-th backward task and setting the link level as K+L-S until the link level K+L-S of the S-th backward task is equal to K.
5. The task combing method of the dispatching platform according to claim 1, wherein the extracting the task to be combed according to the obtained task information to obtain the queue to be combed includes:
Judging whether the target task is stored in a to-be-combed queue according to the task state of the target task, wherein the target task is a task of any one of the dispatching platforms, the task state comprises an enabling state and a stopping state, and the to-be-combed queue is a preset list;
if the task state of the target task is determined to be the starting state, determining that the target task is a task to be combed, and storing the task name of the target task into the queue to be combed in a list element mode;
and if the task state of the target task is determined to be the stop state, not storing the task name of the target task into the queue to be combed.
6. The task grooming method of the dispatch platform according to claim 1, wherein the selecting one task to be groomed randomly from the queue to be groomed as a target task to be groomed, setting a link level of the target task to be groomed to K, where K is any integer, includes:
extracting an element in the queue to be combed through a random sampling function to obtain a task name of a target task to be combed;
and setting a link level of the target task to be combed to K according to the task name of the target task to be combed, wherein K is any integer, and the link level is a preset dynamic statement variable.
7. The task orchestration method according to any one of claims 1-6, wherein the outputting the link level of each task to be orchestrated and storing it in a preset task relationship table, comprises:
acquiring a link level, a task name and a front task of the task to be combed;
and storing the task names, the front tasks and the link levels of the tasks to be combed into a preset task relation table in a record form according to the order from small link levels to large link levels, wherein each task to be combed has at least one record in the preset task relation table by taking the task names and the front tasks as main keys, and one record only comprises one front task.
8. A task orchestration device for a dispatch platform, comprising:
the task information comprises task names, task states, front task information and rear task information;
the extraction unit is used for extracting tasks to be combed according to the acquired task information to obtain a queue to be combed; the server judges whether the target task is stored in the queue to be combed according to the task state of the target task; the server stores the task names of the tasks into the queue to be combed in the form of an element list;
The first setting unit is used for randomly selecting one task to be combed from the queue to be combed as a target task to be combed, and setting the link level of the target task to be combed as K, wherein K is any integer;
the second setting unit is used for setting a link level of a target dependent task based on a link level K of the target task to be combed according to task information of the task to be combed in the queue, wherein the target dependent task is the task to be combed with a dependency relationship with the target task to be combed; the server obtains link levels of all tasks in a task link comprising the target task to be combed by circularly searching the task to be combed, which is not after the target task to be combed, in a task link and circularly searching the task to be combed, which is not before the target task to be combed, in the task link according to the task information of the task to be combed;
the deleting unit is used for deleting the task to be combed, which is already provided with the link level, from the queue to be combed;
the third setting unit is used for randomly selecting other target tasks to be combed and setting link levels of the other target tasks to be combed, combing the tasks to be combed with dependency relation with the other target tasks to be combed and setting the link levels, and deleting the tasks to be combed with the link levels from the queue to be combed until no task to be combed exists in the queue to be combed;
The output unit outputs the link level of each task to be combed and stores the link level into a preset task relation table;
the deleting unit is further configured to delete, by the server, a task to be combed, for which a link level has been set, from the queue to be combed by removing a preset element from the list.
9. Task grooming device for a dispatch platform, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing a task grooming method for a dispatch platform according to any one of claims 1-7 when executing the computer program.
10. A computer readable storage medium comprising instructions which, when run on a computer, cause the computer to perform the task orchestration method of the scheduling platform according to any one of claims 1-7.
CN201910967574.9A 2019-10-12 2019-10-12 Task combing method, device, equipment and storage medium of dispatching platform Active CN110908780B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910967574.9A CN110908780B (en) 2019-10-12 2019-10-12 Task combing method, device, equipment and storage medium of dispatching platform

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910967574.9A CN110908780B (en) 2019-10-12 2019-10-12 Task combing method, device, equipment and storage medium of dispatching platform

Publications (2)

Publication Number Publication Date
CN110908780A CN110908780A (en) 2020-03-24
CN110908780B true CN110908780B (en) 2023-07-21

Family

ID=69815544

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910967574.9A Active CN110908780B (en) 2019-10-12 2019-10-12 Task combing method, device, equipment and storage medium of dispatching platform

Country Status (1)

Country Link
CN (1) CN110908780B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112134739B (en) * 2020-09-24 2022-07-08 平安科技（深圳）有限公司 Link state tracking method and device, electronic equipment and computer storage medium
CN113051019A (en) * 2021-03-31 2021-06-29 北京和信融慧信息科技有限公司 Flow task execution control method, device and equipment

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8387066B1 (en) * 2007-09-28 2013-02-26 Emc Corporation Dependency-based task management using set of preconditions to generate scheduling data structure in storage area network
CN103019691A (en) * 2012-11-20 2013-04-03 北京思特奇信息技术股份有限公司 Transformation method for extract, transform and load (ETL) operation relation graph and implementation system thereof
CN105389207A (en) * 2015-11-06 2016-03-09 成都视达科信息技术有限公司 Adaptive data request processing method and apparatus, task scheduler and adaptive data request processing system
WO2016192604A1 (en) * 2015-06-05 2016-12-08 阿里巴巴集团控股有限公司 Visualization method, device and system for global task node dependence relationship
CN108139931A (en) * 2015-10-16 2018-06-08 高通股份有限公司 It synchronizes to accelerate task subgraph by remapping
CN108287756A (en) * 2018-01-25 2018-07-17 联动优势科技有限公司 A kind of method and device of processing task
CN109144695A (en) * 2018-08-30 2019-01-04 百度在线网络技术（北京）有限公司 A kind of processing method, device, equipment and the medium of task topological relation
CN109918432A (en) * 2019-01-28 2019-06-21 中国平安财产保险股份有限公司 Extract method, apparatus, computer equipment and the storage medium of task nexus chain

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8387066B1 (en) * 2007-09-28 2013-02-26 Emc Corporation Dependency-based task management using set of preconditions to generate scheduling data structure in storage area network
CN103019691A (en) * 2012-11-20 2013-04-03 北京思特奇信息技术股份有限公司 Transformation method for extract, transform and load (ETL) operation relation graph and implementation system thereof
WO2016192604A1 (en) * 2015-06-05 2016-12-08 阿里巴巴集团控股有限公司 Visualization method, device and system for global task node dependence relationship
CN106293928A (en) * 2015-06-05 2017-01-04 阿里巴巴集团控股有限公司 A kind of overall situation task node dependence method for visualizing, device and system
CN108139931A (en) * 2015-10-16 2018-06-08 高通股份有限公司 It synchronizes to accelerate task subgraph by remapping
CN105389207A (en) * 2015-11-06 2016-03-09 成都视达科信息技术有限公司 Adaptive data request processing method and apparatus, task scheduler and adaptive data request processing system
CN108287756A (en) * 2018-01-25 2018-07-17 联动优势科技有限公司 A kind of method and device of processing task
CN109144695A (en) * 2018-08-30 2019-01-04 百度在线网络技术（北京）有限公司 A kind of processing method, device, equipment and the medium of task topological relation
CN109918432A (en) * 2019-01-28 2019-06-21 中国平安财产保险股份有限公司 Extract method, apparatus, computer equipment and the storage medium of task nexus chain

Also Published As

Publication number Publication date
CN110908780A (en) 2020-03-24

Similar Documents

Publication Publication Date Title
CN109582551B (en) Log data analysis method and device, computer equipment and storage medium
US11423053B2 (en) Log event cluster analytics management
KR101600129B1 (en) Application efficiency engine
US9361343B2 (en) Method for parallel mining of temporal relations in large event file
US11169994B2 (en) Query method and query device
US20170132638A1 (en) Relevant information acquisition method and apparatus, and storage medium
CN110908780B (en) Task combing method, device, equipment and storage medium of dispatching platform
JP5730386B2 (en) Computer system and parallel distributed processing method
US9471362B2 (en) Correlating hypervisor data for a virtual machine with associated operating system data
CN110275889B (en) Feature processing method and device suitable for machine learning
JP2014078085A (en) Execution control program, execution control method and information processor
CN103995863B (en) A kind of method and device of data de-duplication
CN104572085A (en) Method and device for analyzing application program
Ting et al. Constructing a cloud computing based social networks data warehousing and analyzing system
CN113868434A (en) Data processing method, device and storage medium for graph database
CN106940710B (en) Information pushing method and device
CN112433757A (en) Method and device for determining interface calling relationship
CN112527904A (en) Automatic synchronization method and device for associated scripts, computer equipment and storage medium
CN112445776A (en) Presto-based dynamic barrel dividing method, system, equipment and readable storage medium
Masouleh et al. Optimization of ETL process in data warehouse through a combination of parallelization and shared cache memory
CN112783417A (en) Data reduction method and device, computing equipment and storage medium
CN110753090A (en) Task execution method and device of server cluster, computer equipment and storage medium
US20220350804A1 (en) Event Sequences Search
CN108958967A (en) A kind of method and server of data processing
CN114547336A (en) Text data processing method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant