CN110609740A - Method and device for determining dependency relationship between tasks - Google Patents
Method and device for determining dependency relationship between tasks Download PDFInfo
- Publication number
- CN110609740A CN110609740A CN201910885703.XA CN201910885703A CN110609740A CN 110609740 A CN110609740 A CN 110609740A CN 201910885703 A CN201910885703 A CN 201910885703A CN 110609740 A CN110609740 A CN 110609740A
- Authority
- CN
- China
- Prior art keywords
- task
- information
- data table
- signal
- table information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5033—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering data affinity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
Abstract
The invention discloses a method and a device for determining a dependency relationship between tasks, wherein the method comprises the following steps: acquiring data table information of each task; the data table information of each task comprises source data table information and target data table information; determining the dependency relationship of each task according to the data table information of each task; when the source data table information of a first task is the target data table information of a second task, determining that the first task depends on the second task; the first task and the second task are any two different tasks in the tasks. When the method is applied to financial technology (Fintech), because the relationship between the table information of each task is the root factor forming the dependency relationship, the dependency relationship of each task can be analyzed radically according to the relationship between the data table information of each task.
Description
Technical Field
The invention relates to the field of financial technology (Fintech) and the field of big data, in particular to a method and a device for determining a dependency relationship between tasks.
Background
With the development of computer technology, more and more technologies (big data, distributed, Blockchain (Blockchain), artificial intelligence, etc.) are applied in the financial field, and the traditional financial industry is gradually changing to financial technology (Fintech). Currently, in the field of financial technology, in order to perform some specific functions, a financial dispatching platform often needs to dispatch tasks, such as a task for connecting two data tables. Because the running of one task may be based on the running result of another task, there may be dependency of the running order between tasks, if the tasks do not run according to the sequence in the dependency relationship, the task scheduling may fail, the financial scheduling platform may retry the tasks frequently, consume a large amount of scheduling resources, and further cause the financial scheduling platform to malfunction and disorder. Therefore, the method clearly and correctly determines the dependency relationship between tasks and has a crucial meaning for maintaining the stability of the financial dispatching platform.
However, in the current financial dispatching platform, the dependency relationship between tasks is determined in a human analysis mode. Obviously, the dependency relationship determined by human analysis is easy to be omitted, so that the configuration of the dependency relationship between tasks is not accurate, which is a problem to be solved urgently.
Disclosure of Invention
The embodiment of the application provides a method and a device for determining a dependency relationship between tasks, and solves the problem that the dependency relationship determined by manual analysis in the prior art is easy to omit.
In a first aspect, an embodiment of the present application provides a method for determining a dependency relationship between tasks, including: acquiring data table information of each task; the data table information of each task comprises source data table information and target data table information; the source data table information is predefined data information required for task operation; the target data table information is predefined data information used for recording the data obtained when the task runs; determining the dependency relationship of each task according to the data table information of each task; when the source data table information of a first task is the target data table information of a second task, determining that the first task depends on the second task; the first task and the second task are any two different tasks in the tasks.
According to the method, the data table information of each task is acquired, the data table information of each task comprises the source data table information and the target data table information, and the relationship between the table information of each task is the root factor for forming the dependency relationship, so that the dependency relationship of each task can be analyzed from the root factor according to the relationship between the data table information of each task, and the dependency relationship of each task can be accurately acquired.
In an optional implementation manner, the determining a dependency relationship of each task according to the data table information of each task includes: aiming at any task, converting the data table information of the task into task associated information of the task; the task associated information of the task comprises precursor associated information and successor associated information; the precursor associated information includes: a predecessor task defaulted to a null value, a source data table of the task, the task; the subsequent associated information comprises: the task, a target data table of the task and a subsequent task with a default value of null; and for any two tasks, when the precursor associated information of the first task and the subsequent associated information of the second task comprise the same data table, determining that the first task and the second task have a dependency relationship.
In the method, aiming at any task, the data table information of the task is converted into the task associated information of the task, the undetermined associated relationship which is formed by connecting the data tables of each task and comprises the precursor associated information and the subsequent associated information can be clearly seen, and the dependency relationship between the first task and the second task can be determined according to the data tables which are included in the precursor associated information and the subsequent associated information subsequently aiming at any two tasks.
In an optional implementation manner, when the source data table information of the first task is the target data table information of the second task, the predecessor associated information of the first task is paired with the successor associated information of the second task, the predecessor task of the first task is the second task, and the successor task of the second task is the first task; and storing the dependency relationship of each task in a directed graph form according to the task association information of each task.
In the method, the precursor task or the subsequent task of the task is determined by comparing the source data table information of the first task with the target data table information of the second task, so that the dependency relationship of each task can be stored in a directed graph mode according to the task association information of each task, and a storage mode of the dependency relationship of each task is provided.
In an optional implementation manner, according to the operation starting time and the operation ending time of each task, the task meeting a first preset rule and/or a second preset rule in each task is taken as a recommended dependent task combination of the signal task; the first preset rule is as follows: the running starting time of the task is behind the running finishing time of the signal task; the second preset rule is as follows: the running end time of the task is before the running start time of the signal task; the signal task is a task which is triggered after one task is finished running in each task, and another task is triggered to start running after the signal task is finished running; and determining the task dependent on the signal task and/or the task dependent on the signal task from the recommended dependent task combination according to the task description information of the task in the recommended dependent task combination and the task description information of the signal task.
In the above manner, according to the operation starting time and the operation ending time of each task, a recommended dependent task combination is screened out according to a first preset rule and/or a second preset rule, and then further screening is performed according to task description information, so that a manner of determining the task on which the signal task depends and/or the task dependent on the signal task is provided.
In an optional implementation manner, the determining, according to the task description information of the task in the recommended dependent task combination and the task description information of the signal task, the task dependent on the signal task and/or the task dependent on the signal task from the recommended dependent task combination includes: performing word segmentation on the task description information of the signal task to obtain a first task characteristic; for any task in the recommendation dependent task combination, performing word segmentation on task description information of the task to obtain a second task characteristic; determining the task similarity of the signal task and the task according to the similarity of the first task characteristic and the second task characteristic; taking the task which meets the first preset rule and has task similarity with the signal task larger than a first threshold value as a task depending on the signal task; and taking the task which meets the second preset rule and has similarity with the signal task larger than a second threshold value as the task on which the signal task depends.
In the above manner, by segmenting the task description information, a first task feature and a second task feature are respectively obtained, the task similarity between the signal task and the task is obtained, and then the task which meets the first preset rule and has the task similarity with the signal task larger than a first threshold value is used as the task depending on the signal task, so that the task depending on the signal task is screened out from the recommended dependent task combination through the task description information.
In a second aspect, the present application provides an apparatus for determining inter-task dependencies, comprising: the acquisition module is used for acquiring the data table information of each task; the data table information of each task comprises source data table information and target data table information; the source data table information is predefined data information required for task operation; the target data table information is predefined data information used for recording the data obtained when the task runs; the processing module is used for determining the dependency relationship of each task according to the data table information of each task; when the source data table information of a first task is the target data table information of a second task, determining that the first task depends on the second task; the first task and the second task are any two different tasks in the tasks.
In an optional implementation manner, the processing module is specifically configured to: aiming at any task, converting the data table information of the task into task associated information of the task; the task associated information of the task comprises precursor associated information and successor associated information; the precursor associated information includes: a predecessor task defaulted to a null value, a source data table of the task, the task; the subsequent associated information comprises: the task, a target data table of the task and a subsequent task with a default value of null; and for any two tasks, when the precursor associated information of the first task and the subsequent associated information of the second task comprise the same data table, determining that the first task and the second task have a dependency relationship.
In an optional embodiment, the processing module is further configured to: when the source data table information of the first task is the target data table information of the second task, the predecessor associated information of the first task is paired with the successor associated information of the second task, the predecessor task of the first task is the second task, and the successor task of the second task is the first task; and storing the dependency relationship of each task in a directed graph form according to the task association information of each task.
In an optional embodiment, the processing module is further configured to: according to the operation starting time and the operation ending time of each task, taking the task meeting a first preset rule and/or a second preset rule in each task as a recommended dependent task combination of the signal tasks; the first preset rule is as follows: the running starting time of the task is behind the running finishing time of the signal task; the second preset rule is as follows: the running end time of the task is before the running start time of the signal task; the signal task is a task which is triggered after one task is finished running in each task, and another task is triggered to start running after the signal task is finished running; and determining the task dependent on the signal task and/or the task dependent on the signal task from the recommended dependent task combination according to the task description information of the task in the recommended dependent task combination and the task description information of the signal task.
In an optional implementation manner, the processing module is specifically configured to: performing word segmentation on the task description information of the signal task to obtain a first task characteristic; for any task in the recommendation dependent task combination, performing word segmentation on task description information of the task to obtain a second task characteristic; determining the task similarity of the signal task and the task according to the similarity of the first task characteristic and the second task characteristic; taking the task which meets the first preset rule and has task similarity with the signal task larger than a first threshold value as a task depending on the signal task; and taking the task which meets the second preset rule and has similarity with the signal task larger than a second threshold value as the task on which the signal task depends.
For the advantages of the second aspect and the embodiments of the second aspect, reference may be made to the advantages of the first aspect and the embodiments of the first aspect, which are not described herein again.
In a third aspect, an embodiment of the present application provides a computer device, which includes a program or instructions, and when the program or instructions are executed, the computer device is configured to perform the method of each embodiment of the first aspect and the first aspect.
In a fourth aspect, an embodiment of the present application provides a storage medium, which includes a program or instructions, and when the program or instructions are executed, the program or instructions are configured to perform the method of the first aspect and the embodiments of the first aspect.
Drawings
Fig. 1 is a schematic flowchart illustrating steps of a method for determining a dependency relationship between tasks according to an embodiment of the present application;
fig. 2 is a schematic flowchart illustrating a step of a task association information pairing method for each task according to an embodiment of the present application;
fig. 3 is a schematic flowchart illustrating steps of a method for determining a dependency relationship between signal tasks according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an apparatus for determining inter-task dependency provided in an embodiment of the present application.
Detailed Description
In order to better understand the technical solutions, the technical solutions will be described in detail below with reference to the drawings and the specific embodiments of the specification, and it should be understood that the specific features in the embodiments and examples of the present application are detailed descriptions of the technical solutions of the present application, but not limitations of the technical solutions of the present application, and the technical features in the embodiments and examples of the present application may be combined with each other without conflict.
For convenience of description, abbreviations and key term definitions in the embodiments of the present application are listed first below.
Task: refers to the carrier of scripts or code that perform certain functions, which may involve processing between table levels, communication between systems, outputting of data, inputting of data, etc.
A dispatching platform: refers to a scheduling platform that completes tasks through a queue mechanism.
Blood margin analysis: the method is used for analyzing the incidence relation between a data table and a data table (also referred to as a table for short) in the big data and between other subjects based on the data table. For example, a subject is a task.
Task dependence: the task B with the later operation order can be operated after the task A with the earlier operation order is operated, and the task B is determined to depend on the task A, and the task A is the task on which the task B depends. When the tasks are all tasks for processing the data table to achieve a specific function, the root cause of the dependency between the tasks is the dependency of the tasks on the data table information. For example, task B needs to use data table B for normal operation, and data table B is obtained by task a through processing data table a, so task B must be executed after task a is executed.
Dependence detection: and checking the incidence relation between one or more table-based subjects, wherein the detection of the dependence is divided into upstream dependence detection, downstream dependence detection and self-dependence detection. For example, the subject is task a. The upstream dependency detection of the task A refers to detecting the task on which the task A depends, the downstream dependency detection of the task A refers to detecting the task depending on the task A, and the self-dependency detection refers to detecting the data table depending on the task A.
In the operation process of carrying out business (such as loan business, deposit business and the like of banks) by a financial institution (a banking institution, an insurance institution or a security institution), in order to complete some specific functions, a financial dispatching platform is often required to dispatch tasks. The method has the advantages that the dependence relationship between tasks is clearly and correctly determined, and the method has important significance for maintaining the stability of the financial dispatching platform.
However, in the current financial dispatching platform, the dependency relationship between tasks is determined in a human analysis mode. Obviously, the dependency relationship determined by human analysis is easy to be omitted, so that the configuration of the dependency relationship between tasks is not accurate, the condition is not in line with the requirements of financial institutions such as banks, and the efficient operation of various services of the financial institutions cannot be ensured.
To this end, as shown in the step flow chart shown in fig. 1, the embodiment of the present application provides a method for determining a dependency relationship between tasks.
Step 101: and acquiring data table information of each task.
Step 102: and determining the dependency relationship of each task according to the data table information of each task.
In step 101, the data table information of each task includes source data table information and target data table information; the source data table information is predefined data information required for task operation; the target data table information is predefined data information obtained when the task runs.
In step 102, when the source data table information of a first task is the target data table information of a second task, determining that the first task depends on the second task; the first task and the second task are any two different tasks in the tasks.
The specific manner of acquiring the data table information of each task in step 101 is not limited. For example, by analyzing the log of each task in the scheduling platform, the association relationship taking the task as the core is obtained, and the blood relationship of the task is reconstructed. The method can be implemented by complementing two strategies of real-time analysis and batch analysis of the logs. And analyzing the logs in real time, namely incrementally acquiring the logs of each task in real time, monitoring whether a newly added log exists or not in real time, and analyzing the newly added log every time one newly added log is detected. The batch analysis of the logs refers to analyzing a plurality of logs at a time, and the logs can be obtained and analyzed according to a certain period or a certain number.
An alternative embodiment (hereinafter referred to as the first embodiment) of step 102 is as follows:
aiming at any task, converting the data table information of the task into task associated information of the task; the task associated information of the task comprises precursor associated information and successor associated information; the precursor associated information includes: a predecessor task defaulted to a null value, a source data table of the task, the task; the subsequent associated information comprises: the task, a target data table of the task and a subsequent task with a default value of null; and for any two tasks, when the precursor associated information of the first task and the subsequent associated information of the second task comprise the same data table, determining that the first task and the second task have a dependency relationship.
For example, the form of the task related information of the task C may be (source database one, source data table 1) - [ relationship: task C ] - > (target database two, target data table 2), and this form represents that the source data table 1 of the source database one and the target data table 2 of the target database two are the join relationship existing with the task C as the core. The task associated information of the task C can be disassembled into: precursor associated information: (pending task) - (source database I, source data Table 1) - > [ relationship: task C ]; subsequent correlation information: [ relationship: task C ] - (target database two, target data Table 2) - > (pending task).
In the first embodiment, the dependency relationship of each task may be paired and stored by the following embodiments (hereinafter referred to as a second embodiment):
when the source data table information of the first task is the target data table information of the second task, the predecessor associated information of the first task is paired with the successor associated information of the second task, the predecessor task of the first task is the second task, and the successor task of the second task is the first task; and storing the dependency relationship of each task in a directed graph form according to the task association information of each task.
For example, if the source data table information of the first task is data table 1 and the target data table information of the second task is data table 1, the predecessor association information of the first task is paired with successor association information of the second task. It should be noted that, the database storing the dependency relationships of the tasks in the form of the directed graph is not limited, such as Neo4 j. In addition, besides storing the task related information of the task according to the graph mechanism, the task related information can also be stored in a relational database.
Specific forms after pairing are exemplified as follows:
the first task is task D. The task associated information of the task D can be disassembled into: precursor associated information: (undetermined task) - (source database III, source data table 3) - > [ relationship: task D ]; subsequent correlation information: [ relationship: task D ] - (target database four, target data Table 4) - > (pending task).
The second task is task C. The task associated information of the task C can be disassembled into: precursor associated information: (pending task) - (source database I, source data Table 1) - > [ relationship: task C ]; subsequent correlation information: [ relationship: task C ] - (target database two, target data Table 2) - > (pending task).
Then after the first task (task D) is paired with the second task (task C), the predecessor task of the first task is the second task and the successor task of the second task is the first task. The predecessor association information for the first task may be updated to: (relationship: task C) - [ Source database three, Source data Table 3] - > (relationship: task D). The subsequent association information of the second task may be updated as: (relationship: task C) - [ target database four, target data Table 4] - > (relationship: task D).
The embodiments of steps 101 to 102 and the first and second embodiments are applicable to tasks including source data table information and target data table information, but there may be tasks that do not include source data table information and target data table information, such as signal tasks. The signal task is a task triggered after one task is finished running in each task, and another task is triggered to start running after the signal task is finished running, but the signal task does not relate to the processing of a data table. For example, i.e. task X to task Y may depend on a certain instruction (signal), task Y also needs a signal task M for its execution. Here, task X may be referred to as an upstream task of signal task M, and task Y may be referred to as a downstream task of signal task M. The relationship between the signals is constructed in three ways and complementally.
The first is a registration type strategy, that is, when each signal task is added, the database directly defines the association mapping from the signal task to other tasks, which is a hard-coded mode.
The second is an automatic analysis strategy, which follows two principles, one of which is that the analysis is implemented according to the operation ending time of the signal task and the operation starting time of the current task, the operation starting time of the signal task should be after the operation ending time of the upstream task, that is, the operation mechanism of the signal task should wait for the upstream task to end, the operation ending time of the signal task should be before the operation ending time of the downstream task, that is, the operation mechanism of the downstream task should wait for the signal task to arrive, and if the operation ending time of the signal task is after the operation starting time of a certain task, the task should not be recommended to depend on the completed signal task. Secondly, if two tasks are in task characteristics (for example, the similarity ratio is more than 85%), it is assumed that the two tasks should have the same dependency relationship on the signal task. Specifically, the task description information may be participled to obtain a weight to obtain task characteristics, and the task description information may include information such as a task name, a task category, and a service category. For example, task X is similar to task Y, task X depends on signal task M, and task Y also depends on signal task M.
The third is a semi-automated enrollment policy. When a user configures a task, the table related to the current task is output in a page form, so that the user can query other tasks related to the table by himself, manually configure the task into a dependency relationship, and store the dependency relationship in a database.
The dependency of the signal tasks can be determined by the following embodiment (hereinafter referred to as a third embodiment):
according to the operation starting time and the operation ending time of each task, taking the task meeting a first preset rule and/or a second preset rule in each task as a recommended dependent task combination of the signal tasks; and determining the task dependent on the signal task and/or the task dependent on the signal task from the recommended dependent task combination according to the task description information of the task in the recommended dependent task combination and the task description information of the signal task.
Wherein the first preset rule is as follows: the running starting time of the task is behind the running finishing time of the signal task; the second preset rule is as follows: the running end time of the task is before the running start time of the signal task.
In the third embodiment, the embodiment (fourth embodiment) of the task dependent on the signal task and/or the task dependent on the signal task is determined from the recommended task-dependent combination, and specifically may be as follows:
performing word segmentation on the task description information of the signal task to obtain a first task characteristic; for any task in the recommendation dependent task combination, performing word segmentation on task description information of the task to obtain a second task characteristic; determining the task similarity of the signal task and the task according to the similarity of the first task characteristic and the second task characteristic; taking the task which meets the first preset rule and has task similarity with the signal task larger than a first threshold value as a task depending on the signal task; and taking the task which meets the second preset rule and has similarity with the signal task larger than a second threshold value as the task on which the signal task depends.
In the methods of steps 101 to 102, the data table information of each task is acquired, the data table information of each task includes the source data table information and the target data table information, and the relationship between the table information of each task is a root factor for forming the dependency relationship, so that the dependency relationship of each task can be analyzed from the root factor according to the relationship between the data table information of each task, and the dependency relationship of each task can be accurately acquired.
The following describes in further detail a flowchart of steps of the inter-task dependency method proposed in the embodiment of the present application with reference to fig. 2 and fig. 3.
Step 201: and acquiring data table information of each task.
Step 202: and acquiring task associated information of each task.
Step 202 may be performed with reference to the first embodiment.
Step 203: and pairing the precursor relation information and the subsequent association information.
Step 203 may be performed with reference to the second embodiment.
After step 203 is completed, the predecessor and successor relationships of the task are determined, and step 204 may be performed.
Step 204: and storing the updated task association information according to the node relation.
Step 301: and acquiring the operation starting time and the operation ending time of each task.
Step 302: and acquiring task description information of each task and each signal task.
Step 303: and acquiring task characteristics of each task and the signal task.
It should be noted that step 301 and step 303 may not both be performed; step 302 may not be performed when step 303 is not performed.
Step 304: and acquiring task similarity between the signal task and each task.
Step 303 and step 304 may be performed with reference to the fourth embodiment.
Step 305: and determining whether the task similarity is in a preset interval.
If yes, go to step 306; if not, return to step 301 and/or step 302.
Step 306: outputting a dependent task of the presence signal task.
As shown in fig. 4, the present application provides an apparatus for determining inter-task dependency, including: an obtaining module 401, configured to obtain data table information of each task; the data table information of each task comprises source data table information and target data table information; the source data table information is predefined data information required for task operation; the target data table information is predefined data information used for recording the data obtained when the task runs; a processing module 402, configured to determine a dependency relationship of each task according to the data table information of each task; when the source data table information of a first task is the target data table information of a second task, determining that the first task depends on the second task; the first task and the second task are any two different tasks in the tasks.
In an optional implementation manner, the processing module 402 is specifically configured to: aiming at any task, converting the data table information of the task into task associated information of the task; the task associated information of the task comprises precursor associated information and successor associated information; the precursor associated information includes: a predecessor task defaulted to a null value, a source data table of the task, the task; the subsequent associated information comprises: the task, a target data table of the task and a subsequent task with a default value of null; and for any two tasks, when the precursor associated information of the first task and the subsequent associated information of the second task comprise the same data table, determining that the first task and the second task have a dependency relationship.
In an optional implementation, the processing module 402 is further configured to: when the source data table information of the first task is the target data table information of the second task, the predecessor associated information of the first task is paired with the successor associated information of the second task, the predecessor task of the first task is the second task, and the successor task of the second task is the first task; and storing the dependency relationship of each task in a directed graph form according to the task association information of each task.
In an optional implementation, the processing module 402 is further configured to: according to the operation starting time and the operation ending time of each task, taking the task meeting a first preset rule and/or a second preset rule in each task as a recommended dependent task combination of the signal tasks; the first preset rule is as follows: the running starting time of the task is behind the running finishing time of the signal task; the second preset rule is as follows: the running end time of the task is before the running start time of the signal task; the signal task is a task which is triggered after one task is finished running in each task, and another task is triggered to start running after the signal task is finished running; and determining the task dependent on the signal task and/or the task dependent on the signal task from the recommended dependent task combination according to the task description information of the task in the recommended dependent task combination and the task description information of the signal task.
In an optional implementation manner, the processing module 402 is specifically configured to: performing word segmentation on the task description information of the signal task to obtain a first task characteristic; for any task in the recommendation dependent task combination, performing word segmentation on task description information of the task to obtain a second task characteristic; determining the task similarity of the signal task and the task according to the similarity of the first task characteristic and the second task characteristic; taking the task which meets the first preset rule and has task similarity with the signal task larger than a first threshold value as a task depending on the signal task; and taking the task which meets the second preset rule and has similarity with the signal task larger than a second threshold value as the task on which the signal task depends.
Embodiments of the present application provide a computer device, which includes a program or instructions, and when the program or instructions are executed, the program or instructions are configured to perform a method for determining a dependency relationship between tasks and any optional method provided by embodiments of the present application.
Embodiments of the present application provide a storage medium, which includes a program or instructions, and when the program or instructions are executed, the program or instructions are configured to perform a method for determining a dependency relationship between tasks and any optional method provided in embodiments of the present application.
Finally, it should be noted that: as will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (12)
1. A method of determining dependencies between tasks, comprising:
acquiring data table information of each task; the data table information of each task comprises source data table information and target data table information; the source data table information is predefined data information required for task operation; the target data table information is predefined data information used for recording the data obtained when the task runs;
determining the dependency relationship of each task according to the data table information of each task; when the source data table information of a first task is the target data table information of a second task, determining that the first task depends on the second task; the first task and the second task are any two different tasks in the tasks.
2. The method of claim 1, wherein determining the dependency of each task based on the spreadsheet information for each task comprises:
aiming at any task, converting the data table information of the task into task associated information of the task; the task associated information of the task comprises precursor associated information and successor associated information; the precursor associated information includes: a predecessor task defaulted to a null value, a source data table of the task, the task; the subsequent associated information comprises: the task, a target data table of the task and a subsequent task with a default value of null;
and for any two tasks, when the precursor associated information of the first task and the subsequent associated information of the second task comprise the same data table, determining that the first task and the second task have a dependency relationship.
3. The method of claim 2, further comprising:
when the source data table information of the first task is the target data table information of the second task, the predecessor associated information of the first task is paired with the successor associated information of the second task, the predecessor task of the first task is the second task, and the successor task of the second task is the first task;
and storing the dependency relationship of each task in a directed graph form according to the task association information of each task.
4. The method of any of claims 1 to 3, further comprising:
according to the operation starting time and the operation ending time of each task, taking the task meeting a first preset rule and/or a second preset rule in each task as a recommended dependent task combination of the signal tasks; the first preset rule is as follows: the running starting time of the task is behind the running finishing time of the signal task; the second preset rule is as follows: the running end time of the task is before the running start time of the signal task; the signal task is a task which is triggered after one task is finished running in each task, and another task is triggered to start running after the signal task is finished running;
and determining the task dependent on the signal task and/or the task dependent on the signal task from the recommended dependent task combination according to the task description information of the task in the recommended dependent task combination and the task description information of the signal task.
5. The method of claim 4, wherein the determining the dependent tasks of the signal task and/or the tasks dependent on the signal task from the recommended dependent task combination according to the task description information of the tasks in the recommended dependent task combination and the task description information of the signal task comprises:
performing word segmentation on the task description information of the signal task to obtain a first task characteristic;
for any task in the recommendation dependent task combination, performing word segmentation on task description information of the task to obtain a second task characteristic; determining the task similarity of the signal task and the task according to the similarity of the first task characteristic and the second task characteristic;
taking the task which meets the first preset rule and has task similarity with the signal task larger than a first threshold value as a task depending on the signal task; and taking the task which meets the second preset rule and has similarity with the signal task larger than a second threshold value as the task on which the signal task depends.
6. An apparatus for determining dependencies between tasks, comprising:
the acquisition module is used for acquiring the data table information of each task; the data table information of each task comprises source data table information and target data table information; the source data table information is predefined data information required for task operation; the target data table information is predefined data information used for recording the data obtained when the task runs;
the processing module is used for determining the dependency relationship of each task according to the data table information of each task; when the source data table information of a first task is the target data table information of a second task, determining that the first task depends on the second task; the first task and the second task are any two different tasks in the tasks.
7. The apparatus of claim 6, wherein the processing module is specifically configured to:
aiming at any task, converting the data table information of the task into task associated information of the task; the task associated information of the task comprises precursor associated information and successor associated information; the precursor associated information includes: a predecessor task defaulted to a null value, a source data table of the task, the task; the subsequent associated information comprises: the task, a target data table of the task and a subsequent task with a default value of null;
and for any two tasks, when the precursor associated information of the first task and the subsequent associated information of the second task comprise the same data table, determining that the first task and the second task have a dependency relationship.
8. The apparatus of claim 7, wherein the processing module is further to:
when the source data table information of the first task is the target data table information of the second task, the predecessor associated information of the first task is paired with the successor associated information of the second task, the predecessor task of the first task is the second task, and the successor task of the second task is the first task;
and storing the dependency relationship of each task in a directed graph form according to the task association information of each task.
9. The apparatus of any of claims 6 to 8, wherein the processing module is further to:
according to the operation starting time and the operation ending time of each task, taking the task meeting a first preset rule and/or a second preset rule in each task as a recommended dependent task combination of the signal tasks; the first preset rule is as follows: the running starting time of the task is behind the running finishing time of the signal task; the second preset rule is as follows: the running end time of the task is before the running start time of the signal task; the signal task is a task which is triggered after one task is finished running in each task, and another task is triggered to start running after the signal task is finished running;
and determining the task dependent on the signal task and/or the task dependent on the signal task from the recommended dependent task combination according to the task description information of the task in the recommended dependent task combination and the task description information of the signal task.
10. The apparatus of claim 9, wherein the processing module is specifically configured to:
performing word segmentation on the task description information of the signal task to obtain a first task characteristic;
for any task in the recommendation dependent task combination, performing word segmentation on task description information of the task to obtain a second task characteristic; determining the task similarity of the signal task and the task according to the similarity of the first task characteristic and the second task characteristic;
taking the task which meets the first preset rule and has task similarity with the signal task larger than a first threshold value as a task depending on the signal task; and taking the task which meets the second preset rule and has similarity with the signal task larger than a second threshold value as the task on which the signal task depends.
11. A computer device comprising a program or instructions that, when executed, perform the method of any of claims 1 to 5.
12. A storage medium comprising a program or instructions which, when executed, perform the method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910885703.XA CN110609740A (en) | 2019-09-19 | 2019-09-19 | Method and device for determining dependency relationship between tasks |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910885703.XA CN110609740A (en) | 2019-09-19 | 2019-09-19 | Method and device for determining dependency relationship between tasks |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110609740A true CN110609740A (en) | 2019-12-24 |
Family
ID=68891589
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910885703.XA Pending CN110609740A (en) | 2019-09-19 | 2019-09-19 | Method and device for determining dependency relationship between tasks |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110609740A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111581207A (en) * | 2020-04-13 | 2020-08-25 | 深圳市云智融科技有限公司 | Method and device for generating files of Azkaban project and terminal equipment |
| CN111858065A (en) * | 2020-07-28 | 2020-10-30 | 中国平安财产保险股份有限公司 | Data processing method, device, storage medium and device |
| CN112506957A (en) * | 2020-12-18 | 2021-03-16 | 杭州数梦工场科技有限公司 | Method and device for determining workflow dependency relationship |
| CN112732669A (en) * | 2020-12-31 | 2021-04-30 | 北京达佳互联信息技术有限公司 | Data processing method and device |
| CN113098960A (en) * | 2021-03-31 | 2021-07-09 | 北京三快在线科技有限公司 | Service operation method, device, server and storage medium |
| CN114048236A (en) * | 2021-11-17 | 2022-02-15 | 中电金信软件有限公司 | Data information query method and device, electronic equipment and readable storage medium |
-
2019
- 2019-09-19 CN CN201910885703.XA patent/CN110609740A/en active Pending
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111581207A (en) * | 2020-04-13 | 2020-08-25 | 深圳市云智融科技有限公司 | Method and device for generating files of Azkaban project and terminal equipment |
| CN111581207B (en) * | 2020-04-13 | 2023-12-29 | 深圳市云智融科技有限公司 | File generation method and device of Azkaban project and terminal equipment |
| CN111858065A (en) * | 2020-07-28 | 2020-10-30 | 中国平安财产保险股份有限公司 | Data processing method, device, storage medium and device |
| CN111858065B (en) * | 2020-07-28 | 2023-02-03 | 中国平安财产保险股份有限公司 | Data processing method, device, storage medium and device |
| CN112506957A (en) * | 2020-12-18 | 2021-03-16 | 杭州数梦工场科技有限公司 | Method and device for determining workflow dependency relationship |
| CN112732669A (en) * | 2020-12-31 | 2021-04-30 | 北京达佳互联信息技术有限公司 | Data processing method and device |
| CN112732669B (en) * | 2020-12-31 | 2024-03-19 | 北京达佳互联信息技术有限公司 | Data processing method and device |
| CN113098960A (en) * | 2021-03-31 | 2021-07-09 | 北京三快在线科技有限公司 | Service operation method, device, server and storage medium |
| CN114048236A (en) * | 2021-11-17 | 2022-02-15 | 中电金信软件有限公司 | Data information query method and device, electronic equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110609740A (en) | Method and device for determining dependency relationship between tasks | |
| US10347019B2 (en) | Intelligent data munging | |
| CN108595157B (en) | Block chain data processing method, device, equipment and storage medium | |
| CN108764674B (en) | Risk control method and device based on rule engine | |
| TWI723528B (en) | Computer-executed event risk assessment method and device, computer-readable storage medium and computing equipment | |
| JP6419859B2 (en) | Interactive interface for machine learning model evaluation | |
| US20170109657A1 (en) | Machine Learning-Based Model for Identifying Executions of a Business Process | |
| US20170109676A1 (en) | Generation of Candidate Sequences Using Links Between Nonconsecutively Performed Steps of a Business Process | |
| JP7125900B2 (en) | A multidimensional recursive learning process and system used to discover complex dyadic or multiple counterparty relationships | |
| US20170109668A1 (en) | Model for Linking Between Nonconsecutively Performed Steps in a Business Process | |
| US11769008B2 (en) | Predictive analysis systems and methods using machine learning | |
| US20170109636A1 (en) | Crowd-Based Model for Identifying Executions of a Business Process | |
| US20210209486A1 (en) | System and method for anomaly detection for time series data | |
| US20170109639A1 (en) | General Model for Linking Between Nonconsecutively Performed Steps in Business Processes | |
| US20170109638A1 (en) | Ensemble-Based Identification of Executions of a Business Process | |
| CN109992588A (en) | It is a kind of to divide folk prescription method and relevant device based on data processing | |
| CN109918408A (en) | A kind of blacklist update method, device, equipment and computer readable storage medium | |
| CN112907358A (en) | Loan user credit scoring method, loan user credit scoring device, computer equipment and storage medium | |
| CN111221873A (en) | Inter-enterprise homonym identification method and system based on associated network | |
| US20170109640A1 (en) | Generation of Candidate Sequences Using Crowd-Based Seeds of Commonly-Performed Steps of a Business Process | |
| Vig et al. | Test effort estimation and prediction of traditional and rapid release models using machine learning algorithms | |
| US20190379623A1 (en) | Message recognition system and method configurable to define new message formats | |
| US20240086165A1 (en) | Systems and methods for building and deploying machine learning applications | |
| KR102155839B1 (en) | Method and apparatus for monitoring worker based on the reason for not wokring | |
| US20170109637A1 (en) | Crowd-Based Model for Identifying Nonconsecutive Executions of a Business Process |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |