CN111061551A - Node merging and scheduling method, device, equipment and storage medium - Google Patents
Node merging and scheduling method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN111061551A CN111061551A CN201911249012.7A CN201911249012A CN111061551A CN 111061551 A CN111061551 A CN 111061551A CN 201911249012 A CN201911249012 A CN 201911249012A CN 111061551 A CN111061551 A CN 111061551A
- Authority
- CN
- China
- Prior art keywords
- dag
- node
- nodes
- scheduling
- serial
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method, a device, equipment and a storage medium for merging and scheduling nodes, wherein the method comprises the steps of merging and generating a corresponding serial DAG according to a DAG dependency relationship and a node dependency relationship corresponding to each node in a DAG by acquiring the DAG dependency relationship corresponding to each directed acyclic graph DAG and the node dependency relationship corresponding to each node in the DAG; when a target DAG in the serial DAG is executed, judging whether all nodes of a previous DAG corresponding to the target DAG are scheduled and executed according to the DAG dependency relationship and the node dependency relationship; and if the scheduling execution of all nodes of the last DAG corresponding to the target DAG is completed, scheduling and executing the target DAG so as to schedule and execute all nodes of the DAG in the serial DAG. According to the method, the multiple DAGs are combined into the serial DAG based on the DAG dependency relationship and the node dependency relationship, so that the layer-by-layer nesting of the DAG is avoided, the logic of the scheduling code is simplified, the development efficiency of the scheduling code is improved, and the convenience and the efficiency of subsequent business process scheduling are improved.
Description
Technical Field
The present invention relates to the field of financial technology (Fintech), and in particular, to a method, an apparatus, a device, and a computer-readable storage medium for merged scheduling of nodes.
Background
With the development of computer technology, more and more technologies are applied in the financial field, the traditional financial industry is gradually changing to financial technology (Fintech), and the process engine scheduling technology is not exceptional, but higher requirements are also put forward on the process engine scheduling technology due to the requirements of security and real-time performance of the financial industry. The existing business process is generally scheduled by a process engine, each link of the business process is abstracted to be a process node, and when the process engine is scheduled, the precedence and the dependency of all Dag (direct Acyclic graphs) need to be configured in advance. For example: when scheduling the DAG, in order to improve efficiency, while scheduling the head node Dag1 of the DAG, all nodes (e.g., B1, B2, B3) in Dag1 are also scheduled, and when scheduling of all nodes of Dag1 is completed, scheduling of Dag2 and Dag3 is continued until scheduling of 3 Dag nodes of the DAG is completed. At present, due to the complexity of services, DAG scheduling refers to simultaneous scheduling of multiple Dag, and besides scheduling of each Dag, scheduling of nodes in Dag is also needed, that is, two layers of Dag nesting or multiple layers of Dag nesting are generated, in this case, the complexity of scheduling code logic is easily increased, and the development efficiency of scheduling codes is reduced, so that the scheduling of subsequent service flows is more complex and the efficiency is reduced.
Disclosure of Invention
The invention mainly aims to provide a node merging and scheduling method, a node merging and scheduling device, node merging and scheduling equipment and a computer readable storage medium, and aims to solve the technical problems that the scheduling code logic of the conventional business process node is high in complexity and the development efficiency of the scheduling code is low.
In order to achieve the above object, the present invention provides a method for merging and scheduling nodes, where the method for merging and scheduling nodes includes the following steps:
acquiring DAG dependency corresponding to each directed acyclic graph DAG and node dependency corresponding to each node in the DAG, and combining and generating corresponding serial DAG according to the DAG dependency and the node dependency;
when a target DAG in the serial DAG is executed, judging whether all nodes of a previous DAG corresponding to the target DAG are scheduled and executed according to the DAG dependency relationship and the node dependency relationship;
and if the scheduling execution of all nodes of the last DAG corresponding to the target DAG is completed, scheduling and executing the target DAG so as to schedule and execute all nodes of the DAG in the serial DAG.
Optionally, after the step of obtaining the DAG dependency corresponding to each directed acyclic graph DAG and the node dependency corresponding to each node in the DAG and merging and generating the corresponding serial DAG according to the DAG dependency and the node dependency, the method further includes:
when the fact that the multitask DAG with the end node being the parallel node exists in the serial DAG is detected, after the end node of the multitask DAG is scheduled and executed, an end mark is set behind the end node of the multitask DAG to represent that all nodes of the multitask DAG are scheduled and executed completely.
Optionally, the end identifier comprises an end placeholder or a virtual node.
Optionally, when executing the target DAG in the serial DAG, the step of determining whether all nodes of a previous DAG corresponding to the target DAG are scheduled to be executed and completed according to the DAG dependency relationship and the node dependency relationship specifically includes:
when a target DAG in the serial DAG is executed, determining a previous DAG corresponding to the target DAG according to the DAG dependency relationship and the node dependency relationship;
and when the last DAG corresponding to the target DAG is a multi-task DAG, determining whether the ending identifier exists behind the ending node of the last DAG corresponding to the target DAG so as to judge whether all nodes of the last DAG corresponding to the target DAG are scheduled and executed.
Optionally, the method for merging and scheduling of nodes further includes:
judging whether two connected end marks exist in the serial DAG;
and if two connected end identifiers exist in the serial DAG, deleting an end identifier in the two connected end placeholders.
Optionally, the method for merging and scheduling of nodes further includes:
judging whether the parallel end nodes in the multi-task DAG correspond to two or more end identifiers;
and if the parallel end node in the multi-task DAG corresponds to two or more end identifiers, combining the two or more end identifiers corresponding to the parallel end node into an end identifier.
In addition, to achieve the above object, the present invention further provides a node merge scheduling apparatus, including:
a DAG serial merging module, configured to obtain a DAG dependency relationship corresponding to each directed acyclic graph DAG and a node dependency relationship corresponding to each node in the DAG, and merge and generate a corresponding serial DAG according to the DAG dependency relationship and the node dependency relationship;
a DAG execution judgment module, configured to, when executing a target DAG in the serial DAG, judge whether all nodes of a previous DAG corresponding to the target DAG are scheduled and executed according to the DAG dependency relationship and the node dependency relationship;
and the DAG execution scheduling module is used for scheduling and executing the target DAG to schedule and execute all DAG nodes in the serial DAG if all the nodes of the last DAG corresponding to the target DAG are scheduled and executed.
Optionally, the merge scheduling apparatus of the node further includes an end identifier adding module, where the end identifier adding module is configured to:
when the fact that the multitask DAG with the end node being the parallel node exists in the serial DAG is detected, after the end node of the multitask DAG is scheduled and executed, an end mark is set behind the end node of the multitask DAG to represent that all nodes of the multitask DAG are scheduled and executed completely.
Optionally, the DAG execution determination module specifically includes:
an upper DAG determining unit, configured to determine, when a target DAG in the serial DAG is executed, an upper DAG corresponding to the target DAG according to the DAG dependency and the node dependency;
and an end identifier determining unit, configured to determine whether the end identifier exists after an end node of the previous DAG corresponding to the target DAG when the previous DAG corresponding to the target DAG is the multitask DAG, so as to determine whether all nodes of the previous DAG corresponding to the target DAG are scheduled and executed.
Optionally, the merge scheduling device of the node further includes a connected placeholder deleting module, where the connected placeholder deleting module is configured to:
judging whether two connected end marks exist in the serial DAG;
and if two connected end identifiers exist in the serial DAG, deleting an end identifier in the two connected end placeholders.
Optionally, the merge scheduler of the node further includes a parallel placeholder merge module, where the parallel placeholder merge module is configured to:
judging whether the parallel end nodes in the multi-task DAG correspond to two or more end identifiers;
and if the parallel end node in the multi-task DAG corresponds to two or more end identifiers, combining the two or more end identifiers corresponding to the parallel end node into an end identifier.
In addition, to achieve the above object, the present invention further provides a merged scheduling device of a node, where the merged scheduling device of the node includes: a memory, a processor and a merged scheduler of nodes stored on the memory and operable on the processor, the merged scheduler of nodes implementing the steps of the merged scheduling method of nodes as described above when executed by the processor.
In addition, to achieve the above object, the present invention also provides a computer readable storage medium having stored thereon a merge scheduler of nodes, which when executed by a processor implements the steps of the merge scheduling method of nodes as described above.
The invention provides a node merging and scheduling method, which comprises the steps of acquiring DAG dependency corresponding to each directed acyclic graph DAG and node dependency corresponding to each node in the DAG, merging and generating corresponding serial DAG according to the DAG dependency and the node dependency; when a target DAG in the serial DAG is executed, judging whether all nodes of a previous DAG corresponding to the target DAG are scheduled and executed according to the DAG dependency relationship and the node dependency relationship; and if the scheduling execution of all nodes of the last DAG corresponding to the target DAG is completed, scheduling and executing the target DAG so as to schedule and execute all nodes of the DAG in the serial DAG. Through the mode, the multiple DAGs are combined into the serial DAG based on the DAG dependency relationship and the node dependency relationship, so that the multi-level nesting of the DAGs is avoided, the logic of the scheduling codes is simplified, the development efficiency of the scheduling codes is improved, the technical problems that the logic of the scheduling codes of the conventional business process nodes is high in complexity and the development efficiency of the scheduling codes is low are solved, and the convenience and the efficiency of subsequent business process scheduling are improved.
Drawings
FIG. 1 is a schematic diagram of an apparatus architecture of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a first embodiment of a node merge scheduling method according to the present invention;
FIG. 3 is a diagram illustrating DAG dependencies in the present invention;
FIG. 4 is a schematic diagram of a node dependency relationship corresponding to each node in a DAG according to the present invention;
FIG. 5 is a diagram of a merged serial DAG according to the present invention;
FIG. 6 is a schematic diagram of an end node setting a virtual node in the present invention;
FIG. 7 is a schematic diagram of 3-node tasks in the present invention;
FIG. 8 is a diagram of a DAG after merging serializations of multitasking nodes in the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As shown in fig. 1, fig. 1 is a schematic device structure diagram of a hardware operating environment according to an embodiment of the present invention.
The merging and dispatching equipment of the node of the embodiment of the invention can be a PC (personal computer) or server equipment, and a Java virtual machine runs on the merging and dispatching equipment.
As shown in fig. 1, the merge scheduling apparatus of the node may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the configuration of the apparatus shown in fig. 1 is not intended to be limiting of the apparatus and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a kind of computer storage medium, may include therein a merged scheduler of an operating system, a network communication module, a user interface module, and a node.
In the device shown in fig. 1, the network interface 1004 is mainly used for connecting to a backend server and performing data communication with the backend server; the user interface 1003 is mainly used for connecting a client (user side) and performing data communication with the client; and the processor 1001 may be configured to call the merge scheduler of the node stored in the memory 1005 and perform the operations in the merge scheduling method of the node described below.
Based on the hardware structure, the embodiment of the node merging and scheduling method is provided.
Referring to fig. 2, fig. 2 is a flowchart illustrating a first embodiment of a merged scheduling method of a node according to the present invention, where the merged scheduling method of the node includes:
step S10, acquiring DAG dependency corresponding to each directed acyclic graph DAG and node dependency corresponding to each node in the DAG, and merging and generating a corresponding serial DAG according to the DAG dependency and the node dependency;
as shown in fig. 3, the DAG contains 4 Dag, Dag1, Dag2, Dag3, and Dag4, respectively. As shown in fig. 4, each Dag has start and end equal nodes, Dag1 includes nodes A, B1, B2, and B3, where a is the start node, B1, B2, B3 are the end nodes, and Dag2 includes start nodes a1, a2, A3, and end nodes B. All Dag precedence and dependencies are configured in advance. Currently, the DAG in fig. 3 is instantiated first, since the head node of the DAG is Dag1, and meanwhile Dag1 in fig. 4 is also instantiated, after all nodes in Dag1 are instantiated, the instantiation of Dag2 and Dag3 is continued until Dag4 nodes in fig. 3 are instantiated, and the whole DAG instantiation ends. Since multiple Dag are instantiated at the same time, besides the scheduling between Dag is done, the child node A, B and the C node in Dag are also scheduled, that is, two layers of Dag nesting are generated (multiple layers of Dag nesting may occur in a real scene), and therefore, the complexity of scheduling code logic is increased. In addition, most of the existing third-party process engines can only support a single Dag scene, so that most of the third-party process engines cannot be accessed by two or more layers of Dag nests, and the adaptability of node scheduling is reduced.
In order to solve the above problems, in this embodiment, based on the DAG dependency relationship and the node dependency relationship, multiple DAGs are merged into one serial DAG, so that DAG layer-by-layer nesting is avoided, scheduling code logic is simplified, and development efficiency of scheduling codes is improved. The concrete instantiation process is illustrated by the following concrete business scenarios: a company does remote disaster recovery exercise, and relates to M departments, wherein each department has tens of different colleagues to participate, and because the number of the concerned persons is large, an initiator only knows the sequence of the M departments, and cannot clearly know the colleagues and the sequence of the participation of each department. According to the existing scheme, each department is an independent Dag, so that all Dag cannot be instantiated at the beginning of a drill, when the drill is half executed, due to the fact that the dependency relationship of a drill co-worker of a certain department is matched, when Dag of the department is instantiated, the drill sequence of the drill co-workers of the department is not correct, the drill can be stopped in advance, the whole business process is affected, the previous workload is invalid, and the efficiency is reduced. In addition, in the existing scheme, the initiator cannot directly configure and drill all the colleagues in the sequence due to the fact that the sequence of colleagues in other departments cannot be clear and the departments cannot be directly ignored, and needs to be coordinated with the responsible person of the departments, so that the difficulty of configuration of the initiator is increased. In the process of configuring by the initiator, multiple nodes may exist in a multi-node dependent scenario (multiple colleagues in the last step of the department a are parallel, multiple colleagues in the beginning step of the department B are parallel, and the beginning step of configuring the department B depends on the department a to end the colleagues respectively because of no configuration of the department), so that the complexity of the instantiation logic is increased.
According to the scheme provided by the embodiment, firstly, independent Dag corresponding to M departments are combined into a serial Dag according to the Dag dependency corresponding to each Dag and the node dependency corresponding to each node in Dag, so that the scheduling logic of the service flow nodes is simplified, errors in configuration sequence during instantiation can be prevented, and the configuration difficulty of an initiator is reduced. And the abnormity can be conveniently found in advance when the drilling is started (instantiated), and adjustment is carried out, so that the problem found in the middle of the drilling can be avoided, and the drilling can not be continued.
Specifically, the directed Acyclic graph is referred to as dag (directed Acyclic graph), and the directed Acyclic graph is referred to as: any one side has a direction and there is no graph of loops, which in this case refers to a flow (parallel or serial). Acquiring a DAG dependency relationship (such as Dag1-Dag2-Dag3-Dag4) corresponding to each DAG and a node dependency relationship corresponding to each node in the DAG, and determining the dependency relationship corresponding to each node in the DAG according to the DAG dependency relationship and the node dependency relationship. And then combining the DAGs to generate corresponding serial DAGs based on the dependency relationship corresponding to each node in the DAGs. For example, Dag1 and Dag2 in the DAG of fig. 3 are disassembled to finally generate the DAG as shown in fig. 5, and the scheduling of Dag1, Dag2 and Dag3 … child nodes is not considered after the DAG is instantiated, and only a single Dag is processed. For example, the merge is preceded by: dag2 dependency Dag1 configured as shown in fig. 3 needs to replace dependency relationship points of (Dag2-a1, Dag2-a2, Dag2-A3) of Dag2 with (Dag1-B1 … B3) of Dag1 when the Dag2 dependency relationship points are combined into a single Dag, and the flow can be scheduled by adopting a single Dag scheduling mode after the replacement is completed.
Step S20, when executing the target DAG in the serial DAG, judging whether all nodes of the last DAG corresponding to the target DAG are scheduled and executed according to the DAG dependency relationship and the node dependency relationship;
in this embodiment, each of the serial DAGs is sequentially executed, that is, each of the serial DAGs is sequentially used as a target DAG. When executing a target DAG in the serial DAG, determining a last DAG of the target DAG according to DAG dependency relationship. And determining the execution states of all nodes in the last DAG according to the node dependency relationship corresponding to each node in the DAG, namely judging whether all nodes of the last DAG corresponding to the target DAG are scheduled and executed.
Step S30, if the scheduling and executing of all nodes of the previous DAG corresponding to the target DAG are completed, then scheduling and executing the target DAG to schedule and execute all nodes of the DAG in the serial DAG.
Since all Dag are not instantiated from the beginning, possibly at the time of later Dag instantiation, the nesting of two or more layers Dag in the existing solution is prone to instantiation errors due to dependencies and the like, and therefore a scheduling flow is interrupted, and therefore manual intervention is needed. In this embodiment, after it is detected that the scheduling execution of all nodes of the previous DAG corresponding to the target DAG is completed, the target DAG is scheduled and executed. Thus, a single execution of the DAG is ensured, and the nodes of all DAGs in the serial DAG are completed in sequence. And if detecting that the execution of a certain node of the last DAG corresponding to the target DAG is not scheduled and completed, continuing waiting until the execution of the incomplete node of the last DAG corresponding to the target DAG is completed, and starting to schedule and execute the target DAG.
The embodiment provides a node merging and scheduling method, which includes acquiring a DAG dependency relationship corresponding to each directed acyclic graph DAG and a node dependency relationship corresponding to each node in the DAG, and merging and generating a corresponding serial DAG according to the DAG dependency relationship and the node dependency relationship; when a target DAG in the serial DAG is executed, judging whether all nodes of a previous DAG corresponding to the target DAG are scheduled and executed according to the DAG dependency relationship and the node dependency relationship; and if the scheduling execution of all nodes of the last DAG corresponding to the target DAG is completed, scheduling and executing the target DAG so as to schedule and execute all nodes of the DAG in the serial DAG. Through the mode, the multiple DAGs are combined into one serial DAG based on the DAG dependency relationship and the node dependency relationship, so that the multi-level nesting of the DAGs is avoided, the logic of the scheduling codes is simplified, the development efficiency of the scheduling codes is improved, and the technical problems that the existing scheduling code logic is high in complexity and the development efficiency of the scheduling codes is low are solved.
Further, based on the first embodiment of the merging scheduling method of the node of the present invention, a second embodiment of the merging scheduling method of the node of the present invention is provided.
In this embodiment, after the step S10, the method further includes:
when the fact that the multitask DAG with the end node being the parallel node exists in the serial DAG is detected, after the end node of the multitask DAG is scheduled and executed, an end mark is set behind the end node of the multitask DAG to represent that all nodes of the multitask DAG are scheduled and executed completely. Wherein the end identifier comprises an end placeholder or a virtual node.
In the first embodiment, as shown in fig. 5, when there are many end nodes of Dag1, for example, when Dag1-B1, Dag1-B2, and Dag1-B3 of Dag1 are end nodes that are parallel nodes, Dag1 is a multi-task DAG, and the first nodes Dag2-a1, Dag2-a2, and Dag2-A3 of Dag2 need to perform traversal detection execution results on the last nodes Dag1-B1, Dag1-B2, and Dag1-B3 of Dag1, respectively. In this embodiment, in order to facilitate detecting whether all nodes of the last DAG are completely executed, when detecting that the last step of Dag is multitask parallel, after the parallel is finished, an end identifier, such as an end placeholder or a virtual node, is added at the end of the end node. Where the end placeholder (virtual node) does not handle anything in the flow node for convenience in joining the back Dag. If an end placeholder (virtual node) is found, flow automatically skips executing the node in the next Dag. Specifically, two layers Dag and multiple layers Dag are illustrated separately:
1. as shown in fig. 6, a complex two-layer multi-Dag scene is converted into a single Dag, and virtual nodes are added at a specific Dag, after Dag1 and Dag2 are disassembled in the DAG shown in fig. 3, the DAG shown in fig. 6 is finally generated, and after instantiation is used, the scheduling of Dag1, Dag2 and Dag3 … child nodes is not considered, and only a single Dag is used for processing.
2. Dag1-B1 is a 3-node task, as shown in FIG. 7, which is merged into a serial DAG, Dag1-B1 is partially replaced by dotted lines, as shown in FIG. 8.
By the above manner, after all the end nodes of Dag1 are executed, the virtual nodes are added, so that when the join between Dag is performed, the previous Dag only has one node joined with the first node of the next Dag. Such as Dag2-a1, Dag2-a2, Dag2-A3, Dag3 rely on virtual nodes rather than relying on Dag1-B1, Dag1-B2, Dag1-B3, respectively. And multiple layers Dag are combined into a single Dag to facilitate access to third party process engines. Multiple layers Dag compose a single Dag that only needs to be instantiated once, as some Dag configuration exceptions may be discovered in advance of instantiation.
Wherein, the step S20 specifically includes:
when a target DAG in the serial DAG is executed, determining a previous DAG corresponding to the target DAG according to the DAG dependency relationship and the node dependency relationship;
and when the last DAG corresponding to the target DAG is a multi-task DAG, determining whether the ending identifier exists behind the ending node of the last DAG corresponding to the target DAG so as to judge whether all nodes of the last DAG corresponding to the target DAG are scheduled and executed.
In this embodiment, by adding the end placeholder, the execution state of the node of the last DAG can be determined only by judging one end placeholder, and the query efficiency of the execution state of the node of the last DAG is improved. Namely, when a target DAG in the serial DAG is executed, the last DAG corresponding to the target DAG is determined according to the DAG dependency relationship and the node dependency relationship. If the last DAG is a single-task DGA, that is, the end node is a single node, whether the execution state of the single node is finished or not can be detected. And if the last DAG is the multi-task DGA, detecting whether the end mark exists after the end node of the last DAG, and if the end mark exists, indicating that all the nodes of the last DAG are completely executed.
In this embodiment, because multiple layers Dag compose a single Dag, only one instantiation is needed, and some Dag configuration exceptions, for example, can be discovered in advance of instantiation. The embodiment supports the configuration of multiple Dag scenes without intervening the configuration, and converts the multiple Dag configuration configured by the user into a single Dag instance through a flattening or dimension reduction method. In a scenario where multiple nodes depend on multiple nodes, the embodiment ensures that multiple nodes depend on a single node by adding virtual nodes, and the single node depends on multiple nodes, i.e., one-to-many, thereby simplifying the dependency relationship, and thus reducing the probability of instantiation errors.
Further, the method for merging and scheduling nodes further includes:
judging whether two connected end marks exist in the serial DAG;
and if two connected end identifiers exist in the serial DAG, deleting an end identifier in the two connected end placeholders.
In this embodiment, the end identifier after the end node is used to identify that the DAG is completely executed by the node, so that only one end identifier is needed for one DAG. And deleting an end identifier in the two connected end placeholders if two connected end identifiers exist in the serial DGA, namely one end identifier is an invalid identifier.
Further, the method for merging and scheduling nodes further includes:
judging whether the parallel end nodes in the multi-task DAG correspond to two or more end identifiers;
and if the parallel end node in the multi-task DAG corresponds to two or more end identifiers, combining the two or more end identifiers corresponding to the parallel end node into an end identifier.
In this embodiment, the end identifier after the end node is used to identify that the DAG of the node is completely executed, so that the corresponding end identifier can be added only after all the parallel end nodes in the multi-task DAG are completely executed. Therefore, if two or more ending identifiers corresponding to the parallel ending node in the multi-task DAG are detected, the two or more ending identifiers corresponding to the parallel ending node are merged into an ending identifier, that is, the ending identifier corresponding to the multi-task DAG is used.
The present invention also provides a node combining and scheduling apparatus, which includes:
a DAG serial merging module, configured to obtain a DAG dependency relationship corresponding to each directed acyclic graph DAG and a node dependency relationship corresponding to each node in the DAG, and merge and generate a corresponding serial DAG according to the DAG dependency relationship and the node dependency relationship;
a DAG execution judgment module, configured to, when executing a target DAG in the serial DAG, judge whether all nodes of a previous DAG corresponding to the target DAG are scheduled and executed according to the DAG dependency relationship and the node dependency relationship;
and the DAG execution scheduling module is used for scheduling and executing the target DAG to schedule and execute all DAG nodes in the serial DAG if all the nodes of the last DAG corresponding to the target DAG are scheduled and executed.
Further, the merge scheduling apparatus of the node further includes an end identifier adding module, where the end identifier adding module is configured to:
when the fact that the multitask DAG with the end node being the parallel node exists in the serial DAG is detected, after the end node of the multitask DAG is scheduled and executed, an end mark is set behind the end node of the multitask DAG to represent that all nodes of the multitask DAG are scheduled and executed completely.
Further, the end identifier includes an end placeholder or a virtual node.
Further, the DAG execution determination module specifically includes:
an upper DAG determining unit, configured to determine, when a target DAG in the serial DAG is executed, an upper DAG corresponding to the target DAG according to the DAG dependency and the node dependency;
and an end identifier determining unit, configured to determine whether the end identifier exists after an end node of the previous DAG corresponding to the target DAG when the previous DAG corresponding to the target DAG is the multitask DAG, so as to determine whether all nodes of the previous DAG corresponding to the target DAG are scheduled and executed.
Further, the merge scheduling device of the node further includes a connected placeholder deleting module, where the connected placeholder deleting module is configured to:
judging whether two connected end marks exist in the serial DAG;
and if two connected end identifiers exist in the serial DAG, deleting an end identifier in the two connected end placeholders.
Further, the merge scheduler of the node further includes a parallel placeholder merge module, and the parallel placeholder merge module is configured to:
judging whether the parallel end nodes in the multi-task DAG correspond to two or more end identifiers;
and if the parallel end node in the multi-task DAG corresponds to two or more end identifiers, combining the two or more end identifiers corresponding to the parallel end node into an end identifier.
The method executed by each program module can refer to each embodiment of the node merge scheduling method of the present invention, and is not described herein again.
The invention also provides a computer readable storage medium.
The computer readable storage medium of the present invention stores thereon a merge scheduler for a node, which when executed by a processor implements the steps of the merge scheduling method for a node as described above.
The method implemented when the merged scheduler of the node running on the processor is executed may refer to each embodiment of the merged scheduling method of the node of the present invention, and is not described herein again.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A method for merging and scheduling nodes is characterized in that the method for merging and scheduling the nodes comprises the following steps:
acquiring DAG dependency corresponding to each directed acyclic graph DAG and node dependency corresponding to each node in the DAG, and combining and generating corresponding serial DAG according to the DAG dependency and the node dependency;
when a target DAG in the serial DAG is executed, judging whether all nodes of a previous DAG corresponding to the target DAG are scheduled and executed according to the DAG dependency relationship and the node dependency relationship;
and if the scheduling execution of all nodes of the last DAG corresponding to the target DAG is completed, scheduling and executing the target DAG so as to schedule and execute all nodes of the DAG in the serial DAG.
2. The method for merging and scheduling nodes according to claim 1, wherein after the step of obtaining the DAG dependency corresponding to each directed acyclic graph DAG and the node dependency corresponding to each node in the DAG and merging and generating the corresponding serial DAG according to the DAG dependency and the node dependency, the method further comprises:
when the fact that the multitask DAG with the end node being the parallel node exists in the serial DAG is detected, after the end node of the multitask DAG is scheduled and executed, an end mark is set behind the end node of the multitask DAG to represent that all nodes of the multitask DAG are scheduled and executed completely.
3. The method of merged scheduling of nodes of claim 2, wherein the end identifier comprises an end placeholder or a virtual node.
4. The method for merged scheduling of nodes according to claim 2, wherein, when executing a target DAG in the serial DAG, the step of determining whether all nodes of a previous DAG corresponding to the target DAG are scheduled to be executed completely according to the DAG dependency relationship and the node dependency relationship specifically includes:
when a target DAG in the serial DAG is executed, determining a previous DAG corresponding to the target DAG according to the DAG dependency relationship and the node dependency relationship;
and when the last DAG corresponding to the target DAG is a multi-task DAG, determining whether the ending identifier exists behind the ending node of the last DAG corresponding to the target DAG so as to judge whether all nodes of the last DAG corresponding to the target DAG are scheduled and executed.
5. The merged scheduling method of a node of claim 2, wherein the merged scheduling method of a node further comprises:
judging whether two connected end marks exist in the serial DAG;
and if two connected end identifiers exist in the serial DAG, deleting an end identifier in the two connected end placeholders.
6. The merged scheduling method of a node according to any one of claims 2 to 5, wherein the merged scheduling method of a node further comprises:
judging whether the parallel end nodes in the multi-task DAG correspond to two or more end identifiers;
and if the parallel end node in the multi-task DAG corresponds to two or more end identifiers, combining the two or more end identifiers corresponding to the parallel end node into an end identifier.
7. A merge scheduler for a node, the merge scheduler for the node comprising:
a DAG serial merging module, configured to obtain a DAG dependency relationship corresponding to each directed acyclic graph DAG and a node dependency relationship corresponding to each node in the DAG, and merge and generate a corresponding serial DAG according to the DAG dependency relationship and the node dependency relationship;
a DAG execution judgment module, configured to, when executing a target DAG in the serial DAG, judge whether all nodes of a previous DAG corresponding to the target DAG are scheduled and executed according to the DAG dependency relationship and the node dependency relationship;
and the DAG execution scheduling module is used for scheduling and executing the target DAG to schedule and execute all DAG nodes in the serial DAG if all the nodes of the last DAG corresponding to the target DAG are scheduled and executed.
8. The merge scheduler of a node as claimed in claim 7, wherein the merge scheduler of a node further comprises an end identifier adding module, the end identifier adding module is configured to:
when the fact that the multitask DAG with the end node being the parallel node exists in the serial DAG is detected, after the end node of the multitask DAG is scheduled and executed, an end mark is set behind the end node of the multitask DAG to represent that all nodes of the multitask DAG are scheduled and executed completely.
9. A merge scheduling apparatus of a node, comprising: memory, a processor and a merged scheduler of nodes stored on the memory and being executable on the processor, the merged scheduler of nodes, when executed by the processor, implementing the steps of the merged scheduling method of nodes according to any of claims 1 to 6.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a merge scheduler of nodes, which when executed by a processor implements the steps of the merge scheduling method of nodes according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911249012.7A CN111061551A (en) | 2019-12-06 | 2019-12-06 | Node merging and scheduling method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911249012.7A CN111061551A (en) | 2019-12-06 | 2019-12-06 | Node merging and scheduling method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111061551A true CN111061551A (en) | 2020-04-24 |
Family
ID=70300043
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911249012.7A Pending CN111061551A (en) | 2019-12-06 | 2019-12-06 | Node merging and scheduling method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111061551A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112463140A (en) * | 2020-11-23 | 2021-03-09 | 乐聚(深圳)机器人技术有限公司 | Programming method and device based on electronic building blocks, electronic equipment and storage medium |
| CN112732979A (en) * | 2020-12-29 | 2021-04-30 | 五八有限公司 | Information writing method, information writing device, electronic equipment and computer readable medium |
| CN112764907A (en) * | 2021-01-26 | 2021-05-07 | 网易（杭州）网络有限公司 | Task processing method and device, electronic equipment and storage medium |
| CN113052707A (en) * | 2021-04-30 | 2021-06-29 | 中国工商银行股份有限公司 | Application production method and device, computer equipment and storage medium |
| CN113326402A (en) * | 2021-06-16 | 2021-08-31 | 上海哔哩哔哩科技有限公司 | Directed acyclic graph generation method and system |
| CN114387150A (en) * | 2020-10-20 | 2022-04-22 | 富泰华工业（深圳）有限公司 | Processing method and device for artificial intelligence learning, equipment and storage medium |
| CN114462899A (en) * | 2022-04-13 | 2022-05-10 | 云智慧（北京）科技有限公司 | Service active node merging method, device and equipment |
-
2019
- 2019-12-06 CN CN201911249012.7A patent/CN111061551A/en active Pending
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114387150A (en) * | 2020-10-20 | 2022-04-22 | 富泰华工业（深圳）有限公司 | Processing method and device for artificial intelligence learning, equipment and storage medium |
| CN112463140A (en) * | 2020-11-23 | 2021-03-09 | 乐聚(深圳)机器人技术有限公司 | Programming method and device based on electronic building blocks, electronic equipment and storage medium |
| CN112463140B (en) * | 2020-11-23 | 2023-09-12 | 乐聚(深圳)机器人技术有限公司 | Programming method and device based on electronic building blocks, electronic equipment and storage medium |
| CN112732979A (en) * | 2020-12-29 | 2021-04-30 | 五八有限公司 | Information writing method, information writing device, electronic equipment and computer readable medium |
| CN112732979B (en) * | 2020-12-29 | 2023-02-17 | 五八有限公司 | Information writing method, information writing device, electronic equipment and computer readable medium |
| CN112764907A (en) * | 2021-01-26 | 2021-05-07 | 网易（杭州）网络有限公司 | Task processing method and device, electronic equipment and storage medium |
| CN112764907B (en) * | 2021-01-26 | 2024-05-10 | 网易（杭州）网络有限公司 | Task processing method and device, electronic equipment and storage medium |
| CN113052707A (en) * | 2021-04-30 | 2021-06-29 | 中国工商银行股份有限公司 | Application production method and device, computer equipment and storage medium |
| CN113326402A (en) * | 2021-06-16 | 2021-08-31 | 上海哔哩哔哩科技有限公司 | Directed acyclic graph generation method and system |
| CN113326402B (en) * | 2021-06-16 | 2022-07-19 | 上海哔哩哔哩科技有限公司 | Directed acyclic graph generation method and system |
| CN114462899A (en) * | 2022-04-13 | 2022-05-10 | 云智慧（北京）科技有限公司 | Service active node merging method, device and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111061551A (en) | Node merging and scheduling method, device, equipment and storage medium | |
| US20050182749A1 (en) | Process editing apparatus and method and process management apparatus and method | |
| CN112051993B (en) | Method, device, medium and equipment for generating state machine template and processing task | |
| US20150286615A1 (en) | Workflow schedule authoring tool | |
| CA3161753A1 (en) | Blockchain-smart-contract debugging and releasing method and system thereof | |
| CN108255585B (en) | SDK abnormal control and application program operation method, device and equipment thereof | |
| CN110333940A (en) | Method for scheduling task, device, equipment and storage medium based on condition | |
| CN108256715B (en) | Workflow management method and system | |
| WO2019149032A1 (en) | Distributed transaction processing method and device | |
| CN102572896A (en) | Upgrading method and upgrading device for wireless communication system | |
| CN109408216A (en) | Task creating method, device, equipment and storage medium | |
| CN113961332A (en) | Method and device for realizing workflow engine, electronic equipment and storage medium | |
| CN109445929A (en) | A kind of method and system of scheduler task | |
| CN110543429B (en) | Test case debugging method, device and storage medium | |
| CN115860451A (en) | Flow operation method and device, electronic equipment and storage medium | |
| JP2008112311A (en) | Method and system for business process execution, and program | |
| CN107844363B (en) | Business transaction processing method, device, storage medium and equipment | |
| Petriu et al. | Software performance models from system scenarios | |
| CN112367205B (en) | Processing method and scheduling system for HTTP scheduling request | |
| CN114327673A (en) | Task starting method and device, electronic equipment and storage medium | |
| CN111124386B (en) | Animation event processing method, device, equipment and storage medium based on Unity | |
| KR101732922B1 (en) | GUI-based asynchronous type coding system and method | |
| CN112905270A (en) | Workflow implementation method, device, platform, electronic equipment and storage medium | |
| CN111160403A (en) | Method and device for multiplexing and discovering API (application program interface) | |
| CN112433821B (en) | Method and device for building business model, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |