CN113656468B - Task flow triggering method and device based on NIFI - Google Patents
Task flow triggering method and device based on NIFI Download PDFInfo
- Publication number
- CN113656468B CN113656468B CN202010397799.8A CN202010397799A CN113656468B CN 113656468 B CN113656468 B CN 113656468B CN 202010397799 A CN202010397799 A CN 202010397799A CN 113656468 B CN113656468 B CN 113656468B
- Authority
- CN
- China
- Prior art keywords
- flow
- task
- processor
- execution
- startpoint
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 230000001419 dependent effect Effects 0.000 claims abstract description 9
- 230000002159 abnormal effect Effects 0.000 claims description 20
- 230000008569 process Effects 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 6
- 230000001960 triggered effect Effects 0.000 abstract description 26
- 238000010586 diagram Methods 0.000 description 18
- 238000012545 processing Methods 0.000 description 13
- 238000004891 communication Methods 0.000 description 6
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 238000013480 data collection Methods 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000003999 initiator Substances 0.000 description 1
- 238000013515 script Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the invention provides a task flow triggering method and device based on NIFI, wherein the method comprises the following steps: the second StartPoint Processor configured in the second Flow task is controlled to periodically poll, judgment of the execution record of the first Flow task is triggered, whether the execution instance information of the first Flow task meets the first trigger condition is judged according to the task execution instance information recorded in the first EndPoint Processor and the first trigger condition defined by the first StartPoint Processor, if yes, the subsequent Processor execution task is triggered, and if not, the subsequent Processor execution task is not triggered, and according to the logic relation between the NIFI dependent data acquisition summary tasks, the execution of the trigger task is realized, and the execution task efficiency is improved.
Description
Technical Field
The invention relates to the technical field of communication, in particular to a task flow triggering method and device based on NIFI.
Background
Along with the rapid development of the internet industry, data acquisition has been widely applied to various fields, traditional data acquisition and summarization tasks are all based on time-triggered scripts, or a set of data acquisition and summarization programs are developed to acquire and summarize data, the data acquisition and summarization tasks of the method are performed according to a set time sequence, namely, tasks can be executed only when a set time is reached, but the problem is that task execution logic is fixed, so that when the execution of the prior task fails or has a problem, subsequent tasks can be executed anyway, thereby causing resource waste and lower task execution efficiency.
NIFI is a stable ETL tool, and can realize the operations of data collection, summarization, loading, conversion and the like. However, there is no task link between the NIFI tasks, and no event trigger can be formed, so that each task is independent, the tasks and the tasks cannot form an effective series connection, and the trigger condition of the NIFI tasks is only time trigger.
Therefore, how to propose a method to implement triggering task execution based on NIFI and relying on logical relationships between data collection and summary tasks is a problem to be solved.
Disclosure of Invention
Aiming at the defects in the prior art, the embodiment of the invention provides a task flow triggering method and device based on NIFI.
In a first aspect, an embodiment of the present invention provides a method for triggering a task flow based on NIFI, including:
configuring a first StartPoint Processor in the first Flow task, setting a first triggering condition through a first StartPoint Processor, configuring a first EndPoint Processor in the first Flow task, and recording each execution instance information of the first Flow task through a first EndPoint Processor;
accordingly, a second StartPoint Processor is configured in the second Flow task, a second trigger condition is set through a second StartPoint Processor, and meanwhile, a second EndPoint Processor is configured in the second Flow task, and each execution instance information of the second Flow task is recorded through a second EndPoint Processor;
The first triggering condition is a condition that the first Flow task triggers the execution of a second Flow task, and the second triggering condition is a condition that the second Flow task triggers the execution of any task except the second Flow task;
controlling the second StartPoint Processor to periodically poll, triggering the judgment of the execution record of the first Flow task, and acquiring the execution instance information of the first Flow task and a first triggering condition;
according to the first Flow task execution instance information and the first trigger condition, if the second StartPoint Processor judges that the first Flow task execution instance information meets the first trigger condition, the second StartPoint Processor triggers the subsequent Processor to execute the task; if the first Flow task execution instance information is judged not to meet the first trigger condition, the second StartPoint Processor does not trigger the subsequent Processor to execute the task.
Preferably, the first Flow task starts with a first StartPoint Processor, and designates a cluster node where the first Flow task first executes an instance; all normal conditions and all abnormal conditions in the execution process of the first Flow task are separately processed and finally Flow to the first EndPoint Processor;
correspondingly, the second Flow task starts with a second StartPoint Processor, and the cluster node where the first execution instance of the second Flow task is located is designated; all normal conditions and all abnormal conditions in the second Flow task execution are handled separately and eventually Flow to the second EndPoint Processor.
Preferably, the configuration is realized by dragging the Web page through the first triggering condition and the second triggering condition, and the configuration specifically comprises:
the connection configuration of the dependent triggering conditions among the Flow tasks;
trigger condition configuration of a single Flow task;
logic trigger condition configuration among a plurality of Flow tasks.
Preferably, the first StartPoint Processor specifies an execution granularity of a first Flow task; the second StartPoint Processor specifies an execution granularity of a second Flow task; if the first Flow task is of an hour granularity and the second Flow task is of a day granularity, the first trigger condition is that the first Flow task can trigger the second Flow task to execute after the first Flow task executes 24 tasks within the time range where the second Flow task is completed.
Preferably, the first trigger condition may also trigger the execution of a second Flow task for any or each successful or failed one or more times of the first Flow task execution;
the execution granularity may further include: 1 minute, 5 minutes, 15 minutes, 30 minutes, 1 hour, 1 day, 1 month, one quarter, and one year.
In a second aspect, an embodiment of the present invention provides a task flow triggering device based on NIFI, including:
A configuration unit, configured to configure the first StartPoint Processor in the first Flow task, set a first trigger condition through the first StartPoint Processor, and simultaneously configure the first EndPoint Processor in the first Flow task, and record each execution instance information of the first Flow task through the first EndPoint Processor;
accordingly, a second StartPoint Processor is configured in the second Flow task, a second trigger condition is set through a second StartPoint Processor, and meanwhile, a second EndPoint Processor is configured in the second Flow task, and each execution instance information of the second Flow task is recorded through a second EndPoint Processor;
the first triggering condition is a condition that the first Flow task triggers the execution of a second Flow task, and the second triggering condition is a condition that the second Flow task triggers the execution of any task except the second Flow task;
the control unit is used for controlling the second StartPoint Processor to periodically poll, triggering the judgment of the execution record of the first Flow task and acquiring the execution instance information of the first Flow task and the first triggering condition;
the triggering unit is configured to trigger, according to the first Flow task execution instance information and the first triggering condition, the second StartPoint Processor to trigger the subsequent Processor to execute the task if the second StartPoint Processor determines that the first Flow task execution instance information meets the first triggering condition; if the first Flow task execution instance information is judged not to meet the first trigger condition, the second StartPoint Processor does not trigger the subsequent Processor to execute the task.
Preferably, the first Flow task starts with a first StartPoint Processor, and designates a cluster node where the first Flow task first executes an instance; all normal conditions and all abnormal conditions in the execution process of the first Flow task are separately processed and finally Flow to the first EndPoint Processor;
correspondingly, the second Flow task starts with a second StartPoint Processor, and the cluster node where the first execution instance of the second Flow task is located is designated; all normal conditions and all abnormal conditions in the second Flow task execution are handled separately and eventually Flow to the second EndPoint Processor.
Preferably, the configuration is realized by dragging the Web page through the first triggering condition and the second triggering condition, and the configuration specifically comprises:
the connection configuration of the dependent triggering conditions among the Flow tasks;
trigger condition configuration of a single Flow task;
logic trigger condition configuration among a plurality of Flow tasks.
In a third aspect, an embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the steps of the NIFI-based task flow triggering method of the first aspect as described above when the program is executed.
In a fourth aspect, embodiments of the present invention provide a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the NIFI-based task flow triggering method of the first aspect as described above.
According to the task Flow triggering method and device based on the NIFI, the second StartPoint Processor configured in the second Flow task is controlled to periodically poll, judgment of the execution record of the first Flow task is triggered, whether the execution instance information of the first Flow task meets the first triggering condition or not is judged by the second StartPoint Processor according to the task execution instance information recorded by the first EndPoint Processor on the first Flow task and the first triggering condition defined by the first StartPoint Processor, the subsequent Processor execution task is triggered after the first triggering condition is judged to be met, if the first triggering condition is not met, the subsequent Processor execution task is not triggered, and therefore the execution of the trigger task is realized based on the NIFI according to the logic relation between the data acquisition summary tasks, the application range of the NIFI is wider, and the task execution efficiency is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow diagram of a task flow triggering method based on NIFI in an embodiment of the invention;
FIG. 2 is a Flow diagram of NIFI-based Flow in an embodiment of the invention;
FIG. 3 is a schematic diagram of a StartPoint Processor schedule period configuration page according to an embodiment of the present invention;
FIG. 4 is a Flow diagram of a Flow A trigger Flow B based on NIFI in an embodiment of the invention;
FIG. 5 is a Flow diagram of a Flow A & Flow B trigger Flow C based on NIFI in an embodiment of the invention;
FIG. 6 is a Flow diagram of a NIFI-based Flow A I Flow B trigger Flow C in an embodiment of the invention;
FIG. 7 is a Flow trigger condition configuration page diagram in an embodiment of the present invention;
FIG. 8 is a schematic diagram of a configuration of a StartPoint Processor designated Processor running poll initiator node in an embodiment of the invention;
FIG. 9 is a diagram of a StartPoint Processor defined Flow task granularity configuration in an embodiment of the invention;
fig. 10 is a schematic structural diagram of a task flow triggering device based on NIFI in an embodiment of the present invention;
fig. 11 is a schematic diagram of an entity structure of an electronic device according to an embodiment of the invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In order to facilitate understanding of the technical solution and the principle of the embodiments of the present invention, the related terms appearing in the embodiments of the present invention are explained as follows:
processor: each processing unit is called a Processor, also called a microprocessor, as an independent processing unit.
The NIFI cluster operation mode refers to the Processor running on all nodes of the cluster.
Flow: one task in NIFI is called Flow, which is composed of a plurality of different processors through interconnection.
Stream: the dependency trigger relationship between flows configured in NIFI is called Stream.
Fig. 1 is a flow diagram of a task flow triggering method based on NIFI in an embodiment of the present invention, as shown in fig. 1, where the task flow triggering method based on NIFI provided in the embodiment of the present invention includes:
step 110, configuring a first StartPoint Processor in a first Flow task, setting a first trigger condition through a first StartPoint Processor, configuring a first EndPoint Processor in the first Flow task, and recording information of each execution instance of the first Flow task through a first EndPoint Processor;
accordingly, a second StartPoint Processor is configured in the second Flow task, a second trigger condition is set through a second StartPoint Processor, and meanwhile, a second EndPoint Processor is configured in the second Flow task, and each execution instance information of the second Flow task is recorded through a second EndPoint Processor;
the first triggering condition is a condition that the first Flow task triggers the execution of a second Flow task, and the second triggering condition is a condition that the second Flow task triggers the execution of any task except the second Flow task.
Specifically, fig. 2 is a schematic Flow chart based on NIFI in the embodiment of the present invention, where a first Flow task is formed by connecting a plurality of different processors, and each Processor is used as an independent processing unit to complete data processing. Two specific Processor processors, first StartPoint Processor and first EndPoint Processor, are configured in the first Flow task. The first StartPoint Processor sets a first trigger condition, i.e., a condition that the first Flow task triggers the execution of the second Flow task, i.e., a dependency trigger relationship between the first Flow task and the second Flow task. The first EndPoint Processor is responsible for recording each execution instance information of the first Flow task, including each execution success and execution failure information, and each execution record one piece of information, including mainly the name of the task, execution start time, execution end time, execution state, and the like. In the prior art, the open source NIFI is not used for recording the execution instance of the task, so that the situation of each execution instance cannot be known, namely, how many times the task is executed in one day and the execution time of each time is consumed, therefore, the embodiment of the invention can realize the recording of the task execution information by adding the specific first StartPoint Processor and the specific first EndPoint Processor, can analyze the time consumption time of each task, thereby optimizing the task, and can inform operation and maintenance personnel of processing the task in time without warning, thereby guaranteeing the timeliness and accuracy of data.
Correspondingly, the second Flow task is formed by connecting a plurality of different processors, and each Processor is used as an independent processing unit to finish data processing. Two specific Processor processors, second StartPoint Processor and second EndPoint Processor, respectively, are configured in the second Flow task. The second StartPoint Processor sets a second trigger condition, that is, a condition for executing any task other than the second Flow task, that is, a dependency trigger relationship between the second Flow task and other Flow tasks. The second EndPoint Processor is responsible for recording each execution instance information of the second Flow task, including each execution success and execution failure, and each execution record one piece of information, mainly including the name of the task, execution start time, execution end time, execution state, and the like.
And 120, controlling the second StartPoint Processor to periodically poll, triggering the judgment of the first Flow task execution record, and acquiring the first Flow task execution instance information and the first triggering condition.
Specifically, fig. 3 is a schematic diagram of a StartPoint Processor scheduling period configuration page in an embodiment of the present invention, where the second StartPoint Processor may implement polling switching between different nodes of an NIFI cluster by performing periodic polling setting on an execution instance of a second Flow task according to an interface shown in fig. 3 through the second StartPoint Processor, and sequentially inquire whether the nodes on the NIFI cluster need to execute the execution instance, so that each execution instance of the NIFI task may be executed on different execution nodes in the same-scale NIFI cluster, and further may configure more NIFI tasks to implement load balancing with a greater degree of task instance granularity.
In the NIFI cluster operation mode, when one Processor fails, the second StartPoint Processor does not poll the node any more, so that each instance of the task can be guaranteed to operate normally as expected. Meanwhile, due to the existence of a trigger mechanism, if the task execution record is not processed, the running task is still triggered when the judgment logic is run again, so that the triggered task is ensured not to be missed.
Fig. 4 is a schematic Flow chart of a Flow a trigger Flow B based on NIFI in which Flow a and Flow B exist in the embodiment of the present invention, and Flow a triggers Flow B to execute. The StartPoint Processor of the Flow B task periodically polls, triggers a judgment of the execution record of the Flow a when the Flow B runs, and the judgment of the execution record of the Flow a is an incremental judgment. The Flow B finds the dependency relationship of the Flow B according to the ID of the Flow B, that is, the condition for triggering the execution of the Flow B, and further finds the ID of the Flow a, and obtains the execution information of the task recorded on the Flow a EndPoint Processor through the ID of the Flow a, and at the same time obtains the triggering condition of StartPoint Processor on the Flow a.
Step 130, according to the first Flow task execution instance information and the first trigger condition, if the second StartPoint Processor determines that the first Flow task execution instance information meets the first trigger condition, the second StartPoint Processor triggers the subsequent Processor to execute the task; if the first Flow task execution instance information is judged not to meet the first trigger condition, the second StartPoint Processor does not trigger the subsequent Processor to execute the task.
Specifically, there are Flow a and Flow B in the NIFI, and Flow a triggers Flow B execution. According to the acquired execution instance information of the Flow A and the condition triggering the execution of the Flow B, startPoint Processor of the Flow B judges that StartPoint Processor of the Flow B triggers the subsequent Processor to execute if the execution record of the Flow A meets the execution condition, otherwise StartPoint Processor of the Flow B does not trigger the subsequent Processor to execute, thereby saving system resources. The method provided by the embodiment of the invention realizes the realization of the triggering task execution by relying on the logic relation between the data acquisition and summarization tasks, meets the arrangement requirement of the task execution sequence among the tasks, and makes up the functional deficiency of the NIFI in the aspect of task dependency triggering, so that the application range of the NIFI is wider, and the requirements of various industries on production scenes such as data acquisition, summarization and the like can be met.
It should be noted that, in the embodiment of the present invention, not only the Flow a triggering Flow B may be implemented, but also the corresponding triggering conditions may be set in StartPoint Processor in each Flow according to specific situations, so as to implement complex dependency relationships such as Flow A And Flow B triggering Flow C and Flow a Or Flow B triggering Flow C.
For example, with Flow a, flow B, and Flow C, the following trigger conditions may be implemented: 1) The Flow A triggers the Flow B; 2) The Flow A triggers the Flow B, and the Flow B triggers the Flow C; 3) The Flow A & Flow B triggers the Flow C; 4) Flow a||flow B triggers Flow C. Wherein the Flow A and the Flow B have respective corresponding trigger conditions.
Fig. 5 is a schematic Flow chart of a Flow a & Flow B trigger Flow C based on NIFI in an embodiment of the present invention, as shown in fig. 5, the Flow a and the Flow B trigger the Flow C simultaneously, that is, the Flow a meets the trigger condition configured by itself and the Flow B meets the trigger condition configured by itself, but triggers the Flow C to execute. Fig. 6 is a schematic Flow chart of a Flow a/B trigger Flow C based on NIFI in the embodiment of the present invention, as shown in fig. 6, the Flow a or the Flow B trigger Flow C, that is, the Flow a meets the trigger condition of self-configuration or the Flow B triggers the Flow C to execute when meeting the trigger condition of self-configuration. Fig. 7 is a page diagram of a Flow trigger condition configuration in an embodiment of the present invention, where, as shown in fig. 7, the trigger condition of the Flow a configuration is: at least one execution of Flow a succeeds at any time.
According to the task Flow triggering method based on the NIFI, the second StartPoint Processor configured in the second Flow task is controlled to periodically poll, judgment of the execution record of the first Flow task is triggered, whether the execution instance information of the first Flow task meets the first triggering condition or not is judged by the second StartPoint Processor according to the task execution instance information recorded by the first EndPoint Processor on the first Flow task and the first triggering condition defined by the first StartPoint Processor, the subsequent Processor execution task is triggered after the first triggering condition is judged to be met, if the first triggering condition is not met, the subsequent Processor execution task is not triggered, and therefore the execution of the trigger task is realized based on the NIFI according to the logic relation between the data acquisition summary tasks, the application range of the NIFI is wider, and the task execution efficiency is improved.
Based on the content of the foregoing embodiment, as an alternative embodiment, the first Flow task starts with the first StartPoint Processor, and designates a cluster node where the first Flow task first executes an instance; all normal conditions and all abnormal conditions in the execution process of the first Flow task are separately processed and finally Flow to the first EndPoint Processor;
correspondingly, the second Flow task starts with a second StartPoint Processor, and the cluster node where the first execution instance of the second Flow task is located is designated; all normal conditions and all abnormal conditions in the second Flow task execution are handled separately and eventually Flow to the second EndPoint Processor.
Specifically, fig. 8 is a schematic diagram illustrating configuration of a designated Processor running polling start node in StartPoint Processor in an embodiment of the present invention, where a first Flow task starts with a first StartPoint Processor, so that polling is performed on nodes on an NIFI cluster, thereby designating a cluster node where a first execution instance of the first Flow task is located, and starting execution instance. In the first Flow task execution process, two conditions of success and failure exist, namely normal conditions and abnormal conditions are respectively corresponding, the two conditions are separately processed and finally Flow to the first EndPoint Processor, information of successful and failed execution examples is recorded, when a task record list has failed record information, the failed record can be re-run on an interface, the failed task can be re-run, the trigger condition is not relied on, and the follow-up task execution can be continuously triggered after the task execution is successful.
For example, in the scenario where an acquisition task triggers a summary task, when the acquisition data is missing, conventional processing logic is to re-acquire and re-perform all related summary tasks that follow. However, in the embodiment of the invention, the re-execution of all subsequent trigger tasks can be realized only by re-acquiring data.
Accordingly, the second Flow task starts with the second StartPoint Processor, so that the nodes on the NIFI cluster can be conveniently polled, and thus the cluster node where the first execution instance of the second Flow task is located is designated, and the execution instance starts. In the execution process of the second Flow task, two conditions of success and failure exist, namely normal conditions and abnormal conditions are respectively corresponding, the two conditions are separately processed and finally Flow to the second EndPoint Processor, information of successful and failed execution examples is recorded, when a task record list has failed record information, the failed record can be re-run on an interface, the failed task can be re-run, the trigger condition is not relied on, and the subsequent task execution can be continuously triggered after the task execution is successful.
According to the task Flow triggering method based on the NIFI, all normal conditions and all abnormal conditions in the execution process of the Flow task are processed separately and finally Flow to EndPoint Processor by starting the Flow task at StartPoint Processor, each execution instance can be processed separately, so that automatic task re-running triggering and other abnormal conditions are processed, operation is simple, and task execution efficiency is improved.
Based on the foregoing embodiment, as an optional embodiment, the configuring of the first trigger condition and the second trigger condition by drag of the Web page specifically includes:
the connection configuration of the dependent triggering conditions among the Flow tasks; trigger condition configuration of a single Flow task; logic trigger condition configuration among a plurality of Flow tasks.
Specifically, the first trigger condition is that configuration information is stored in a database through drag of a Web page, background configuration is not needed, a foreground configuration mode is completely realized, a Flow A trigger Flow B can be realized through connection configuration of the Web page, namely, the configuration of dependent trigger conditions among Flow tasks, the trigger conditions of the Flow A can be configured through the Web page, namely, the trigger conditions of a single Flow task, and a Flow A & Flow B trigger Flow C can be configured through the Web page, namely, the logic trigger conditions among a plurality of Flow tasks. The trigger condition configuration is flexible and simple on the web page in a dragging mode, the modification and logic change of the dependency relationship of subsequent personnel are very convenient, and the safety and maintainability of the system are improved.
Similarly, the second trigger condition can be configured by dragging the Web page in the same manner as described above.
According to the task flow triggering method based on the NIFI, the triggering conditions can be configured through dragging of the Web page, the method is flexible and simple, modification and logic change of the dependency relationship of subsequent personnel are very convenient, and the safety and maintainability of the system are improved.
Based on the content of the above embodiment, as an alternative embodiment, the first StartPoint Processor specifies the execution granularity of the first Flow task; the second StartPoint Processor specifies an execution granularity of a second Flow task; if the first Flow task is of an hour granularity and the second Flow task is of a day granularity, the first trigger condition is that the first Flow task can trigger the second Flow task to execute after the first Flow task executes 24 tasks within the time range where the second Flow task is completed.
Specifically, the first StartPoint Processor may define an execution granularity of the first Flow task, which represents an execution time interval of the task, so that a determination of triggering between tasks may be made. Similarly, the second StartPoint Processor can also define the execution granularity of the second Flow task. If the first Flow task is of an hour granularity, that is, the task execution time interval is 1 hour, and the second Flow task is of a day granularity, that is, the task execution time interval is 1 day, the first Flow task can trigger the second Flow task to execute after the first Flow task executes 24 tasks within the time range where the second Flow task is located, that is, the first trigger condition. For example, flow a is an hour granularity task, flow B is a day granularity task, flow a triggers Flow B, there may be two cases: 1) And if the Flow A executes the task at the 8 points, recording that the execution time of the Flow A is '8:00:00-8:59:59', and the triggered Flow B task is the date on which the Flow A task is executed, such as '2020-02-29:00:00:00-2020-02-29:23:59:59'. I.e., each time Flow a is executed, flow B execution is triggered. 2) The execution of the Flow A is successful at each moment, and the Flow B task is triggered. That is, execution of tasks with granularity of 24 hours in one day by Flow a successfully triggers Flow B, and execution of Flow B is not triggered if Flow a only executes 23 tasks.
According to the task Flow triggering method based on NIFI, the execution granularity of the tasks is defined through StartPoint Processor configured on the Flow, so that task triggering judgment is performed, the task execution is triggered by relying on the logical relationship among the data acquisition summary tasks, and the task execution efficiency is improved.
Based on the foregoing embodiments, as an optional embodiment, the first triggering condition may further trigger the execution of the second Flow task for any or each success or failure of the first Flow task;
the execution granularity may further include: 1 minute, 5 minutes, 15 minutes, 30 minutes, 1 hour, 1 day, 1 month, one quarter, and one year.
Specifically, the condition that the first Flow task triggers the second Flow task may also be: any or each success or failure of the first Flow task execution may trigger the second Flow task execution. The execution granularity of the first Flow task and the second Flow task may be set to any one of 1 minute, 5 minutes, 15 minutes, 30 minutes, 1 hour, 1 day, 1 month, one quarter, and one year, and fig. 9 is a configuration diagram of the granularity of the first Flow task defined by StartPoint Processor in the embodiment of the present invention, and may be set according to the specific situation on the page in fig. 9.
For example, there are Flow a and Flow B, and any time/each time, execution success/execution failure/execution, one time/not less than one time/a specified number of times, and a specified number of times selected when the specified number of times is present, the Flow a task may be used as a trigger condition for triggering execution of Flow B.
According to the task Flow triggering method based on NIFI, task triggering judgment is carried out by defining execution granularity and triggering conditions of tasks through StartPoint Processor configured on Flow, the task execution is realized by relying on logical relations among data acquisition and summarization tasks, and task execution efficiency is improved.
Fig. 10 is a schematic structural diagram of a task flow triggering device based on NIFI in an embodiment of the present invention, as shown in fig. 10, where the task flow triggering device based on NIFI provided in the embodiment of the present invention includes:
a configuration unit 1010, configured to configure a first StartPoint Processor in a first Flow task, set a first trigger condition through a first StartPoint Processor, and simultaneously configure a first EndPoint Processor in the first Flow task, and record per-execution instance information of the first Flow task through a first EndPoint Processor;
Accordingly, a second StartPoint Processor is configured in the second Flow task, a second trigger condition is set through a second StartPoint Processor, and meanwhile, a second EndPoint Processor is configured in the second Flow task, and each execution instance information of the second Flow task is recorded through a second EndPoint Processor;
the first triggering condition is a condition that the first Flow task triggers the execution of a second Flow task, and the second triggering condition is a condition that the second Flow task triggers the execution of any task except the second Flow task.
Specifically, the first Flow task is formed by connecting a plurality of different processors, and each Processor is used as an independent processing unit to finish data processing. The configuration unit 1010 configures two specific Processor processors in the first Flow task, which are first StartPoint Processor and first EndPoint Processor respectively. The first StartPoint Processor sets a first trigger condition, i.e., a condition that the first Flow task triggers the execution of the second Flow task, i.e., a dependency trigger relationship between the first Flow task and the second Flow task. The first EndPoint Processor is responsible for recording each execution instance information of the first Flow task, including each execution success and execution failure information, and each execution record one piece of information, including mainly the name of the task, execution start time, execution end time, execution state, and the like. In the prior art, the open source NIFI is not used for recording the execution instance of the task, so that the situation of each execution instance cannot be known, namely, how many times the task is executed in one day and the execution time of each time is consumed, therefore, the embodiment of the invention can realize the recording of the task execution information by adding the specific first StartPoint Processor and the specific first EndPoint Processor, can analyze the time consumption time of each task, thereby optimizing the task, and can inform operation and maintenance personnel of processing the task in time without warning, thereby guaranteeing the timeliness and accuracy of data.
Correspondingly, the second Flow task is formed by connecting a plurality of different processors, and each Processor is used as an independent processing unit to finish data processing. The configuration unit 1010 configures two specific Processor processors in the second Flow task, which are a second StartPoint Processor and a second EndPoint Processor respectively. The second StartPoint Processor sets a second trigger condition, that is, a condition for executing any task other than the second Flow task, that is, a dependency trigger relationship between the second Flow task and other Flow tasks. The second EndPoint Processor is responsible for recording each execution instance information of the second Flow task, including each execution success and execution failure, and each execution record one piece of information, mainly including the name of the task, execution start time, execution end time, execution state, and the like. In the prior art, the open source NIFI does not record the execution instance of the task, so that each execution instance of the task cannot be known, and the embodiment of the invention can record the execution information of the task by adding the specific second StartPoint Processor and the specific second EndPoint Processor.
The control unit 1020 is configured to control the second StartPoint Processor to periodically poll, trigger the judgment of the first Flow task execution record, and obtain the first Flow task execution instance information and the first trigger condition.
Specifically, the second StartPoint Processor may implement polling switching of the execution instance of the second Flow task between different nodes of the NIFI cluster, and the control unit 1020 sequentially queries the nodes on the NIFI cluster by controlling the second StartPoint Processor to perform periodic polling, so that each execution instance of the NIFI task may be executed on a different execution node under the same-scale NIFI cluster, and more NIFI tasks may be configured to implement load balancing with a greater degree of task instance granularity.
In the NIFI cluster operation mode, when one Processor fails, the second StartPoint Processor does not poll the node any more, so that each instance of the task can be guaranteed to operate normally as expected. Meanwhile, due to the existence of a trigger mechanism, if the task execution record is not processed, the running task is still triggered when the judgment logic is run again, so that the triggered task is ensured not to be missed.
The triggering unit 1030 is configured to trigger, according to the first Flow task execution instance information and the first triggering condition, the second StartPoint Processor to trigger the subsequent Processor to execute the task if the second StartPoint Processor determines that the first Flow task execution instance information meets the first triggering condition; if the first Flow task execution instance information is judged not to meet the first trigger condition, the second StartPoint Processor does not trigger the subsequent Processor to execute the task.
Specifically, there are Flow a and Flow B in the NIFI, and Flow a triggers Flow B execution. The trigger unit 1030 determines StartPoint Processor of the Flow B according to the acquired execution instance information of the Flow a and the condition for triggering the execution of the Flow B, if the execution record of the Flow a task meets the execution condition, startPoint Processor of the Flow B triggers the subsequent Processor to execute, otherwise StartPoint Processor of the Flow B does not trigger the subsequent Processor to execute, thereby saving system resources. The method provided by the embodiment of the invention realizes the realization of the triggering task execution by relying on the logic relation between the data acquisition and summarization tasks, meets the arrangement requirement of the task execution sequence among the tasks, and makes up the functional deficiency of the NIFI in the aspect of task dependency triggering, so that the application range of the NIFI is wider, and the requirements of various industries on production scenes such as data acquisition, summarization and the like can be met.
It should be noted that, in the embodiment of the present invention, not only the Flow a triggering Flow B may be implemented, but also the corresponding triggering conditions may be set in StartPoint Processor in each Flow according to specific situations, so as to implement complex dependency relationships such as Flow A And Flow B triggering Flow C and Flow a Or Flow B triggering Flow C.
The task flow triggering device based on the NIFI provided by the embodiment of the present invention is used for executing the task flow triggering method based on the NIFI, and the specific implementation manner and the method implementation manner of the task flow triggering device are consistent, and are not repeated here.
According to the task Flow triggering device based on the NIFI, the second StartPoint Processor configured in the second Flow task is controlled to periodically poll, judgment of the execution record of the first Flow task is triggered, and according to the task execution instance information recorded by the first EndPoint Processor on the first Flow task and the first triggering condition defined by the first StartPoint Processor, the second StartPoint Processor judges whether the first Flow task execution instance information meets the first triggering condition, and after judging that the first triggering condition is met, the subsequent Processor execution task is triggered, and if judging that the first triggering condition is not met, the subsequent Processor execution task is not triggered, so that the execution of the trigger task is realized based on the NIFI according to the logic relation between the data acquisition summary tasks, the application range of the NIFI is wider, and the task execution efficiency is improved.
Based on the content of the foregoing embodiment, as an alternative embodiment, the first Flow task starts with the first StartPoint Processor, and designates a cluster node where the first Flow task first executes an instance; all normal conditions and all abnormal conditions in the execution process of the first Flow task are separately processed and finally Flow to the first EndPoint Processor;
correspondingly, the second Flow task starts with a second StartPoint Processor, and the cluster node where the first execution instance of the second Flow task is located is designated; all normal conditions and all abnormal conditions in the second Flow task execution are handled separately and eventually Flow to the second EndPoint Processor.
Specifically, the first Flow task starts with the first StartPoint Processor, so that the nodes on the NIFI cluster can be conveniently polled, and thus the cluster node where the first execution instance of the first Flow task is located is designated, and the execution instance starts. In the first Flow task execution process, two conditions of success and failure exist, namely normal conditions and abnormal conditions are respectively corresponding, the two conditions are separately processed and finally Flow to the first EndPoint Processor, information of successful and failed execution examples is recorded, when a task record list has failed record information, the failed record can be re-run on an interface, the failed task can be re-run, the trigger condition is not relied on, and the follow-up task execution can be continuously triggered after the task execution is successful.
Accordingly, the second Flow task starts with the second StartPoint Processor, so that the nodes on the NIFI cluster can be conveniently polled, and thus the cluster node where the first execution instance of the second Flow task is located is designated, and the execution instance starts. In the execution process of the second Flow task, two conditions of success and failure exist, namely normal conditions and abnormal conditions are respectively corresponding, the two conditions are separately processed and finally Flow to the second EndPoint Processor, information of successful and failed execution examples is recorded, when a task record list has failed record information, the failed record can be re-run on an interface, the failed task can be re-run, the trigger condition is not relied on, and the subsequent task execution can be continuously triggered after the task execution is successful.
The task flow triggering device based on the NIFI provided by the embodiment of the present invention is used for executing the task flow triggering method based on the NIFI, and the specific implementation manner and the method implementation manner of the task flow triggering device are consistent, and are not repeated here.
According to the task Flow triggering device based on the NIFI, provided by the embodiment of the invention, all normal conditions and all abnormal conditions in the execution process of the Flow task are processed separately and finally Flow to EndPoint Processor by starting the Flow task at StartPoint Processor, and each execution instance can be processed separately, so that the automatic task re-running triggering and other abnormal conditions are processed, the operation is simple, and the task executing efficiency is improved.
Based on the foregoing embodiment, as an optional embodiment, the configuring of the first trigger condition and the second trigger condition by drag of the Web page specifically includes:
the connection configuration of the dependent triggering conditions among the Flow tasks; trigger condition configuration of a single Flow task; logic trigger condition configuration among a plurality of Flow tasks.
Specifically, the first trigger condition is that configuration information is stored in a database through drag of a Web page, background configuration is not needed, a foreground configuration mode is completely realized, a Flow A trigger Flow B can be realized through connection configuration of the Web page, namely, the configuration of dependent trigger conditions among Flow tasks, the trigger conditions of the Flow A can be configured through the Web page, namely, the trigger conditions of a single Flow task, and a Flow A & Flow B trigger Flow C can be configured through the Web page, namely, the logic trigger conditions among a plurality of Flow tasks. The trigger condition configuration is flexible and simple on the web page in a dragging mode, the modification and logic change of the dependency relationship of subsequent personnel are very convenient, and the safety and maintainability of the system are improved.
Similarly, the second trigger condition can be configured by dragging the Web page in the same manner as described above.
The task flow triggering device based on the NIFI provided by the embodiment of the present invention is used for executing the task flow triggering method based on the NIFI, and the specific implementation manner and the method implementation manner of the task flow triggering device are consistent, and are not repeated here.
The task flow triggering device based on NIFI provided by the embodiment of the invention has the advantages that the triggering condition can be configured through the dragging of the Web page, the configuration is flexible and simple, the modification and logic change of the dependency relationship of subsequent personnel are very convenient, and the safety and maintainability of the system are improved.
Fig. 11 is a schematic diagram of an entity structure of an electronic device according to an embodiment of the present invention, as shown in fig. 11, the electronic device may include: processor 1110, communication interface Communications Interface 1120, memory 1130 and communication bus 1140, wherein processor 1110, communication interface 1120 and memory 1130 communicate with each other via communication bus 1140. Processor 1110 may invoke logic instructions in memory 1130 to perform the steps of the NIFI-based task flow triggering method described above.
Further, the logic instructions in the memory 1130 described above may be implemented in the form of software functional units and sold or used as a stand-alone product, stored on a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, an embodiment of the present invention further provides a non-transitory computer readable storage medium, where a computer program is stored, where the computer program is implemented when executed by a processor to perform the NIFI-based task flow triggering method provided in the foregoing embodiments.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (9)
1. The task flow triggering method based on NIFI is characterized by comprising the following steps:
configuring a first StartPoint Processor in the first Flow task, setting a first triggering condition through a first StartPoint Processor, configuring a first EndPoint Processor in the first Flow task, and recording each execution instance information of the first Flow task through a first EndPoint Processor;
accordingly, a second StartPoint Processor is configured in the second Flow task, a second trigger condition is set through a second StartPoint Processor, and meanwhile, a second EndPoint Processor is configured in the second Flow task, and each execution instance information of the second Flow task is recorded through a second EndPoint Processor;
The first triggering condition is a condition that the first Flow task triggers the execution of a second Flow task, and the second triggering condition is a condition that the second Flow task triggers the execution of any task except the second Flow task;
controlling the second StartPoint Processor to periodically poll, triggering the judgment of the execution record of the first Flow task, and acquiring the execution instance information of the first Flow task and a first triggering condition;
according to the first Flow task execution instance information and the first trigger condition, if the second StartPoint Processor judges that the first Flow task execution instance information meets the first trigger condition, the second StartPoint Processor triggers the subsequent Processor to execute the task; if the first Flow task execution instance information is judged not to meet the first trigger condition, the second StartPoint Processor does not trigger the subsequent Processor to execute the task.
2. The NIFI-based task Flow triggering method of claim 1, wherein the first Flow task starts with a first StartPoint Processor and designates a cluster node where a first execution instance of the first Flow task is located; all normal conditions and all abnormal conditions in the execution process of the first Flow task are separately processed and finally Flow to the first EndPoint Processor;
Correspondingly, the second Flow task starts with a second StartPoint Processor, and the cluster node where the first execution instance of the second Flow task is located is designated; all normal conditions and all abnormal conditions in the second Flow task execution are handled separately and eventually Flow to the second EndPoint Processor.
3. The NIFI-based task flow triggering method according to claim 1, wherein the first triggering condition and the second triggering condition are configured by dragging a Web page, and specifically include:
the connection configuration of the dependent triggering conditions among the Flow tasks;
trigger condition configuration of a single Flow task;
logic trigger condition configuration among a plurality of Flow tasks.
4. The NIFI based task Flow triggering method of claim 1, wherein the first StartPoint Processor specifies an execution granularity of a first Flow task; the second StartPoint Processor specifies an execution granularity of a second Flow task; if the first Flow task is of an hour granularity and the second Flow task is of a day granularity, the first trigger condition is that the first Flow task can trigger the second Flow task to execute after the first Flow task executes 24 tasks within the time range where the second Flow task is completed.
5. A task flow triggering device based on NIFI, comprising:
a configuration unit, configured to configure the first StartPoint Processor in the first Flow task, set a first trigger condition through the first StartPoint Processor, and simultaneously configure the first EndPoint Processor in the first Flow task, and record each execution instance information of the first Flow task through the first EndPoint Processor;
accordingly, a second StartPoint Processor is configured in the second Flow task, a second trigger condition is set through a second StartPoint Processor, and meanwhile, a second EndPoint Processor is configured in the second Flow task, and each execution instance information of the second Flow task is recorded through a second EndPoint Processor;
the first triggering condition is a condition that the first Flow task triggers the execution of a second Flow task, and the second triggering condition is a condition that the second Flow task triggers the execution of any task except the second Flow task;
the control unit is used for controlling the second StartPoint Processor to periodically poll, triggering the judgment of the execution record of the first Flow task and acquiring the execution instance information of the first Flow task and the first triggering condition;
The triggering unit is configured to trigger, according to the first Flow task execution instance information and the first triggering condition, the second StartPoint Processor to trigger the subsequent Processor to execute the task if the second StartPoint Processor determines that the first Flow task execution instance information meets the first triggering condition; if the first Flow task execution instance information is judged not to meet the first trigger condition, the second StartPoint Processor does not trigger the subsequent Processor to execute the task.
6. The NIFI-based task Flow triggering device of claim 5, wherein the first Flow task starts with a first StartPoint Processor and designates a cluster node at which a first instance of execution of the first Flow task is located; all normal conditions and all abnormal conditions in the execution process of the first Flow task are separately processed and finally Flow to the first EndPoint Processor;
correspondingly, the second Flow task starts with a second StartPoint Processor, and the cluster node where the first execution instance of the second Flow task is located is designated; all normal conditions and all abnormal conditions in the second Flow task execution are handled separately and eventually Flow to the second EndPoint Processor.
7. The NIFI-based task flow triggering device of claim 5, wherein the first triggering condition and the second triggering condition are configured by dragging a Web page, and specifically comprise:
The connection configuration of the dependent triggering conditions among the Flow tasks;
trigger condition configuration of a single Flow task;
logic trigger condition configuration among a plurality of Flow tasks.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the NIFI-based task flow triggering method of any one of claims 1 to 4 when the program is executed.
9. A non-transitory computer readable storage medium, having stored thereon a computer program, which when executed by a processor, implements the steps of the NIFI-based task flow triggering method of any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010397799.8A CN113656468B (en) | 2020-05-12 | 2020-05-12 | Task flow triggering method and device based on NIFI |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010397799.8A CN113656468B (en) | 2020-05-12 | 2020-05-12 | Task flow triggering method and device based on NIFI |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113656468A CN113656468A (en) | 2021-11-16 |
| CN113656468B true CN113656468B (en) | 2024-02-27 |
Family
ID=78488706
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010397799.8A Active CN113656468B (en) | 2020-05-12 | 2020-05-12 | Task flow triggering method and device based on NIFI |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113656468B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104965754A (en) * | 2015-03-31 | 2015-10-07 | 腾讯科技（深圳）有限公司 | Task scheduling method and task scheduling apparatus |
| CN106293971A (en) * | 2016-08-15 | 2017-01-04 | 张家林 | A kind of method and apparatus of distributed task dispatching |
| CN107463356A (en) * | 2017-08-17 | 2017-12-12 | 北京云纵信息技术有限公司 | The execution method and apparatus of flow of task |
| CN109344189A (en) * | 2018-09-19 | 2019-02-15 | 浪潮软件集团有限公司 | Big data calculation method and device based on NiFi |
| CN109634652A (en) * | 2018-11-28 | 2019-04-16 | 郑州云海信息技术有限公司 | A kind of method, apparatus of data processing, computer storage medium and terminal |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190392002A1 (en) * | 2018-06-25 | 2019-12-26 | BigStream Solutions, Inc. | Systems and methods for accelerating data operations by utilizing dataflow subgraph templates |
| US20200081841A1 (en) * | 2018-09-07 | 2020-03-12 | BigStream Solutions, Inc. | Cache architecture for column-oriented database management systems |
-
2020
- 2020-05-12 CN CN202010397799.8A patent/CN113656468B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104965754A (en) * | 2015-03-31 | 2015-10-07 | 腾讯科技（深圳）有限公司 | Task scheduling method and task scheduling apparatus |
| CN106293971A (en) * | 2016-08-15 | 2017-01-04 | 张家林 | A kind of method and apparatus of distributed task dispatching |
| CN107463356A (en) * | 2017-08-17 | 2017-12-12 | 北京云纵信息技术有限公司 | The execution method and apparatus of flow of task |
| CN109344189A (en) * | 2018-09-19 | 2019-02-15 | 浪潮软件集团有限公司 | Big data calculation method and device based on NiFi |
| CN109634652A (en) * | 2018-11-28 | 2019-04-16 | 郑州云海信息技术有限公司 | A kind of method, apparatus of data processing, computer storage medium and terminal |
Non-Patent Citations (1)
| Title |
|---|
| "一种基于层次分割和聚合的大数据流水线任务处理方法";陈天乐 等;《科研信息化技术与应用》;第10卷(第1期);第3-11页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113656468A (en) | 2021-11-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107734035B (en) | Virtual cluster automatic scaling method in cloud computing environment | |
| US7340654B2 (en) | Autonomic monitoring in a grid environment | |
| US8938421B2 (en) | Method and a system for synchronizing data | |
| EP2940596B1 (en) | Data acquisition method and device | |
| CN106325984B (en) | Big data task scheduling device | |
| WO2017167021A1 (en) | Abnormality monitoring method and device | |
| CN113569987A (en) | Model training method and device | |
| CN110611707A (en) | Task scheduling method and device | |
| WO2019223174A1 (en) | Automatic task rerunning method and system, computer device and storage medium | |
| CN111240935B (en) | Automatic intelligent operation and maintenance system and operation and maintenance method | |
| US20100179957A1 (en) | Polling Method of Switch Status Based on Timer-triggered Scheduler of Stored Procedures | |
| CN111796954A (en) | Watchdog control method, device, equipment and storage medium based on JVM | |
| CN111190892A (en) | Method and device for processing abnormal data in data backfilling | |
| CN110619014A (en) | ETL-based data extraction method | |
| CN113051134A (en) | Application system monitoring management method, device, equipment and storage medium | |
| CN113656468B (en) | Task flow triggering method and device based on NIFI | |
| CN109725916B (en) | Topology updating system and method for stream processing | |
| CN111553545B (en) | Method for optimizing batch task scheduling dependence | |
| CN111538738B (en) | Service upgrading method and device, electronic equipment and storage medium | |
| CN113467908A (en) | Task execution method and device, computer readable storage medium and terminal equipment | |
| CN109684229B (en) | Chain dependency analysis method and system of directed acyclic graph | |
| CN117149378A (en) | Task scheduling method, device, equipment and medium of intelligent automobile operating system | |
| CN115357395A (en) | Fault equipment task transfer method and system, electronic equipment and storage medium | |
| CN117667362B (en) | Method, system, equipment and readable medium for scheduling process engine | |
| CN112581080A (en) | Lightweight distributed workflow engine construction system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |