WO2020259392A1 - 一种异常任务的根因任务确定方法及装置 - Google Patents
一种异常任务的根因任务确定方法及装置 Download PDFInfo
- Publication number
- WO2020259392A1 WO2020259392A1 PCT/CN2020/096906 CN2020096906W WO2020259392A1 WO 2020259392 A1 WO2020259392 A1 WO 2020259392A1 CN 2020096906 W CN2020096906 W CN 2020096906W WO 2020259392 A1 WO2020259392 A1 WO 2020259392A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- task
- upstream
- abnormal
- depends
- current target
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/079—Root cause analysis, i.e. error or fault diagnosis
Definitions
- the invention relates to the field of financial technology (Fintech) and task execution, and more particularly to a method and device for determining the root cause of an abnormal task.
- the embodiments of the present application provide a method and device for determining a root cause task of an abnormal task, which solves the problem that the root cause task determination in the prior art is low in efficiency and is prone to errors.
- the embodiment of the present application provides a method for determining the root cause task of an abnormal task: according to a preset dependency rule, it is determined whether the abnormal task has an upstream task it depends on; if the abnormal task has an upstream task it depends on, then Regard the abnormal task as the current target task; determine whether there is an executing or running error task among the upstream tasks on which the current target task depends; if not, set the upstream tasks on which the current target task depends Tasks that have not yet started to be executed are re-used as the current target task to update the current target task and return to execute the determination to determine whether the current target task depends on the upstream tasks that are currently executing or Steps to run the wrong task.
- the upstream task that the abnormal task depends on is determined according to the preset dependency rule, it is not easy to make mistakes, and the determined upstream task that it depends on is selective, and there is no need to deal with the upstream task that it depends on.
- the tasks that have not started to be executed in the upstream tasks that the current target task depends on are reset As the current target task, update the current target task and return to the step of executing the step of determining whether there is an executing or running error task among the upstream tasks on which the current target task depends, thereby Selectively continue to investigate the root cause tasks of abnormal tasks based on the target tasks.
- the method before determining whether an abnormal task has an upstream task it depends on according to a preset dependency rule, the method further includes: parsing an orchestration file of the task scheduling platform to obtain task orchestration information stored in the orchestration file
- the task scheduling information is information indicating the task dependency relationship; according to the task dependency relationship indicated in the task scheduling information, the task scheduling information is converted into at least one data record; each data in the at least one data record The record corresponds to a task and the upstream task that the task depends on; and the preset dependency rule is generated according to the at least one data record.
- the task scheduling information is obtained by parsing the scheduling file of the task scheduling platform, and the task scheduling information is converted into at least one data record according to the task dependency relationship indicated in the task scheduling information;
- the data record generates the preset dependency rule, and the preset dependency rule can be generated in the data record through the above method, which avoids manual scheduling and analysis work.
- the determining whether an abnormal task has an upstream task that it depends on according to a preset dependency rule includes: if the task type of the abnormal task is an event waiting type, matching with the event waiting type The task corresponding to the unique matching event completion type is determined as the upstream task on which the abnormal task depends; if the abnormal task is the end task in the predefined task connection, then the start task in the predefined task is connected , It is determined as the upstream task on which the abnormal task depends; if the abnormal task is the first task in the sub-group task, and the sub-group task has a predefined sub-group upstream task, then the sub-group upstream task As the upstream task on which the abnormal task depends; or, if the child group task does not have a predefined child group upstream task, and the child group task belongs to a parent group task that has a predefined parent group upstream task, it is regarded as The upstream task on which the abnormal task depends.
- the upstream task of the abnormal task is determined by the predetermined dependency rules such as the task type of the abnormal task, whether it is the end task in the predefined task connection, whether it is the first task in the subgroup task, etc.
- the upstream task that handles the abnormal task is determined accurately, reducing the scope of investigation for determining the root cause task.
- the abnormal task is taken as the root cause task that caused the abnormal task to not start execution.
- the above method is a method for determining the root cause task of the abnormal task when the task status of the upstream task on which the current target task depends is successfully executed. It also gives a situation for determining the abnormal task as the abnormal task itself. There is no need to check other tasks.
- determining whether the current target task depends on whether there is a task that is being executed or is running incorrectly it further includes: if it exists, determining the task that is being executed or that is running incorrectly For the root cause task.
- the root cause task can be directly determined by the above method.
- the determining whether there is a task that is executing or running incorrectly among the upstream tasks on which the current target task depends includes: obtaining the upstream tasks on which the current target task depends according to a preset period According to the keyword information in the log information of each upstream task in the predetermined period, the task status of the upstream task is determined according to the matching result of the keyword information in the log information of the upstream task and the preset task status rule in the preset period.
- the above method is a method for determining the task status of at least one upstream task.
- the upstream task is determined by the keyword information in the log information of each upstream task on which the current target task depends in a preset period.
- the status of the task since the log records the information of the execution status of the upstream task, the above method can update the status of the upstream task in a timely and accurate manner.
- this application provides a device for determining the root cause of an abnormal task, including: a determining module for determining whether the abnormal task has an upstream task on which it depends according to a preset dependency rule; a processing module for If there is an upstream task on which the abnormal task depends, the abnormal task is regarded as the current target task; it is determined whether there is a task that is being executed or is running incorrectly among the upstream tasks on which the current target task depends; if it does not exist, the Tasks that have not started execution among the upstream tasks on which the current target task depends are re-used as the current target task to update the current target task and return to the place where the current target task is determined to be executed. Whether there are steps in the dependent upstream tasks that are executing or running the wrong task.
- the processing module is further configured to: parse an orchestration file of the task scheduling platform, and obtain task orchestration information stored in the orchestration file; the task orchestration information is information indicating task dependency; according to According to the task dependency relationship indicated in the task scheduling information, the task scheduling information is converted into at least one data record; each data record in the at least one data record corresponds to a task and the upstream task on which the task depends; The at least one data record generates the preset dependency rule.
- the determining module is specifically configured to: if the task type of the abnormal task is an event waiting type, determine the task corresponding to the event completion type that uniquely matches the event waiting type as The upstream task on which the abnormal task depends; if the abnormal task is the end task in the predefined task connection, then the start task in the predefined task connection is determined as the upstream task on which the abnormal task depends; If the abnormal task is the first task in the subgroup task, and the subgroup task has a predefined subgroup upstream task, then the subgroup upstream task is taken as the upstream task upon which the abnormal task depends; or, If the child group task does not have a predefined child group upstream task, and the parent group task to which the child group task belongs has a predefined parent group upstream task, it is regarded as the upstream task on which the abnormal task depends.
- the processing module is further configured to: if it exists, determine the task being executed or running incorrectly as the root cause task.
- the processing module is further configured to: according to a preset cycle, obtain keyword information in the log information of each upstream task among the upstream tasks on which the current target task depends, and according to the The matching result of the keyword information in the log information of the upstream task and the preset task status rule in the preset period determines the task status of the upstream task.
- the processing module is further configured to: if the task status of the upstream task on which the current target task depends is all successfully executed, then the abnormal task is regarded as causing the abnormal task to fail. The root cause task to start.
- this application provides a computing device, which includes:
- processor memory, transceiver, and bus interface; among them, the processor, memory and transceiver are connected by a bus;
- the processor is configured to read the program in the memory and execute the method for determining the root cause of the abnormal task
- the memory is used to store one or more executable programs, and can store data used by the processor when performing operations.
- this application provides a non-transitory computer-readable storage medium with instructions stored in the computer storage medium, which when run on a computer, cause the computer to execute the root cause task determination method of the above abnormal task.
- the present application provides a computer program product containing instructions that, when run on a computer, causes the computer to execute the root cause task determination method of the above abnormal task.
- FIG. 1 is a schematic flowchart of steps of a method for determining the root cause of an abnormal task according to an embodiment of the application;
- FIG. 2 is a schematic flow chart of the steps of determining upstream tasks in a method for determining root cause tasks of abnormal tasks provided by an embodiment of the application;
- FIG. 3 is a schematic diagram of the dependency relationship between tasks in a method for determining the root cause of an abnormal task provided in an embodiment of the application;
- FIG. 4 is a schematic structural diagram of a root cause task determination device for an abnormal task provided in an embodiment of the application
- FIG. 5 is a schematic structural diagram of a computing device provided by this application.
- Fintech tasks can be scheduled and executed on multiple platforms, for example, a big data platform (BDP).
- BDP big data platform
- computing tasks such as batch tasks: refers to the computing tasks that are executed on the big data platform and perform statistics, cleaning, and analysis of data.
- scheduling platform refers to a client tool used to orchestrate and schedule the execution, re-execution, and skipping of batch tasks on the big data platform.
- Arrangement file Used to define the grouping, job type, specific execution instructions of each batch task, and its position on the scheduling platform.
- the scheduling platform will display and control the client interface according to the content of the layout file.
- Service-Level Agreement (SLA) is a mutually recognized agreement defined between service providers and users to ensure the performance and reliability of services at a certain cost, which includes the guarantee of service effectiveness. For example, guarantees for troubleshooting time and service overtime. Since there will be a dependency relationship between computing tasks, that is, the execution of one task will be constrained by the execution results of other tasks, so it is necessary to analyze and investigate the root cause of the abnormal task.
- FIG. 1 it is a schematic flowchart of steps of a method for determining the root cause task of an abnormal task provided by an embodiment of this application.
- Step 101 According to a preset dependency rule, determine whether an abnormal task has an upstream task on which it depends.
- Step 102 If the abnormal task has an upstream task that it depends on, the abnormal task is taken as the current target task.
- Step 103 Determine whether there is a task that is being executed or is running incorrectly among the upstream tasks on which the current target task depends.
- Step 104 If it does not exist, re-take the tasks that have not started execution among the upstream tasks on which the current target task depends on the current target task, so as to update the current target task and return to the execution place. The step of determining whether there is a task that is being executed or is running incorrectly among the upstream tasks on which the current target task depends.
- step 101 the dependent upstream tasks are defined as follows:
- N first tasks and second tasks satisfy the following conditions (1) and (2) at the same time, and N is a positive integer, then N first tasks are called upstream tasks on which the second task depends: (1) Nth Before any of the first tasks in a task is successfully executed, the second task cannot be executed; (2) After all the N first tasks are successfully executed, the execution of the second task is triggered.
- the definition of the upstream task relied on in this application is not transitive, and the upstream task on which the first task depends is not the upstream task on which the second upstream task depends.
- the second task may also be referred to as a downstream task of any one of the N first tasks. At this time, it is also said that there is an upstream and downstream relationship between the second task and the N first tasks.
- execution not started means that it is not in the execution state at the preset moment.
- the upstream tasks on which the abnormal task depends can be determined through the process shown in FIG. 2.
- FIG. 2 a method for determining the root cause task of the abnormal task provided in this embodiment of the application Schematic diagram of the steps in determining upstream tasks in the.
- Step 201 Analyze an orchestration file of the task scheduling platform, and obtain task orchestration information stored in the orchestration file.
- Step 201 can be parsed through shell language.
- the task scheduling information is information indicating task dependency.
- Step 202 Convert the task scheduling information into at least one data record according to the task dependency relationship indicated in the task scheduling information.
- Each data record in the at least one data record corresponds to a task and the upstream task on which the task depends.
- the predefined task relationship stored in the extensible markup language (xml) format and the task definition information can be spliced into a structured query language (structured query language, SQL) insert statements are imported into the mysql database.
- structured query language structured query language
- Step 203 Generate the preset dependency rule according to the at least one data record.
- the upstream task on which the abnormal task depends is determined according to the preset dependency rules.
- the task scheduling information is obtained by parsing the scheduling file of the task scheduling platform, and the task scheduling information is converted into at least one data record according to the task dependency relationship indicated in the task scheduling information;
- the data record generates the preset dependency rule, and the preset dependency rule can be generated in the data record through the above method, which avoids manual scheduling and analysis work.
- an optional implementation for determining whether an abnormal task has an upstream task on which it depends may be as follows, including:
- the task type of the abnormal task is the event waiting type
- the task corresponding to the event completion type that uniquely matches the event waiting type is determined as the upstream task on which the abnormal task depends.
- task A of the event waiting type and task B of the event completion type are a predefined pair of matching tasks. After task B is successfully executed, task A will be triggered to execute.
- the trigger method is not limited, for example, it can be triggered by waiting for a signal.
- the start task in the predefined task connection is determined as the upstream task on which the abnormal task depends.
- the start task and the end task are defined in the predefined task connection, and the start task is the upstream task of the end task.
- the connection L defines the start task L1 and the end task L2.
- L1 is an upstream task of the abnormal task.
- the subgroup upstream task is taken as the upstream task on which the abnormal task depends Or, if the sub-group task does not have a predefined sub-group upstream task, and the sub-group task belongs to a parent group task that has a predefined parent group upstream task, it is used as the upstream task on which the abnormal task depends.
- the sub-group tasks are multiple task groups with upstream task relationships, and the sub-group tasks can be a task unit of the parent group tasks.
- a parent group task there are upstream and downstream relationships between the sub-group tasks.
- the upstream tasks of task C1 are tasks C2 and C3, and the upstream tasks of task C2 are C4, then the combination of C1, C2, C3, and C4 is a subgroup task.
- the first task of the subgroup task is a task that meets the following conditions: (1) The task is an upstream task of a task in the subgroup task. (2) This task has no upstream task. (3) There is one and only one upstream task of the downstream task of the task, and it is the task.
- the upstream task of the abnormal task is determined by the predetermined dependency rules such as the task type of the abnormal task, whether it is the end task in the predefined task connection, whether it is the first task in the subgroup task, etc.
- the upstream task that handles the abnormal task is determined accurately, reducing the scope of investigation for determining the root cause task.
- the abnormal task is taken as the root cause task that caused the abnormal task to not start execution.
- the above method is a method for determining the root cause task of the abnormal task when the task status of the upstream task on which the current target task depends is successfully executed. It also gives a situation for determining the abnormal task as the abnormal task itself. There is no need to check other tasks.
- step 102 to step 104 it is necessary to obtain the status of the upstream task on which the current target task depends, and the task status on which the current target task depends can be obtained in the following manner:
- the keyword information in the log information of each upstream task in the upstream task on which the current target task depends is obtained, and the keyword information in the log information of the upstream task in the preset period is compared with The matching result of the preset task status rule determines the task status of the upstream task.
- the above method is a method for determining the task status of at least one upstream task.
- the upstream task is determined by the keyword information in the log information of each upstream task on which the current target task depends in a preset period.
- the status of the task since the log records the information of the execution status of the upstream task, the above method can update the status of the upstream task in a timely and accurate manner.
- the update time of each task log is captured on the batch server, combined with the keywords in the log information, to determine the current status of the task: including batch date, running status (successful execution Completion, running error, not starting execution, etc.), task start/end time, running time, etc.
- the task of the subgroup of the task and the task of the parent group of the subgroup are traversed upwards, and the corresponding status is calculated according to the built-in logic; the entire analysis process is relatively independent, supporting tasks in normal operation and individual re-running.
- step 102 to step 104 there are two ways to determine the root cause of the abnormal task:
- the first type If there is an upstream task that is executing or running incorrectly among the upstream tasks on which the current target task depends, the upstream task that is executing or running incorrectly is regarded as the root cause task of the abnormal task.
- the abnormal task is taken as the root cause task that caused the abnormal task not to be executed.
- the above method is a method for determining the root cause task of the abnormal task when the task status of the upstream task on which the current target task depends is successfully executed. It also gives a situation for determining the abnormal task as the abnormal task itself. There is no need to check other tasks.
- FIG. 3 a method for determining the root cause task of an abnormal task provided in an embodiment of the present application.
- FIG. 3 it is a schematic diagram of the dependency relationship between tasks in a method for determining the root cause of an abnormal task provided in an embodiment of this application.
- task K1 has two upstream tasks: task K2 and task K3, and task K3 has two upstream tasks: task K4 and task K5.
- the precondition for task K1 to be executed normally is that all upstream tasks including task K2 and task K3 have been completed normally.
- K1 does not start execution at the preset time, indicating that it may be affected by upstream tasks, then:
- the first step is to substitute the information of task K1 to find out the upstream tasks of the nearest level, including task K2 and task K3.
- Task K2 is successfully executed but task K3 did not start normally, indicating that it is the upstream task of task K3 or task K3 There is a problem.
- the second step is to query the upstream task status of task K3.
- task K4 was successfully executed, and task K5 displayed a running error.
- task K5 was the root cause task that caused task K1 to not start execution.
- the third step is to display the query result as task K5, and the operation and maintenance personnel can directly investigate the abnormal cause of task K5, and then solve the problem.
- the task K5 obtained at this time is the root cause task that affects the improper execution of the downstream task, and only needs to be directly processed for the task K5. If task K5 is currently running, you can view the execution time and log to analyze whether the task efficiency has changed due to code updates or accidental increments of data pushed upstream; if there is an execution error, you can locate the cause of the error according to the log record; The exception is an external signal waiting timeout. You can directly determine the responsible party based on the task name or description, and contact it to confirm the cause of the signal delay. Combining this technology with monitoring prompts such as WeChat, email, etc., you can clearly and directly hit the root cause of the problem, saving time for layer-by-layer analysis and judgment.
- This application uses technical means to automatically parse the layout file, convert the upstream and downstream seen on the interface into a dependency relationship that can be identified by the code, and backtrack and traverse the status of all upstream tasks according to the level, until it finds the upstream and downstream tasks that have not ended normally.
- the task can be positioned as the root cause task affecting the abnormal task.
- the determined at least one upstream task of the abnormal task is determined according to the preset dependency rule, it is not easy to make mistakes, and the determined at least one upstream task is selective, and there is no need to perform tasks other than at least one upstream task.
- the investigation is more efficient.
- the root cause task that caused the first upstream task to not start execution will be determined as the one that caused the abnormal task to not start execution Root cause tasks, so as to selectively continue to investigate the root cause tasks of abnormal tasks based on upstream tasks.
- the above methods greatly reduce invalid investigation work and improve the efficiency of root cause task determination.
- FIG. 4 it is a schematic structural diagram of an apparatus for determining a root cause task of an abnormal task provided in an embodiment of the application.
- the present application provides an apparatus for determining root cause tasks of abnormal tasks, including: a determining module 401, configured to determine whether the abnormal task has an upstream task it depends on according to a preset dependency rule; a processing module 402, configured to determine whether the abnormal task If there is a dependent upstream task, the abnormal task is regarded as the current target task; it is determined whether the current target task depends on whether there is a task that is being executed or is running incorrectly; if it does not exist, the Tasks that have not been executed in the upstream tasks on which the current target task depends are re-used as the current target task, so as to update the current target task and return to execute the determined current target task. Whether there is a step in the upstream task that is executing or running the wrong task.
- the processing module 402 is further configured to: parse an orchestration file of the task scheduling platform, and obtain task orchestration information stored in the orchestration file; the task orchestration information is information indicating task dependency; Converting the task scheduling information into at least one data record according to the task dependency relationship indicated in the task scheduling information; each data record in the at least one data record corresponds to a task and an upstream task on which the task depends; According to the at least one data record, the preset dependency rule is generated.
- the determining module 401 is specifically configured to: if the task type of the abnormal task is an event waiting type, determine the task corresponding to the event completion type that uniquely matches the event waiting type Is the upstream task on which the abnormal task depends; if the abnormal task is the end task in the predefined task connection, then the start task in the predefined task connection is determined as the upstream task on which the abnormal task depends If the abnormal task is the first task in the sub-group task, and the sub-group task has a predefined sub-group upstream task, then the sub-group upstream task is taken as the upstream task on which the abnormal task depends; or If the child group task does not have a predefined child group upstream task, and the parent group task to which the child group task belongs has a predefined parent group upstream task, it is regarded as the upstream task on which the abnormal task depends.
- the processing module 402 is further configured to: if it exists, determine the task being executed or running incorrectly as the root cause task.
- the processing module 402 is further configured to: according to a preset period, obtain keyword information in the log information of each upstream task on which the current target task depends, according to the According to the matching result between the keyword information in the log information of the upstream task and the preset task status rule in the preset period, the task status of the upstream task is determined.
- the processing module 402 is further configured to: if the task status of the upstream task that the current target task depends on is successfully executed, then the abnormal task is regarded as the task that caused the abnormality. Root cause tasks that have not started.
- this application also provides a computing device. As shown in FIG. 5, the computing device includes:
- the processor 501 is configured to read the program in the memory 502 and execute the method for determining the root cause of the abnormal task;
- the processor 501 may be a central processing unit (central processing unit, CPU for short), a network processor (NP for short), or a combination of CPU and NP. It can also be a hardware chip.
- the aforementioned hardware chip may be an application-specific integrated circuit (ASIC for short), a programmable logic device (PLD for short), or a combination thereof.
- ASIC application-specific integrated circuit
- PLD programmable logic device
- the above-mentioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (generic array logic, GAL), or any of them combination.
- the memory 502 is configured to store one or more executable programs, and can store data used by the processor 501 when performing operations.
- the program may include program code, and the program code includes computer operation instructions.
- the memory 502 may include a volatile memory (volatile memory), such as random-access memory (RAM for short); the memory 502 may also include a non-volatile memory (non-volatile memory), such as flash memory ( flash memory, hard disk drive (HDD for short) or solid-state drive (SSD for short); the memory 502 may also include a combination of the foregoing types of memory.
- volatile memory volatile memory
- RAM random-access memory
- non-volatile memory non-volatile memory
- flash memory flash memory, hard disk drive (HDD for short) or solid-state drive (SSD for short
- SSD solid-state drive
- the memory 502 stores the following elements, executable modules or data structures, or their subsets, or their extended sets:
- Operating instructions including various operating instructions, used to implement various operations.
- Operating system including various system programs, used to implement various basic services and process hardware-based tasks.
- the bus 505 may be a peripheral component interconnect standard (PCI) bus or an extended industry standard architecture (EISA) bus, etc.
- PCI peripheral component interconnect standard
- EISA extended industry standard architecture
- the bus can be divided into address bus, data bus, control bus, etc. For ease of presentation, only one thick line is used in FIG. 5 to represent, but it does not mean that there is only one bus or one type of bus.
- the bus interface 504 may be a wired communication access port, a wireless bus interface or a combination thereof, where the wired bus interface may be, for example, an Ethernet interface.
- the Ethernet interface can be an optical interface, an electrical interface, or a combination thereof.
- the wireless bus interface may be a WLAN interface.
- the embodiments of the present application also provide a non-transitory computer-readable storage medium, in which instructions are stored in the computer storage medium.
- the root cause of the abnormal task is determined by the computer. method.
- the embodiments of the present application provide a computer program product containing instructions, which when run on a computer, cause the computer to execute the root cause task determination method of the above abnormal task.
- the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, optical storage, etc.) containing computer-usable program codes.
- a computer-usable storage media including but not limited to disk storage, optical storage, etc.
- These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device.
- the device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
Description
Claims (13)
- 一种异常任务的根因任务确定方法,其特征在于,包括:根据预设依赖规则,确定异常任务是否存在所依赖的上游任务;若所述异常任务存在所依赖的上游任务,则将所述异常任务作为当前的目标任务;确定所述当前的目标任务所依赖的上游任务中是否存在正在执行或运行错误的任务;若不存在,则将所述当前的目标任务所依赖的上游任务中未开始执行的任务重新作为所述当前的目标任务,以对所述当前的目标任务进行更新,并返回执行所述确定所述当前的目标任务所依赖的上游任务中是否存在正在执行或运行错误的任务的步骤。
- 如权利要求1所述的方法,其特征在于,所述根据预设依赖规则,确定异常任务是否存在所依赖的上游任务之前,还包括:解析任务调度平台的编排文件,获取所述编排文件中存储的任务编排信息;所述任务编排信息为指示任务依赖关系的信息;根据所述任务编排信息中指示的任务依赖关系,将所述任务编排信息转化为至少一条数据记录;所述至少一条数据记录中每条数据记录对应一个任务以及该任务的所依赖的上游任务;根据所述至少一条数据记录,生成所述预设依赖规则。
- 如权利要求1所述的方法,其特征在于,所述根据预设依赖规则,确定异常任务是否存在所依赖的上游任务,包括:若所述异常任务的任务类型为事件等待类型,则将与所述事件等待类型的唯一匹配的事件完成类型对应的任务,确定为所述异常任务所依赖的上游任务;若所述异常任务为预定义任务连接中的结束任务,则将所述预定义任务连接中的开始任务,确定为所述异常任务所依赖的上游任务;若所述异常任务为子组任务中的首任务,且所述子组任务存在预定义的子组上游任务,则将所述子组上游任务作为所述异常任务所依赖的上游任务;或者,若所述子组任务不存在预定义的子组上游任务,且所述子组任务所属父组任务存在预定义的父组上游任务,作为所述异常任务所依赖的上游任务。
- 如权利要求1至3任一所述的方法,其特征在于,所述确定当前的目标任务所依赖的上游任务中是否存在正在执行或运行错误的任务之后,还包括:若存在,则将所述正在执行或运行错误的任务确定为根因任务。
- 如权利要求1至3任一所述的方法,其特征在于,所述确定当前的目标任务所依赖的上游任务中是否存在正在执行或运行错误的任务,包括:按照预设周期,获取所述当前的目标任务所依赖的上游任务中每个上游任务的日志信息中的关键字信息,根据所述预设周期内该上游任务的日志信息中的关键字信息与预设任务状态规则的匹配结果,确定该上游任务的任务状态。
- 一种异常任务的根因任务确定装置,其特征在于,包括:确定模块,用于根据预设依赖规则,确定异常任务是否存在所依赖的上游任务;处理模块,用于若所述异常任务存在所依赖的上游任务,则将所述异常任务作为当前的目标任务;确定所述当前的目标任务所依赖的上游任务中是否存在正在执行或运行错误的任务;若不存在,则将所述当前的目标任务所依赖的上游任务中未开始执行的任务重新作为所述当前的目标任务,以对所述当前的目标任务进行更新,并返回执行所述确定所述当前的目标任务所依赖的上游任务中是否存在正在执行或运行错误的任务的步骤。
- 如权利要求6所述的装置,其特征在于,所述处理模块还用于:解析任务调度平台的编排文件,获取所述编排文件中存储的任务编排信息;所述任务编排信息为指示任务依赖关系的信息;根据所述任务编排信息中指示的任务依赖关系,将所述任务编排信息转化为至少一条数据记录;所 述至少一条数据记录中每条数据记录对应一个任务以及该任务的所依赖的上游任务;根据所述至少一条数据记录,生成所述预设依赖规则。
- 如权利要求6所述的装置,其特征在于,所述确定模块具体用于:若所述异常任务的任务类型为事件等待类型,则将与所述事件等待类型的唯一匹配的事件完成类型对应的任务,确定为所述异常任务所依赖的上游任务;若所述异常任务为预定义任务连接中的结束任务,则将所述预定义任务连接中的开始任务,确定为所述异常任务所依赖的上游任务;若所述异常任务为子组任务中的首任务,且所述子组任务存在预定义的子组上游任务,则将所述子组上游任务作为所述异常任务所依赖的上游任务;或者,若所述子组任务不存在预定义的子组上游任务,且所述子组任务所属父组任务存在预定义的父组上游任务,作为所述异常任务所依赖的上游任务。
- 如权利要求6至8任一所述的装置,其特征在于,所述处理模块还用于:若存在,则将所述正在执行或运行错误的任务确定为根因任务。
- 如权利要求6至8任一所述的装置,其特征在于,所述处理模块还用于:按照预设周期,获取所述当前的目标任务所依赖的上游任务中每个上游任务的日志信息中的关键字信息,根据所述预设周期内该上游任务的日志信息中的关键字信息与预设任务状态规则的匹配结果,确定该上游任务的任务状态。
- 一种计算设备,其特征在于,包括处理器、存储器、收发器、总线接口,其中处理器、存储器与收发器之间通过总线连接;所述处理器,用于读取所述存储器中的程序,执行权利要求1至5任一所述方法;所述存储器,用于存储一个或多个可执行程序,以及存储所述处理器在执行操作时所使用的数据。
- 一种非暂态计算机可读存储介质,其特征在于,所述非暂态计算机可读存储介质存储计算机指令,所述计算机指令用于使所述计算机执行权利要求1至5任一所述方法。
- 一种计算机程序产品,其特征在于,所述计算机程序产品包括存储在非暂态计算机可读存储介质上的计算程序,所述计算机程序包括程序指令,当所述程序指令被计算机执行时,使所述计算机执行权利要求1至5任一所述方法。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910554278.6 | 2019-06-25 | ||
CN201910554278.6A CN110287052B (zh) | 2019-06-25 | 2019-06-25 | 一种异常任务的根因任务确定方法及装置 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020259392A1 true WO2020259392A1 (zh) | 2020-12-30 |
Family
ID=68005564
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/096906 WO2020259392A1 (zh) | 2019-06-25 | 2020-06-18 | 一种异常任务的根因任务确定方法及装置 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN110287052B (zh) |
WO (1) | WO2020259392A1 (zh) |
Cited By (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112667383A (zh) * | 2020-12-31 | 2021-04-16 | 北京高途云集教育科技有限公司 | 一种任务执行及调度方法、系统、装置、计算设备及介质 |
CN112817720A (zh) * | 2021-01-30 | 2021-05-18 | 北京奇保信安科技有限公司 | 一种可视化工作流调度方法、装置及电子设备 |
CN112990766A (zh) * | 2021-04-16 | 2021-06-18 | 成都数之联科技有限公司 | 不良根因设备的确定方法及系统及装置及介质 |
CN113238838A (zh) * | 2021-04-22 | 2021-08-10 | 中国银联股份有限公司 | 一种任务调度方法、装置及计算机可读存储介质 |
CN113347033A (zh) * | 2021-05-31 | 2021-09-03 | 中国工商银行股份有限公司 | 基于区块链的根因定位方法、系统及验证节点 |
CN113590579A (zh) * | 2021-06-22 | 2021-11-02 | 微梦创科网络科技(中国)有限公司 | 一种基于数据仓库的根因分析方法、装置及系统 |
CN113886122A (zh) * | 2021-09-30 | 2022-01-04 | 济南浪潮数据技术有限公司 | 一种系统运行异常处理方法、装置、设备及存储介质 |
CN113918937A (zh) * | 2021-09-10 | 2022-01-11 | 广州博依特智能信息科技有限公司 | 一种基于大数据的非法事件识别方法及系统 |
CN114064609A (zh) * | 2021-11-12 | 2022-02-18 | 中交智运有限公司 | 一种数据仓库任务调度方法、装置、系统及存储介质 |
CN114168275A (zh) * | 2021-10-28 | 2022-03-11 | 厦门国际银行股份有限公司 | 任务调度方法、系统、终端设备及存储介质 |
CN115378792A (zh) * | 2021-05-18 | 2022-11-22 | 腾讯云计算(北京)有限责任公司 | 告警处理方法、装置及存储介质 |
CN116224400A (zh) * | 2023-03-03 | 2023-06-06 | 博泰车联网(南京)有限公司 | 一种定位分析方法、装置、设备及存储介质 |
CN113590579B (zh) * | 2021-06-22 | 2024-05-31 | 微梦创科网络科技(中国)有限公司 | 一种基于数据仓库的根因分析方法、装置及系统 |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110287052B (zh) * | 2019-06-25 | 2022-01-28 | 深圳前海微众银行股份有限公司 | 一种异常任务的根因任务确定方法及装置 |
CN112749034B (zh) * | 2019-10-31 | 2024-05-17 | 北京沃东天骏信息技术有限公司 | 一种数据异常处理方法、装置、存储介质及电子设备 |
CN111290911A (zh) * | 2020-01-21 | 2020-06-16 | 上海悦易网络信息技术有限公司 | 一种用于检测移动终端任务的方法及设备 |
CN111859403B (zh) * | 2020-07-30 | 2023-09-05 | 中国工商银行股份有限公司 | 依赖关系漏洞的确定方法、装置、电子设备及存储介质 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160026542A1 (en) * | 2014-07-22 | 2016-01-28 | Cisco Technology Inc. | Pre-Computation of Backup Topologies in Computer Networks |
CN108958920A (zh) * | 2018-07-13 | 2018-12-07 | 众安在线财产保险股份有限公司 | 一种分布式任务调度方法及系统 |
CN109614400A (zh) * | 2018-11-30 | 2019-04-12 | 深圳前海微众银行股份有限公司 | 失败任务的影响及溯源分析方法、装置、设备及存储介质 |
CN110287052A (zh) * | 2019-06-25 | 2019-09-27 | 深圳前海微众银行股份有限公司 | 一种异常任务的根因任务确定方法及装置 |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP5719974B2 (ja) * | 2012-09-03 | 2015-05-20 | 株式会社日立製作所 | 複数の監視対象デバイスを有する計算機システムの管理を行う管理システム |
CN103034554B (zh) * | 2012-12-30 | 2015-11-18 | 焦点科技股份有限公司 | 一种纠错重启以及自动判断启动的etl调度系统及方法 |
CN105653419B (zh) * | 2014-11-11 | 2018-10-09 | 阿里巴巴集团控股有限公司 | 基于监控点的溯源实现方法及系统 |
CN104965754A (zh) * | 2015-03-31 | 2015-10-07 | 腾讯科技(深圳)有限公司 | 任务调度方法及任务调度装置 |
CN107870948A (zh) * | 2016-09-28 | 2018-04-03 | 平安科技(深圳)有限公司 | 任务调度方法和装置 |
CN107608774A (zh) * | 2017-09-08 | 2018-01-19 | 新智云数据服务有限公司 | 一种任务调度方法、装置、设备及存储介质 |
CN109558292A (zh) * | 2017-09-26 | 2019-04-02 | 阿里巴巴集团控股有限公司 | 一种监控方法及装置 |
-
2019
- 2019-06-25 CN CN201910554278.6A patent/CN110287052B/zh active Active
-
2020
- 2020-06-18 WO PCT/CN2020/096906 patent/WO2020259392A1/zh active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160026542A1 (en) * | 2014-07-22 | 2016-01-28 | Cisco Technology Inc. | Pre-Computation of Backup Topologies in Computer Networks |
CN108958920A (zh) * | 2018-07-13 | 2018-12-07 | 众安在线财产保险股份有限公司 | 一种分布式任务调度方法及系统 |
CN109614400A (zh) * | 2018-11-30 | 2019-04-12 | 深圳前海微众银行股份有限公司 | 失败任务的影响及溯源分析方法、装置、设备及存储介质 |
CN110287052A (zh) * | 2019-06-25 | 2019-09-27 | 深圳前海微众银行股份有限公司 | 一种异常任务的根因任务确定方法及装置 |
Cited By (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112667383A (zh) * | 2020-12-31 | 2021-04-16 | 北京高途云集教育科技有限公司 | 一种任务执行及调度方法、系统、装置、计算设备及介质 |
CN112667383B (zh) * | 2020-12-31 | 2024-02-09 | 北京高途云集教育科技有限公司 | 一种任务执行及调度方法、系统、装置、计算设备及介质 |
CN112817720A (zh) * | 2021-01-30 | 2021-05-18 | 北京奇保信安科技有限公司 | 一种可视化工作流调度方法、装置及电子设备 |
CN112990766A (zh) * | 2021-04-16 | 2021-06-18 | 成都数之联科技有限公司 | 不良根因设备的确定方法及系统及装置及介质 |
CN112990766B (zh) * | 2021-04-16 | 2023-10-20 | 成都数之联科技股份有限公司 | 一种产品搬送路径中不良根因设备的确定方法、系统、装置及介质 |
CN113238838A (zh) * | 2021-04-22 | 2021-08-10 | 中国银联股份有限公司 | 一种任务调度方法、装置及计算机可读存储介质 |
CN113238838B (zh) * | 2021-04-22 | 2023-01-24 | 中国银联股份有限公司 | 一种任务调度方法、装置及计算机可读存储介质 |
CN115378792A (zh) * | 2021-05-18 | 2022-11-22 | 腾讯云计算(北京)有限责任公司 | 告警处理方法、装置及存储介质 |
CN115378792B (zh) * | 2021-05-18 | 2024-05-17 | 腾讯云计算(北京)有限责任公司 | 告警处理方法、装置及存储介质 |
CN113347033A (zh) * | 2021-05-31 | 2021-09-03 | 中国工商银行股份有限公司 | 基于区块链的根因定位方法、系统及验证节点 |
CN113590579A (zh) * | 2021-06-22 | 2021-11-02 | 微梦创科网络科技(中国)有限公司 | 一种基于数据仓库的根因分析方法、装置及系统 |
CN113590579B (zh) * | 2021-06-22 | 2024-05-31 | 微梦创科网络科技(中国)有限公司 | 一种基于数据仓库的根因分析方法、装置及系统 |
CN113918937B (zh) * | 2021-09-10 | 2023-07-18 | 广州博依特智能信息科技有限公司 | 一种基于大数据的非法事件识别方法及系统 |
CN113918937A (zh) * | 2021-09-10 | 2022-01-11 | 广州博依特智能信息科技有限公司 | 一种基于大数据的非法事件识别方法及系统 |
CN113886122B (zh) * | 2021-09-30 | 2024-03-01 | 济南浪潮数据技术有限公司 | 一种系统运行异常处理方法、装置、设备及存储介质 |
CN113886122A (zh) * | 2021-09-30 | 2022-01-04 | 济南浪潮数据技术有限公司 | 一种系统运行异常处理方法、装置、设备及存储介质 |
CN114168275A (zh) * | 2021-10-28 | 2022-03-11 | 厦门国际银行股份有限公司 | 任务调度方法、系统、终端设备及存储介质 |
CN114064609A (zh) * | 2021-11-12 | 2022-02-18 | 中交智运有限公司 | 一种数据仓库任务调度方法、装置、系统及存储介质 |
CN116224400A (zh) * | 2023-03-03 | 2023-06-06 | 博泰车联网(南京)有限公司 | 一种定位分析方法、装置、设备及存储介质 |
CN116224400B (zh) * | 2023-03-03 | 2024-01-02 | 博泰车联网(南京)有限公司 | 一种定位分析方法、装置、设备及存储介质 |
Also Published As
Publication number | Publication date |
---|---|
CN110287052B (zh) | 2022-01-28 |
CN110287052A (zh) | 2019-09-27 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2020259392A1 (zh) | 一种异常任务的根因任务确定方法及装置 | |
US9576037B2 (en) | Self-analyzing data processing job to determine data quality issues | |
US20200250152A1 (en) | System for determining the impact to databases, tables and views by batch processing | |
US8055945B2 (en) | Systems, methods and computer program products for remote error resolution reporting | |
CN108804215B (zh) | 一种任务处理方法、装置以及电子设备 | |
WO2013140608A1 (ja) | イベントの根本原因の解析を支援する方法及びシステム | |
WO2015167466A1 (en) | Query plan post optimization analysis and reoptimization | |
JP2022048088A (ja) | コードをテストするための方法及び装置、電子機器、記憶媒体並びにコンピュータプログラム | |
US10509649B2 (en) | Value stream graphs across heterogeneous software development platforms | |
US10951509B1 (en) | Methods, systems, and computer readable media for providing intent-driven microapps for execution on communications network testing devices | |
CN112506799A (zh) | 业务异常定位方法及装置、电子设备、介质、产品 | |
CN113760677A (zh) | 异常链路分析方法、装置、设备及存储介质 | |
CN103440460A (zh) | 一种应用系统变更验证方法及验证系统 | |
US8380729B2 (en) | Systems and methods for first data capture through generic message monitoring | |
CN110011845B (zh) | 日志采集方法及系统 | |
CN112148762A (zh) | 一种实时数据流的统计方法和装置 | |
US20160041892A1 (en) | System for discovering bugs using interval algebra query language | |
WO2023103344A1 (zh) | 一种数据处理方法、装置、设备及存储介质 | |
CN112132544B (zh) | 业务系统的巡检方法及装置 | |
US10146605B2 (en) | Set-based bugs discovery system via SQL query | |
CN111694686B (zh) | 一种异常服务的处理方法、装置、电子设备及存储介质 | |
CN111143325B (zh) | 一种数据采集的监测方法、监测装置及可读存储介质 | |
CN115168297A (zh) | 绕行日志审计方法及装置 | |
CN112965793A (zh) | 一种面向标识解析数据的数据仓库任务调度方法和系统 | |
CN113094241A (zh) | 一种实时程序准确性的确定方法、装置、设备及存储介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20830943 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20830943 Country of ref document: EP Kind code of ref document: A1 |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 060422) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20830943 Country of ref document: EP Kind code of ref document: A1 |