CN111611127A - Processing method, device and equipment for task running log and storage medium - Google Patents
Processing method, device and equipment for task running log and storage medium Download PDFInfo
- Publication number
- CN111611127A CN111611127A CN202010340464.2A CN202010340464A CN111611127A CN 111611127 A CN111611127 A CN 111611127A CN 202010340464 A CN202010340464 A CN 202010340464A CN 111611127 A CN111611127 A CN 111611127A
- Authority
- CN
- China
- Prior art keywords
- task
- log
- error reporting
- error
- keywords
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title description 4
- 238000000034 method Methods 0.000 claims abstract description 78
- 238000012545 processing Methods 0.000 claims abstract description 51
- 238000004422 calculation algorithm Methods 0.000 claims description 42
- 238000004590 computer program Methods 0.000 claims description 11
- 230000008439 repair process Effects 0.000 description 36
- 238000013507 mapping Methods 0.000 description 15
- 238000010586 diagram Methods 0.000 description 12
- 238000004891 communication Methods 0.000 description 11
- 238000012549 training Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 238000000605 extraction Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 238000013473 artificial intelligence Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000007477 logistic regression Methods 0.000 description 3
- 230000001276 controlling effect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000001902 propagating effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 229910052802 copper Inorganic materials 0.000 description 1
- 239000010949 copper Substances 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 239000010979 ruby Substances 0.000 description 1
- 229910001750 ruby Inorganic materials 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003238 somatosensory effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000008685 targeting Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Debugging And Monitoring (AREA)
Abstract
The disclosure provides a method, a device, equipment and a storage medium for processing a task running log, wherein the method comprises the following steps: acquiring and storing preset error reporting reason keywords and corresponding error reporting rules; when the task operation fails, matching the task operation log according to the saved error reporting rule; and displaying the error reporting reason keywords corresponding to the successfully matched error reporting rules to the front end.
Description
Technical Field
The present invention relates to the field of artificial intelligence, and more particularly, to a method for processing a task execution log, a device for processing a task execution log, an apparatus including at least one computing device and at least one storage device, and a computer-readable storage medium.
Background
At present, in the field of artificial intelligence, automatic detection of operation errors and transparentization of operation states cannot be generally achieved. In the prior art, when a task fails to run, a user needs to search and locate keywords of a log and then manually finds problems, and the method mainly depends on manual problem finding, so that time and workload are greatly wasted, and low efficiency is caused; meanwhile, when the task operation fails, the task operation can be manually analyzed after the task operation is finished, real-time display in the process cannot be achieved, automatic detection of the reason of the operation failure cannot be provided, and the method cannot be practically applied to industry users requiring credibility and reliability for finance, government and the like.
Disclosure of Invention
An object of the embodiments of the present disclosure is to provide a new technical solution for processing a task execution log.
According to a first aspect of the present disclosure, a method for processing a task execution log is provided, which includes:
acquiring and storing preset error reporting reason keywords and corresponding error reporting rules;
when the task operation fails, matching the task operation log according to the saved error reporting rule;
and displaying the error reporting reason keywords corresponding to the successfully matched error reporting rules to the front end.
Optionally, the error reporting reason keyword and the corresponding error reporting rule include at least one of the following:
if the memory is insufficient, the error is reported corresponding to a single keyword;
the license is out of limit, and correspondingly meets a plurality of keywords at the same time without error report in the same row;
the algorithm parameter configuration is wrong, and correspondingly meets a plurality of keywords at the same time and does not report errors in the same row;
script grammar error, which corresponds to a plurality of keywords simultaneously and does not report error in the same line;
cluster time zones are asynchronous, a plurality of keywords are correspondingly met at the same time, and errors are not reported in the same row;
if the authority is not enough, the corresponding single keyword is wrongly reported.
Optionally, the method further comprises: according to the sequence of the execution steps in the task, classifying and collecting the running logs of the task to obtain a plurality of sub-log files;
the matching the task running log according to the saved error reporting rule comprises: and matching according to the reverse order of the generation sequence of the plurality of sub-log files.
Optionally, the plurality of sub-log files comprises: engine logs, non-traffic logs, and traffic logs.
Optionally, the engine log is used for recording system-related information when the execution engine is scheduled; the non-service log is used for recording system related information during task operation; the service log is used for recording algorithm related information during task operation.
Optionally, the engine log is generated at a first stage before the task runs, the non-service log is generated at a second stage of the task runs, the service log is generated at a third stage of the task runs, and the second stage and the third stage are sequentially executed according to a time sequence.
Optionally, the plurality of sub-log files includes an engine log and a non-traffic log,
the matching according to the reverse order of the generation order of the plurality of sub-log files includes:
and sequentially matching from the last non-service log from back to front.
Optionally, the matching according to the reverse order of the generation order of the multiple sub-log files includes:
and sequentially matching from the last service log from back to front.
Optionally, when the task fails to run, an error is reported to the scheduler by the execution engine, and the step of matching and exposing to the front end is executed by the scheduler.
Optionally, the method further comprises:
and acquiring a preset repairing program corresponding to the displayed error-reporting reason keyword, and operating the repairing program.
Optionally, the method further comprises:
and prompting a user whether to execute one-key repair before running the repair program, and executing the repair program when the user confirms.
Optionally, the method further comprises:
and writing the process of running the repairing program into the running log of the task.
Optionally, a communication connection is established with the front end through a Websocket protocol.
Optionally, the method further comprises:
acquiring a preset running state capturing rule model corresponding to the type of the task according to the type of the task;
when the task runs, the running state information of the task is obtained by the running state capture rule model and is sent to the front end for displaying.
Optionally, the operation log comprises a traffic log, the task is a GBDT algorithm training task,
the operation state information of the task is obtained by the operation state capturing rule model and is sent to the front end for displaying, and the method comprises the following steps:
positioning tree building information in the service log by using the operation state capturing rule model to obtain tree building start time, tree building end time, GBDT algorithm effect, resource consumption of the task and the number of processing data corresponding to the task in the tree building information;
and drawing the tree building information and sending the tree building information to a front end in real time for displaying.
Optionally, the execution log comprises a service log, the task is a feature extraction task,
the operation state information of the task is obtained by the operation state capturing rule model and is sent to the front end for displaying, and the method comprises the following steps:
positioning processing information of each line of data and each characteristic method in the service log by using the operation state capture rule model to obtain processing start time, processing end time, whether the characteristic method is effective or not, resources consumed by tasks of the tasks and effective proportion of the characteristic method in the processing information;
and drawing the processing information and sending the processing information to a front end in real time for displaying.
Optionally, the operation log includes a service log, and the method further includes:
and when the task runs, pushing the service log received in real time to a front end so that the front end analyzes the service log and displays the service log in real time.
Optionally, the log of runs comprises a log of traffic,
the method further comprises the following steps:
receiving a service log generated when a task pushed by a scheduling engine runs, wherein the service log is used for recording algorithm related information when the execution engine executes a corresponding operator to run the task;
and analyzing the service log and displaying the service log to a user in real time so that the user can determine whether to continue to execute the task according to the displayed running state of the task.
Optionally, the log of runs comprises a log of traffic,
the method further comprises the following steps:
checking a service log generated during the task running in real time;
judging whether the running state of the task meets a preset task ending running condition or not according to the service log;
and under the condition that the running state of the task meets the preset task ending running condition, ending the running of the task.
According to a second aspect of the present disclosure, there is also provided a processing apparatus for a task execution log, including:
the acquisition module is used for acquiring and storing preset error reporting reason keywords and corresponding error reporting rules;
the matching module is used for matching the task running log according to the stored error reporting rule when the task fails to run;
and the display module is used for displaying the error reporting reason keywords corresponding to the successfully matched error reporting rules to the front end.
Optionally, the error reporting reason keyword and the corresponding error reporting rule include at least one of the following:
if the memory is insufficient, the error is reported corresponding to a single keyword;
the license is out of limit, and correspondingly meets a plurality of keywords at the same time without error report in the same row;
the algorithm parameter configuration is wrong, and correspondingly meets a plurality of keywords at the same time and does not report errors in the same row;
script grammar error, which corresponds to a plurality of keywords simultaneously and does not report error in the same line;
cluster time zones are asynchronous, a plurality of keywords are correspondingly met at the same time, and errors are not reported in the same row;
if the authority is not enough, the corresponding single keyword is wrongly reported.
Optionally, the apparatus further comprises a classification module,
the classification module is used for classifying and collecting the running logs of the tasks according to the sequence of the execution steps in the tasks, so that a plurality of sub-log files are obtained;
and the matching module is used for matching according to the reverse order of the generation order of the plurality of sub-log files.
Optionally, the apparatus further comprises a classification module,
the classification module is used for classifying and collecting the running logs of the tasks according to the sequence of the execution steps in the tasks, so that a plurality of sub-log files are obtained;
and the matching module is used for matching according to the reverse order of the generation order of the plurality of sub-log files.
Optionally, the plurality of sub-log files comprises: engine logs, non-traffic logs, and traffic logs.
Optionally, the engine log is used for recording system-related information when the execution engine is scheduled;
the non-service log is used for recording system related information during task operation; and the number of the first and second groups,
the service log is used for recording algorithm related information during task operation.
Optionally, the engine log is generated at a first stage before the task runs, the non-service log is generated at a second stage of the task runs, the service log is generated at a third stage of the task runs, and the second stage and the third stage are sequentially executed according to a time sequence.
Optionally, the plurality of sub-log files includes an engine log and a non-traffic log,
the service log is used for recording algorithm related information during task operation.
The matching module is further configured to sequentially match from back to front from the last non-service log.
Optionally, the plurality of sub-log files includes an engine log, a non-traffic log and a traffic log,
the matching module is further used for sequentially matching from back to front from the last service log.
Optionally, the matching module is further configured to report an error to the scheduler by the execution engine when the task fails to run, and the scheduler performs the steps of matching and displaying to the front end.
Optionally, the device further comprises an operation module,
and the operation module is used for acquiring a preset repair program corresponding to the displayed error reporting reason keyword and operating the repair program.
Optionally, the running module is further configured to prompt a user whether to execute one-touch repair before running the repair program, and execute the repair program again when the user confirms.
Optionally, the running module is further configured to write a process of running the repair program into a running log of the task.
Optionally, a communication connection is established with the front end through a Websocket protocol.
Optionally, the apparatus further comprises a sending module,
the acquisition module is further used for acquiring a preset running state capture rule model corresponding to the type of the task according to the type of the task;
and the sending module is used for acquiring the running state information of the task by the running state capturing rule model when the task runs and sending the running state information to the front end for displaying.
Optionally, the operation log comprises a traffic log, the task is a GBDT algorithm training task,
the acquisition module is further configured to position tree building information in the service log by using the operating state capture rule model to acquire tree building start time, tree building end time, GBDT algorithm effect, resource consumption of the task, and the number of pieces of processing data corresponding to the task in the tree building information;
and the sending module is also used for drawing the tree building information and sending the tree building information to the front end in real time for displaying.
Optionally, the execution log comprises a service log, the task is a feature extraction task,
the acquisition module is further configured to locate, by using the operating state capture rule model, processing information of each line of data and each feature method in the service log to acquire processing start time, processing end time, whether a feature method is effective, resources consumed by a task of the task, and an effective proportion of the feature method in the processing information;
and the sending module is also used for drawing the processing information and sending the drawing information to the front end in real time for displaying.
Optionally, the log of runs comprises a log of traffic,
the sending module is further configured to push the service log received in real time to the front end when the task runs, so that the front end analyzes the service log and performs real-time display.
Optionally, the operation log includes a service log, and the apparatus further includes a parsing module.
The receiving module is used for receiving a service log generated when a task pushed by a scheduling engine runs, wherein the service log is used for recording algorithm-related information when the execution engine executes a corresponding operator to run the task;
and the analysis module analyzes the service log and displays the service log to a user in real time so that the user can determine whether to continue to execute the task according to the displayed running state of the task.
Optionally, the apparatus further comprises a determining module,
the judging module is used for checking a service log generated during the task running in real time;
judging whether the running state of the task meets a preset task ending running condition or not according to the service log;
and under the condition that the running state of the task meets the preset task ending running condition, ending the running of the task.
According to a third aspect of the present disclosure, there is also provided an apparatus comprising at least one computing device and at least one storage device, wherein the at least one storage device is configured to store instructions for controlling the at least one computing device to perform the method according to the above first aspect.
According to a fourth aspect of the present disclosure, there is also provided a computer readable storage medium, wherein a computer program is stored thereon, which when executed by a processor, implements the method as described above in the first aspect.
According to the method disclosed by the embodiment of the disclosure, the summarized error reporting reason keywords and the error reporting rules corresponding to the error reporting reason keywords can be pre-stored, and when the task operation fails, the task operation log is matched according to the stored error reporting rules, so that the error reporting reason keywords corresponding to the matched error reporting rules are displayed to the front end. Due to the fact that the corresponding relation between the error reporting rule and the error reporting reason keyword is stored in advance, when a task fails to operate, the method can directly match the operation log according to the summarized error reporting rule so as to display the error reporting reason keyword corresponding to the successfully matched error reporting rule to a user, avoid manual problem finding, improve the efficiency of detecting the operation failure reason, and achieve automatic exploration of the operation failure reason and visual display of the operation failure reason.
Drawings
Other features of the present invention and advantages thereof will become apparent from the following detailed description of exemplary embodiments thereof, which proceeds with reference to the accompanying drawings.
FIG. 1 is a schematic block diagram showing a hardware configuration of an electronic device that may be used to implement an embodiment of the present disclosure;
FIG. 2 is a flowchart illustrating a method for processing a task execution log according to an embodiment of the disclosure;
FIG. 3 is a flowchart illustrating a method for processing a task execution log according to another embodiment of the disclosure;
fig. 4 shows a functional block diagram of a processing device for task execution logs according to an embodiment of the present disclosure.
Detailed Description
Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.
The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.
Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.
In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.
Various embodiments and examples according to embodiments of the present invention are described below with reference to the accompanying drawings.
< hardware configuration >
The method of the embodiments of the present disclosure may be implemented by at least one electronic device, i.e., the apparatus 4000 for implementing the method may be disposed on the at least one electronic device. Fig. 1 shows a hardware structure of an arbitrary electronic device. The electronic device shown in fig. 1 may be a portable computer, a desktop computer, a workstation, a server, or the like, or may be any other device having a computing device such as a processor and a storage device such as a memory, and is not limited herein.
As shown in fig. 1, the electronic device 1000 may include a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, an input device 1600, a speaker 1700, a microphone 1800, and the like. Wherein the processor 1100 is adapted to execute computer programs. The computer program may be written in an instruction set of an architecture such as x86, Arm, RISC, MIPS, SSE, etc. The memory 1200 includes, for example, a ROM (read only memory), a RAM (random access memory), a nonvolatile memory such as a hard disk, and the like. The interface device 1300 includes, for example, a USB interface, a headphone interface, and the like. The communication device 1400 is capable of wired or wireless communication, for example, and may specifically include Wifi communication, bluetooth communication, 2G/3G/4G/5G communication, and the like. The display device 1500 is, for example, a liquid crystal display panel, a touch panel, or the like. The input device 1600 may include, for example, a touch screen, a keyboard, a somatosensory input, and the like. The electronic device 1000 may output voice information through the speaker 1700, and may collect voice information through the microphone 1800, and the like.
The electronic device shown in fig. 1 is merely illustrative and is in no way meant to limit the invention, its application, or uses. In an embodiment of the present disclosure, the memory 1200 of the electronic device 1000 is configured to store instructions for controlling the processor 1100 to operate so as to execute the processing method of the task execution log according to the embodiment of the present disclosure. The skilled person can design the instructions according to the disclosed solution. How the instructions control the operation of the processor is well known in the art and will not be described in detail herein.
In one embodiment, an apparatus is provided that includes at least one computing device and at least one storage device to store instructions to control the at least one computing device to perform a method according to any embodiment of the present disclosure.
The apparatus may include at least one electronic device 1000 as shown in fig. 1 to provide at least one computing device, such as a processor, and at least one storage device, such as a memory, without limitation.
< method examples >
In this embodiment, a method for processing a task execution log is provided, where the method for processing a task execution log may be implemented by an electronic device, the electronic device may be the electronic device 1000 shown in fig. 1, and the electronic device 1000 may be a server or a terminal device. That is, the method of this embodiment may be implemented by a server, or may be implemented by a terminal device, or may be implemented by both the server and the terminal device, and the front end in this embodiment may represent the terminal device.
In the application of the method of the embodiment in which the terminal device participates in implementation, the interaction may include human-computer interaction. In the application in which the method of the embodiment is implemented with a server, the interaction may include interaction between the server and the terminal device.
As shown in fig. 2, the method for processing the task execution log according to this embodiment may include the following steps S2100 to S2300:
step S2100, acquiring and saving a preset error reporting reason keyword and a corresponding error reporting rule.
The error reporting rule is used for defining keywords, and the error reporting rule can be manually summarized and generalized based on historical logs describing the reasons of error reporting corresponding to task operation failure. The error reporting rule may include any one or more of a single keyword error report, a plurality of keywords being satisfied simultaneously and not being reported in the same row, and a plurality of keywords being satisfied simultaneously and being reported in the same row (in order). Wherein the plurality of keywords may be different keywords.
The error reporting reason keywords are keywords screened out based on error reporting rules, and can include any one or more of insufficient memory, license overrun, algorithm parameter configuration error, script syntax error, cluster time zone asynchronism and authority insufficiency. Wherein, an error reporting reason keyword can correspond to an error reporting rule, and an error reporting rule can correspond to at least one error reporting reason keyword, and specifically, error reporting reason keyword and error reporting rule include at least one of the following:
if the memory is insufficient, the error is reported corresponding to a single keyword;
the license is out of limit, and correspondingly meets a plurality of keywords at the same time without error report in the same row;
the algorithm parameter configuration is wrong, and correspondingly meets a plurality of keywords at the same time and does not report errors in the same row;
script grammar error, which corresponds to a plurality of keywords simultaneously and does not report error in the same line;
cluster time zones are asynchronous, a plurality of keywords are correspondingly met at the same time, and errors are not reported in the same row;
if the authority is not enough, the corresponding single keyword is wrongly reported.
The corresponding relation reflects the mapping relation between the error reporting reason key words and the error reporting rules. The mapping relationship may be a mapping table, and an index relationship between the error reporting reason keyword and the error reporting rule is stored in the mapping table. It can be understood that after the error reporting rule processing, thirty or more error reporting reason keywords are preset, and the coverage rate of the error reporting reason keywords is more than 80%.
In this embodiment, after the preset error reporting reason keyword and the corresponding error reporting rule are acquired and saved according to the above step S2100, the following steps are entered:
step S2200, when the task fails to run, matching the task running log according to the saved error reporting rule.
Examples of the above tasks include, but are not limited to, tasks such as an algorithm training task, a data processing task, a data splitting task, a feature extraction task, an algorithm prediction task, an algorithm evaluation task, and the like.
In an actual application scenario, when the artificial intelligence task runs, the time sequence may include: 1. submitting tasks to a computing cluster; 2. the cluster receives the task and carries out scheduling; 3. if the dispatching is successful, calculating the processing logic of the cluster for starting to run the task; 4. the processing logic comprises service state information directly related to the algorithm and the data, and also comprises non-service state information. That is to say, the artificial intelligence task naturally has a subtask time sequence, and here, the operation logs in the task can be classified and collected according to the sequence of the execution steps in the task, so as to obtain a plurality of sub-log files, so that when the task fails to operate, the plurality of sub-log files are matched according to the saved error reporting rule, and the error reporting reason keyword corresponding to the successfully matched error reporting rule is displayed to the front end.
In this embodiment, when the task fails to run in step S2200, matching the task running log according to the saved error reporting rule may further include: when the task runs, matching is carried out according to the reverse order of the generation order of the plurality of sub-log files, so that the matching time is saved, and the matching efficiency is improved.
The above plurality of sub-log files may include an engine log, a non-traffic log, and a traffic log. The engine log is used for recording system related information when the execution engine is scheduled; the non-service log is used for recording system related information during task operation; and the service log is used for recording algorithm related information during the task operation. The engine log is generated in the first stage before the task is operated, the non-service log is generated in the second stage of the task operation, the service log is generated in the third stage of the task operation, and the second stage and the third stage are executed in sequence according to the time sequence. It is understood that chronological execution herein is not strictly non-coincident serialization, e.g., engine logs begin at 10, non-service logs begin at 10:20, service logs begin at 10:30, and engine logs continue to be recorded without ending.
It will be appreciated that during the running of a task, if any previous step fails, the subsequent step will not be run, but will be returned directly to the error log, so that the last log, which must be the error (error) log, may only be the one that has failed at all, but may start from the first warning (warning), which is different from error and will not result in the termination of the running, but in most cases the last error log is the error at all, and thus, it may be the stage where the task running failure occurs according to the last error log. For example, if the last error log is the engine log, the task fails in the first stage; for another example, if the last error log is a non-service log, the task fails in the second stage; for another example, if the last error log is a service log, the task fails in the third stage.
In this embodiment, the engine log, the non-service log and the service log may be respectively generated with a mark corresponding to a log type, so that a worker may view log files of different types according to the mark.
In one example, the sub-log file may include only the engine log. When the task fails to run, the matching according to the reverse order of the generation order of the plurality of sub-log files may further include: when the task fails to run, the engine logs recorded from the last engine log of the sub-log file are sequentially matched from back to front.
In one example, the sub-log file may include an engine log and a non-traffic log. When the task fails to run, the matching according to the reverse order of the generation order of the plurality of sub-log files may further include: and when the task fails to run, sequentially matching the last non-service log recorded by the sub-log file from back to front.
In this example, the non-service logs may be sequentially matched from back to front starting from the last non-service log, and sequentially matched from back to front starting from the last engine log, or the non-service logs and the engine logs may be cross-matched from back to front starting from the last non-service log.
In one example, the sub-log file may include an engine log, a non-traffic log, and a traffic log. When the task fails to run, the matching according to the reverse order of the generation order of the plurality of sub-log files may further include: and when the task fails to run, sequentially matching the last service log recorded by the sub-log file from back to front.
In this example, the service logs, the non-service logs and the engine logs may be sequentially matched from the last service log from back to front, sequentially matched from the last non-service log from back to front, and sequentially matched from the last engine log from back to front, or alternatively, the service logs, the non-service logs and the engine logs may be cross-matched from the last service log from back to front.
The above example starts from the sub-log file with the latest recording time, matches the error reporting rule from the last line of the sub-log file from back to front, and if the matching is successful, directly feeds the error reporting reason keyword corresponding to the successfully matched error reporting rule back to the front end for displaying, thereby realizing automatic analysis and automation of matching the error reporting reason.
In this embodiment, when the task fails to run according to the step S2200, after matching the task running log according to the saved error reporting rule, the following steps are performed:
step S2300, displaying the error reporting reason keyword corresponding to the successfully matched error reporting rule to the front end.
In this embodiment, the execution engine and the scheduler may be installed in the server, and when the task fails to run, the execution engine reports an error to the scheduler, and the scheduler performs the steps of matching and exposing to the front end. That is, when the task fails to run, the execution engine reports an error to the scheduler, the scheduling engine performs the matching of the task running log according to the stored error reporting rule in step S2200, and the scheduling engine performs step S2300 to display the error reporting reason keyword corresponding to the successfully matched error reporting rule to the front end.
In an example, the error reporting rule that is successfully matched may be an error reporting rule, for example, the error reporting rule that is successfully matched may be a single keyword error reporting, and the error reporting reason keyword corresponding to the error reporting rule that is successfully matched includes insufficient memory and insufficient authority, where the insufficient memory and the insufficient authority are displayed to the front end. For example, the error reporting rule that is successfully matched may simultaneously satisfy a plurality of keywords and does not report errors in the same row, and the error reporting cause keywords corresponding to the error reporting rule that is successfully matched include license overrun, algorithm parameter configuration error, and cluster time zone asynchronization, where license overrun, algorithm parameter configuration error, and cluster time zone are asynchronously displayed to the front end.
In an example, the error reporting rule that is successfully matched may also be multiple error reporting rules, for example, the error reporting rules include a single keyword error reporting and a plurality of keywords that are simultaneously satisfied and are not reported in the same row, the error reporting reason keyword corresponding to the single keyword error reporting includes insufficient memory and insufficient authority, and the error reporting reason keyword corresponding to the keywords that are simultaneously satisfied and are not reported in the same row includes license overrun, algorithm parameter configuration error, and asynchronous cluster time zone, where the insufficient memory, the insufficient authority, license overrun, algorithm parameter configuration error, and asynchronous cluster time zone may be displayed to the front end.
According to the method disclosed by the embodiment of the disclosure, the summarized error reporting reason keywords and the error reporting rules corresponding to the error reporting reason keywords can be pre-stored, and when the task operation fails, the task operation log is matched according to the stored error reporting rules, so that the error reporting reason keywords corresponding to the matched error reporting rules are displayed to the front end. Due to the fact that the corresponding relation between the error reporting rule and the error reporting reason keyword is stored in advance, when a task fails to operate, the method can directly match the operation log according to the summarized error reporting rule so as to display the error reporting reason keyword corresponding to the successfully matched error reporting rule to a user, avoid manual problem finding, improve the efficiency of detecting the operation failure reason, and achieve automatic exploration of the operation failure reason and visual display of the operation failure reason.
In one embodiment, after the error-reporting reason keyword is displayed to the front end, the task is repaired based on the repair program corresponding to the error-reporting reason keyword, so that the task can continue to run. In this embodiment, the method for processing the task execution log may further include:
and acquiring a preset repairing program corresponding to the displayed error-reporting reason keywords, and operating the repairing program.
The above repair procedures are manually summarized and concluded, and the repair procedures may be repair strategies. In this embodiment, mapping data indicating a mapping relationship between each error-reporting-cause keyword and the corresponding repair program may be stored in advance, where the mapping data may be a mapping table in which an index relationship between the error-reporting-cause keyword and the repair program is stored.
For example, the keyword for the error reporting reason is "insufficient memory", and the corresponding repairing program is "check the current operator resource configuration, generally, the memory resource required for operator calculation is greater than the currently set memory resource, and may be increased on the basis of the auxiliary resource recommendation result, for example, increase the driver/execution memory by 50%, or increase the spare.yann.execution.memoryoverhead (default is 10% of execution, and may be appropriately doubled to 20%), and whether num is down-regulated to consider the expected operating speed and the cluster resource configuration.
For example, the keyword for error reporting reason is "license overrun", and the corresponding repairing program is "please wait for the total running resources to run the task after being under license limit".
For another example, the error-reporting-cause keyword is "script syntax error", the script is, for example, but not limited to, Sql language, Perl language, Python language, Ruby language, etc., and the corresponding repair program is "Sql operator follows SparkSQL syntax and please check the correctness of the Sql script" taking the Sql script syntax error as an example.
For another example, the keyword for the reason of error reporting is "cluster time zone is not synchronous", and the corresponding repairing program is "please synchronize the cluster time zone and then run the task".
For example, the keyword for the error reason is "permission is insufficient", and the corresponding repairing program is "you cannot run the task in the current permission, please contact the administrator for processing".
In this embodiment, a mapping table storing mapping relationships between the error-reporting-cause keywords and the corresponding repair programs may be obtained first, so as to obtain the repair programs corresponding to the error-reporting-cause keywords according to the mapping table and the displayed error-reporting-cause keywords, and run the repair programs, so that the task can continue to run.
In this embodiment, the process of running the repair program may also be written into the running log of the task, so that the user can know the running process of the repair program in real time.
In one embodiment, a human-machine interface is also provided to determine whether to perform a one-touch fix based on current actual needs to improve the flexibility of task fixes. In this embodiment, the method for processing the task execution log further includes:
and prompting the user whether to execute one-key repair before running the repair program, and executing the repair program when the user confirms.
In this embodiment, the electronic device may provide an analog switch for selecting whether to perform one-key repair, and when the switch is in an on state, it indicates that the user determines to perform one-key repair, and further performs a repair program; alternatively, when the switch is in the off state, it indicates that the user is not performing one-touch repair. Of course, the electronic device may also provide a setting interface to set whether the user performs the one-key repair through the setting interface, for example, the setting interface may include at least one form of interface of an input box, a drop-down list, a hook option, a voice input, and the like, so that the user may select whether to perform the one-key repair as needed.
In one embodiment, during the running process of the task, the business log of the task can be visually displayed, so that a user can visually view the business log of the task. In this embodiment, the method for processing the task execution log may include:
and when the task runs, pushing the service log received in real time to the front end so that the front end analyzes the service log and displays the service log in real time.
In this embodiment, a communication connection may be established with the front end through a Websocket protocol, the server may actively send the service log to the front end based on the connection, and after receiving the service log, the front end may analyze the service log and display the analyzed service log in real time.
In one embodiment, when the task runs, the running state information of the business log of the task can be visually displayed, so that a user can visually view the current running state of the task. As shown in fig. 3, the method for processing the task execution log according to the embodiment of the present disclosure further includes the following steps S3100 to S3200:
step S3100, acquiring a preset running state capture rule model corresponding to the type of the task according to the type of the task.
The operation state capturing rule model is used for obtaining operation state information of the corresponding task. In this embodiment, mapping data representing a mapping relationship between each type of task and the corresponding operation state capture rule model may be stored in advance, where the mapping data may be a mapping table in which an index relationship between the task and the operation state capture rule model is stored.
In this embodiment, different tasks correspond to different operation state capture rule models, and the same task also corresponds to different operation state capture rule models according to different algorithm principles.
For example, an algorithm training task, a data processing task, a data splitting task, a feature extraction task, an algorithm prediction task, an algorithm evaluation task and the like all correspond to different operation state capture rule models.
For another example, the algorithm training task includes an LR (logistic regression) algorithm training task, a GBDT (gradient boosting decision tree) algorithm training task, and the like, and the LR algorithm training task and the GBDT algorithm training task correspond to different operation state capture rule models.
Step S3200, when the task runs, the running state information of the task is obtained by the running state capturing rule model and is sent to the front end for displaying.
In this embodiment, a communication connection may be established with the front end through a Websocket protocol, and the server may actively send the running state information of the task to the front end based on the connection, so that the running state information is displayed by the front end.
In one example, the above task is a feature extraction task, and the step S3200 of obtaining the operation state information of the task by the operation state capture rule model and sending the operation state information to the front end for presentation may further include the following steps S3210a to S3220 a:
step S3210a, locating the tree building information in the service log by using the operation state capture rule model to obtain tree building start time, tree building end time, GBDT algorithm effect, resource consumption of the task, and the number of processed data corresponding to the task in the tree building information.
Step S3220a, the tree building information is plotted and sent to the front end in real time for display.
In step S3220a, the drawn graphs include, for example, but not limited to, a time-series line graph showing the operation phases, a pie graph showing the resource consumption ratios of the feature methods, a bar graph showing the effective ratios of the feature methods, and the like.
In an example, the above task is a GBDT algorithm training task, and the step S3200 of obtaining the running state information of the task by the running state capture rule model and sending the running state information to the front end for presentation may further include the following steps S3210b to S3220 b:
step S3210b, locating the tree building information in the service log by using the operation state capture rule model to obtain tree building start time, tree building end time, GBDT algorithm effect, resource consumption of the task, and the number of processed data corresponding to the task in the tree building information.
Step S3220b, the tree building information is plotted and sent to the front end in real time for display.
In one embodiment, it can determine whether to continue to execute the task according to the selection result of the user so as to realize the targeting of the task running. In this embodiment, the operation log includes a service log, and the processing method of the task operation log may further include the following steps S4100 to S4200:
step S4100, receiving a service log generated when the task pushed by the scheduling engine runs.
The service log is used for recording algorithm related information when the execution engine executes a corresponding operator to run a task.
Step S4200, analyzing the service log and displaying the service log to the user in real time, so that the user can determine whether to continue to execute the task according to the displayed running state of the task.
In one embodiment, the method presets the end running condition of the task, and actively ends the running of the task when the end condition is met so as to realize automatic customization. In this embodiment, the operation log includes a service log, and the method for processing the task operation log may further include the following steps S5100 to S5200:
step S5100, checks the service log generated during the task running in real time.
Step S5200, determining whether the running state of the task meets a preset task ending running condition according to the service log.
In step S5300, the operation of the task is terminated when the operation state of the task satisfies a preset task termination operation condition.
< apparatus embodiment >
In this embodiment, a processing apparatus 4000 for task execution log is provided, as shown in fig. 4, including an obtaining module 4100, a matching module 4200, and a display module 4300.
The obtaining module 4100 is configured to obtain and store a preset error reporting reason keyword and a corresponding error reporting rule.
The matching module 4200 is configured to match the task running log according to the saved error reporting rule when the task fails to run.
The display module 4300 is configured to display the error reporting reason keyword corresponding to the successfully matched error reporting rule to the front end.
In one embodiment, the error reporting reason keyword and the corresponding error reporting rule include at least one of the following:
if the memory is insufficient, the error is reported corresponding to a single keyword;
the license is out of limit, and correspondingly meets a plurality of keywords at the same time without error report in the same row;
the algorithm parameter configuration is wrong, and correspondingly meets a plurality of keywords at the same time and does not report errors in the same row;
script grammar error, which corresponds to a plurality of keywords simultaneously and does not report error in the same line;
cluster time zones are asynchronous, a plurality of keywords are correspondingly met at the same time, and errors are not reported in the same row;
if the authority is not enough, the corresponding single keyword is wrongly reported.
In one embodiment, the apparatus 4000 further comprises a classification module (not shown).
The classification module is used for classifying and collecting the running logs of the tasks according to the sequence of the execution steps in the tasks, so that a plurality of sub-log files are obtained.
The matching module 4200 is further configured to perform matching according to a reverse order of the generation order of the plurality of sub-log files.
In one embodiment, the plurality of sub-log files comprises: engine logs, non-traffic logs, and traffic logs.
In one embodiment, the engine log is used to record system-related information when an execution engine is scheduled.
The non-service log is used for recording system related information during task operation; and the number of the first and second groups,
the service log is used for recording algorithm related information during task operation.
In one embodiment, the engine log is generated at a first stage before the task is run, the non-service log is generated at a second stage of the task, the service log is generated at a third stage of the task, and the second stage and the third stage are sequentially executed according to a time sequence.
In an embodiment, the matching module 4200 is further configured to match sequentially from back to front starting from the last non-service log.
In one embodiment, the plurality of sub-log files includes an engine log, a non-traffic log, and a traffic log.
The matching module 4200 is further configured to match sequentially from back to front starting from the last service log.
In one embodiment, the matching module is further configured to report an error to a scheduler by the execution engine when the task fails to run, and the scheduler performs the steps of matching and exposing to the front end.
In one embodiment, the apparatus 4000 further comprises a running module (not shown).
And the operation module is used for acquiring a preset repair program corresponding to the displayed error reporting reason keyword and operating the repair program.
In one embodiment, the running module is further configured to prompt a user whether to execute one-touch repair before running the repair program, and execute the repair program again when the user confirms.
In an embodiment, the execution module is further configured to write a process of executing the repair program into the execution log of the task.
In one embodiment, a communication connection is established with the head-end via the Websocket protocol.
In one embodiment, the apparatus 4000 further comprises a sending module (not shown in the figures).
The obtaining module 4100 is further configured to obtain a preset running state capture rule model corresponding to the type of the task according to the type of the task.
And the sending module is used for acquiring the running state information of the task by the running state capturing rule model when the task runs and sending the running state information to the front end for displaying.
In one embodiment, the operation log comprises a traffic log, and the task is a GBDT algorithm training task.
The obtaining module 4100 is further configured to locate tree building information in the service log by using the operation state capture rule model, so as to obtain tree building start time, tree building end time, GBDT algorithm effect, resource consumption of the task, and the number of processed data corresponding to the task in the tree building information.
And the sending module is also used for drawing the tree building information and sending the tree building information to the front end in real time for displaying.
In one embodiment, the execution log comprises a business log, and the task is a feature extraction task.
The obtaining module 4100 is further configured to locate, by using the running state capture rule model, processing information of each line of data and each feature method in the service log, so as to obtain processing start time, processing end time, whether a feature method is effective, resources consumed by a task of the task, and an effective proportion of the feature method in the processing information.
And the sending module is also used for drawing the processing information and sending the drawing information to the front end in real time for displaying.
In one embodiment, the log of runs comprises a log of traffic.
The sending module is further configured to push the service log received in real time to the front end when the task runs, so that the front end analyzes the service log and performs real-time display.
In one embodiment, the log includes a service log, and the apparatus 4000 further includes a parsing module (not shown).
The receiving module is used for receiving a service log generated when a task pushed by a scheduling engine runs, wherein the service log is used for recording algorithm related information when the execution engine executes a corresponding operator to run the task.
And the analysis module analyzes the service log and displays the service log to a user in real time so that the user can determine whether to continue to execute the task according to the displayed running state of the task.
In one embodiment, the apparatus 4000 further comprises a determination module (not shown).
The judging module is used for checking a service log generated during the task running in real time; judging whether the running state of the task meets a preset task ending running condition or not according to the service log; and under the condition that the running state of the task meets the preset task ending running condition, ending the running of the task.
< storage Medium embodiment >
The present embodiment provides a computer-readable storage medium, wherein a computer program is stored thereon, which computer program, when being executed by a processor, realizes the method according to any one of the above-mentioned method embodiments.
The present invention may be an apparatus, method and/or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied therewith for causing a processor to implement various aspects of the present invention.
The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.
The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.
The computer program instructions for carrying out operations of the present invention may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present invention are implemented by personalizing an electronic circuit, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA), with state information of computer-readable program instructions, which can execute the computer-readable program instructions.
Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, by software, and by a combination of software and hardware are equivalent.
Having described embodiments of the present invention, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.
Claims (10)
1. A method for processing task running logs is characterized by comprising the following steps:
acquiring and storing preset error reporting reason keywords and corresponding error reporting rules;
when the task operation fails, matching the task operation log according to the saved error reporting rule;
and displaying the error reporting reason keywords corresponding to the successfully matched error reporting rules to the front end.
2. The method of claim 1, wherein the error cause keyword and the corresponding error rules comprise at least one of:
if the memory is insufficient, the error is reported corresponding to a single keyword;
the license is out of limit, and correspondingly meets a plurality of keywords at the same time without error report in the same row;
the algorithm parameter configuration is wrong, and correspondingly meets a plurality of keywords at the same time and does not report errors in the same row;
script grammar error, which corresponds to a plurality of keywords simultaneously and does not report error in the same line;
cluster time zones are asynchronous, a plurality of keywords are correspondingly met at the same time, and errors are not reported in the same row;
if the authority is not enough, the corresponding single keyword is wrongly reported.
3. The method of claim 1, wherein the method further comprises: according to the sequence of the execution steps in the task, classifying and collecting the running logs of the task to obtain a plurality of sub-log files;
the matching the task running log according to the saved error reporting rule comprises: and matching according to the reverse order of the generation sequence of the plurality of sub-log files.
4. The method of claim 3, wherein,
the plurality of sub-log files includes: engine logs, non-traffic logs, and traffic logs.
5. The method of claim 4, wherein,
the engine log is used for recording system related information when the execution engine is scheduled;
the non-service log is used for recording system related information during task operation; and the number of the first and second groups,
the service log is used for recording algorithm related information during task operation.
6. The method of claim 4, wherein the engine log is generated at a first stage before the task runs, the non-service log is generated at a second stage of the task runs, the service log is generated at a third stage of the task runs, and the second stage and the third stage are executed in time sequence.
7. The method of claim 4, wherein the plurality of sub-log files comprise an engine log and a non-traffic log,
the matching according to the reverse order of the generation order of the plurality of sub-log files includes:
and sequentially matching from the last non-service log from back to front.
8. A processing device for task execution logs, comprising:
the acquisition module is used for acquiring and storing preset error reporting reason keywords and corresponding error reporting rules;
the matching module is used for matching the task running log according to the stored error reporting rule when the task fails to run;
and the display module is used for displaying the error reporting reason keywords corresponding to the successfully matched error reporting rules to the front end.
9. An apparatus comprising at least one computing device and at least one storage device, wherein the at least one storage device is to store instructions for controlling the at least one computing device to perform the method of any of claims 1-7.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010340464.2A CN111611127B (en) | 2020-04-26 | 2020-04-26 | Task running log processing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010340464.2A CN111611127B (en) | 2020-04-26 | 2020-04-26 | Task running log processing method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111611127A true CN111611127A (en) | 2020-09-01 |
| CN111611127B CN111611127B (en) | 2023-10-31 |
Family
ID=72199997
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010340464.2A Active CN111611127B (en) | 2020-04-26 | 2020-04-26 | Task running log processing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111611127B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113672418A (en) * | 2021-08-02 | 2021-11-19 | 北京每日优鲜电子商务有限公司 | Data processing task detail page display method and device, electronic equipment and medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104750747A (en) * | 2013-12-30 | 2015-07-01 | 腾讯科技（深圳）有限公司 | Method and system for converting error messages into error prompt |
| CN106201754A (en) * | 2016-07-06 | 2016-12-07 | 乐视控股（北京）有限公司 | Mission bit stream analyzes method and device |
| US20180157713A1 (en) * | 2016-12-02 | 2018-06-07 | Cisco Technology, Inc. | Automated log analysis |
| CN108681598A (en) * | 2018-05-21 | 2018-10-19 | 平安科技（深圳）有限公司 | Task runs method, system, computer equipment and storage medium again automatically |
| CN109976933A (en) * | 2019-02-22 | 2019-07-05 | 视联动力信息技术股份有限公司 | A kind of log processing method and device |
| CN110515912A (en) * | 2019-07-18 | 2019-11-29 | 湖南星汉数智科技有限公司 | Log processing method, device, computer installation and computer readable storage medium |
| CN110851324A (en) * | 2019-10-25 | 2020-02-28 | 泰康保险集团股份有限公司 | Log-based routing inspection processing method and device, electronic equipment and storage medium |
-
2020
- 2020-04-26 CN CN202010340464.2A patent/CN111611127B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104750747A (en) * | 2013-12-30 | 2015-07-01 | 腾讯科技（深圳）有限公司 | Method and system for converting error messages into error prompt |
| CN106201754A (en) * | 2016-07-06 | 2016-12-07 | 乐视控股（北京）有限公司 | Mission bit stream analyzes method and device |
| US20180157713A1 (en) * | 2016-12-02 | 2018-06-07 | Cisco Technology, Inc. | Automated log analysis |
| CN108681598A (en) * | 2018-05-21 | 2018-10-19 | 平安科技（深圳）有限公司 | Task runs method, system, computer equipment and storage medium again automatically |
| CN109976933A (en) * | 2019-02-22 | 2019-07-05 | 视联动力信息技术股份有限公司 | A kind of log processing method and device |
| CN110515912A (en) * | 2019-07-18 | 2019-11-29 | 湖南星汉数智科技有限公司 | Log processing method, device, computer installation and computer readable storage medium |
| CN110851324A (en) * | 2019-10-25 | 2020-02-28 | 泰康保险集团股份有限公司 | Log-based routing inspection processing method and device, electronic equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 梅御东等: "一种基于日志信息和CNN-text的软件系统异常检测方法", 《计算机学报》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113672418A (en) * | 2021-08-02 | 2021-11-19 | 北京每日优鲜电子商务有限公司 | Data processing task detail page display method and device, electronic equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111611127B (en) | 2023-10-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109683953B (en) | Method and device for processing configuration file based on visual interface | |
| US10761964B2 (en) | Object monitoring in code debugging | |
| US20150347923A1 (en) | Error classification in a computing system | |
| CN110618940B (en) | Stack information tracking method, device, computer readable medium and computing device | |
| CN110851324B (en) | Log-based routing inspection processing method and device, electronic equipment and storage medium | |
| US9910487B1 (en) | Methods, systems and computer program products for guiding users through task flow paths | |
| CN112000806A (en) | Abnormal log monitoring and analyzing method, system, equipment and storage medium | |
| US20200371754A1 (en) | Use and advancements of assistive technology in automation for the visually-impaired workforce | |
| US9612827B2 (en) | Automatically complete a specific software task using hidden tags | |
| CN111723515A (en) | Method, device and system for operating operator | |
| CN111045911A (en) | Performance test method, performance test device, storage medium and electronic equipment | |
| US20220350690A1 (en) | Training method and apparatus for fault recognition model, fault recognition method and apparatus, and electronic device | |
| CN111708753A (en) | Method, device and equipment for evaluating database migration and computer storage medium | |
| EP4195108A1 (en) | Method and apparatus for generating and applying deep learning model based on deep learning framework | |
| CN113792341A (en) | Privacy compliance automation detection method, device, equipment and medium for application program | |
| US11409704B2 (en) | Method, device and computer program product for managing storage system | |
| CN111611127B (en) | Task running log processing method, device, equipment and storage medium | |
| CN115185797A (en) | Method and system for testing visual algorithm model, electronic equipment and storage medium | |
| CN114090514A (en) | Log retrieval method and device for distributed system | |
| CN114297057A (en) | Design and use method of automatic test case | |
| CN114661571A (en) | Model evaluation method, model evaluation device, electronic equipment and storage medium | |
| CN114546780A (en) | Data monitoring method, device, equipment, system and storage medium | |
| CN113032209A (en) | Operation monitoring method, device, server and medium | |
| CN112381167A (en) | Method for training task classification model, and task classification method and device | |
| CN111694686A (en) | Abnormal service processing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |