CN117130873B - Task monitoring method and device - Google Patents
Task monitoring method and device Download PDFInfo
- Publication number
- CN117130873B CN117130873B CN202311404935.1A CN202311404935A CN117130873B CN 117130873 B CN117130873 B CN 117130873B CN 202311404935 A CN202311404935 A CN 202311404935A CN 117130873 B CN117130873 B CN 117130873B
- Authority
- CN
- China
- Prior art keywords
- task
- predicted
- time
- historical
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 60
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000012545 processing Methods 0.000 claims abstract description 134
- 230000002159 abnormal effect Effects 0.000 claims abstract description 13
- 230000008569 process Effects 0.000 claims description 28
- 238000004422 calculation algorithm Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 9
- 238000012417 linear regression Methods 0.000 claims description 5
- 238000012549 training Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 14
- 238000005457 optimization Methods 0.000 description 14
- 230000005856 abnormality Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000001960 triggered effect Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000003111 delayed effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3017—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is implementing multitasking
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a method and a device for task monitoring, and relates to the technical field of computers. One embodiment of the method comprises the following steps: acquiring a plurality of task paths, wherein each task path comprises one or more single tasks, and the first single task and the last single task in each task path are the same; according to the historical starting time and the historical processing time which are respectively corresponding to the single task and/or the current traffic corresponding to the single task, determining the predicted starting time and the predicted processing time corresponding to each single task; determining the predicted ending time of the task path according to the predicted starting time and the predicted processing time; and generating a task monitoring result corresponding to each task path and the single task according to whether the predicted starting time, the predicted processing time and the predicted ending time meet a preset threshold. The embodiment can timely monitor whether each single task is abnormal or not so as to timely acquire a corresponding processing strategy according to a task monitoring result.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for task monitoring.
Background
In the existing task monitoring process, because the number of task paths is complex, and single tasks involved in each task path are different, the task paths are intricate and complex, whether each single task is abnormal or not cannot be monitored in time, and when the last single task of the task paths is required to be executed, whether the execution time of the whole task path is overtime is judged manually so as to determine whether the task paths are abnormal or not.
When the first single task and the last single task corresponding to the plurality of task paths are identical, the execution time of the last single task is determined by the task path with the longest execution time (for example, task path a: task 1→task 2→task 3, task path B: task 1→task 4→task 5→task 3, after task 1 is completed, task 2 and task 4 are respectively executed by the two task paths, but when task 2 is executed prior to task 5, task 3 is jointly executed after task 5 is required to be waited for being completed, that is, the execution end time of task path a and task path B is identical), therefore, when the task path execution times out, it cannot be determined which single task (task 2 or task 4 and task 5) in the specific task path (task path a or task path B) is abnormal, and the execution of the final task path is abnormal.
Disclosure of Invention
In view of the above, an embodiment of the present invention provides a method and an apparatus for task monitoring, by predicting a task start time and a task execution time of each single task, and determining a predicted end time of a task path according to the predicted start time and the predicted processing time, monitoring whether an abnormality occurs in each single task in time, so as to obtain a corresponding processing policy in time according to a task monitoring result. Meanwhile, the single task with slower execution time can be accurately positioned according to the task monitoring result of the single task, and the optimization can be carried out in a targeted manner, so that the subsequent optimization efficiency is improved.
To achieve the above object, according to one aspect of the embodiments of the present invention, there is provided a method for task monitoring.
The task monitoring method of the embodiment of the invention comprises the following steps: acquiring a plurality of task paths, wherein each task path comprises one or more single tasks, and the first single task and the last single task in each task path are the same; according to the historical starting time and the historical processing time which are respectively corresponding to the single tasks and/or the current traffic corresponding to the single tasks, determining the predicted starting time and the predicted processing time corresponding to each single task; determining the predicted ending time of the task path according to the predicted starting time and the predicted processing time; and generating each task path and a task monitoring result corresponding to the single task according to whether the predicted starting time, the predicted processing time and the predicted ending time meet a preset threshold.
Optionally, the historical starting time and the historical processing time are obtained according to the execution result of the single task executed for a plurality of times in a historical period.
Optionally, the determining the predicted starting time and the predicted processing time corresponding to each single task according to the historical starting time and the historical processing time corresponding to the single task and/or the current traffic corresponding to the single task includes: determining the execution date of the single task in a history period and the weight corresponding to each execution date respectively; and determining the predicted starting time and the predicted processing time according to the weight, the historical starting time and the historical processing time.
Optionally, the determining the predicted starting time and the predicted processing time corresponding to each single task according to the historical starting time and the historical processing time corresponding to the single task and/or the current traffic corresponding to the single task includes: acquiring the current traffic corresponding to the single task; determining the prediction processing time according to a pre-trained service processing model and the current traffic; the business processing model is obtained through training of a linear regression algorithm, and the corresponding relation between the historical business volume and the historical processing time is indicated.
Optionally, the determining the predicted ending time of the task path according to the predicted starting time and the predicted processing time includes: determining the execution sequence of a plurality of single tasks in each task path; determining target prediction ending time corresponding to each task path respectively according to the execution sequence, the prediction starting time corresponding to each single task and the prediction processing time by using a dynamic programming algorithm; and determining the predicted ending time from a plurality of target predicted ending times.
Optionally, the step of determining a predicted end time of the task path according to the predicted start time and the predicted processing time is performed according to a preset time period;
the determining, by using a dynamic programming algorithm, a target prediction end time corresponding to each task path according to the execution sequence, the prediction start time corresponding to each single task, and the prediction processing time, includes: and determining target prediction ending time of the task path according to the actual ending time of the first single task, the execution sequence of the non-executed one or more second single tasks and the prediction processing time corresponding to the second single task aiming at the situation that one or more first single tasks in the task path are executed.
Optionally, the predicted starting time, the predicted processing time and the predicted ending time are respectively and correspondingly provided with different preset thresholds; generating a task monitoring result corresponding to each task path and the single task according to whether the predicted starting time, the predicted processing time and the predicted ending time meet a preset threshold value, including: and determining a task monitoring result indicating abnormal task processing when at least one of the predicted starting time, the predicted processing time and the predicted ending time exceeds a corresponding preset threshold value.
To achieve the above object, according to still another aspect of the embodiments of the present invention, there is provided an apparatus for task monitoring.
The device for task monitoring in the embodiment of the invention comprises the following components: the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a plurality of task paths, each task path comprises one or more single tasks, and the first single task and the last single task in each task path are the same; the determining module is used for determining the predicted starting time and the predicted processing time corresponding to each single task according to the historical starting time and the historical processing time corresponding to the single task respectively and/or the current traffic corresponding to the single task; the prediction module is used for determining the predicted ending time of the task path according to the predicted starting time and the predicted processing time; and the result module is used for generating a task monitoring result corresponding to each task path and the single task according to whether the predicted starting time, the predicted processing time and the predicted ending time meet a preset threshold value.
To achieve the above object, according to still another aspect of the embodiments of the present invention, there is provided an electronic device for task monitoring.
The electronic equipment for task monitoring in the embodiment of the invention comprises: one or more processors; and the storage device is used for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are enabled to realize the task monitoring method according to the embodiment of the invention.
To achieve the above object, according to still another aspect of the embodiments of the present invention, there is provided a computer-readable storage medium.
A computer readable storage medium of an embodiment of the present invention has stored thereon a computer program which, when executed by a processor, implements a method of task monitoring of an embodiment of the present invention.
One embodiment of the above invention has the following advantages or benefits: the task starting time and the task executing time of each single task are predicted, the predicted ending time of the task path is determined according to the predicted starting time and the predicted processing time, whether the abnormality occurs in each single task is monitored in time, and the corresponding processing strategy is obtained in time according to the task monitoring result. Meanwhile, the single task with slower execution time can be accurately positioned according to the task monitoring result of the single task, and the optimization can be carried out in a targeted manner, so that the subsequent optimization efficiency is improved.
Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic flow diagram of a method of task monitoring according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a main flow of determining a predicted start-up time and a predicted processing time according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of another main flow of determining a predicted start-up time and a predicted processing time according to an embodiment of the present invention;
FIG. 4 is a schematic flow chart of a main process for determining a predicted end time of a task path according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of an array construction of a dynamic programming algorithm according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of the major modules of an apparatus for task monitoring according to an embodiment of the present invention;
FIG. 7 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;
fig. 8 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
It should be noted that the embodiments of the present invention and the technical features in the embodiments may be combined with each other without collision.
It should be noted that, in the technical solution of the present disclosure, the related aspects of collecting, updating, analyzing, processing, using, transmitting, storing, etc. of the personal information of the user all conform to the rules of the related laws and regulations, and are used for legal purposes without violating the public order colloquial. Necessary measures are taken for the personal information of the user, illegal access to the personal information data of the user is prevented, and the personal information security, network security and national security of the user are maintained.
In the prior art, each single task is usually executed only once, but may exist in a plurality of different task paths, so that for each single task, it is required that the previous other single tasks can be executed after all the previous single tasks are executed. For example, task path a: task 1→task 2→task 3, task path B: task 1- & gt task 4- & gt task 5- & gt task 3, namely task 1 and task 3 are included in both task path A and task path B, and since task 1 is the starting task, there is no case of waiting for other tasks, but task 3 is the ending task, and execution can be performed only after task 2 in task path A and task 5 in task path B are required to be executed. That is, after the task 1 is completed, the two task paths start to execute the task 2 and the task 4 respectively, and for different tasks, the execution time is usually different, and the execution time of the task 2 may be very short, and after the task 2 is completed, the task 4 is not completed yet, that is, the task path a needs to wait for the task 4 and the task 5 in the task path B to be completed in sequence, and then the task 3 can be executed together. It can be understood that when the first single task and the last single task corresponding to the task paths are the same, the execution end time of the task paths is the same. Therefore, when the execution end time is timed out, the conventional technology cannot determine whether an abnormality has occurred in the task path a or the task path B, and more specifically, whether an abnormality has occurred in the task path a or the task path B.
In view of this, the present invention provides a task monitoring method, and fig. 1 is a schematic diagram of main steps of a task monitoring method according to an embodiment of the present invention.
As shown in fig. 1, the task monitoring method according to the embodiment of the present invention mainly includes the following steps:
step S101: acquiring a plurality of task paths, wherein each task path comprises one or more single tasks, and the first single task and the last single task in each task path are the same;
step S102: according to the historical starting time and the historical processing time which are respectively corresponding to the single task and/or the current traffic corresponding to the single task, determining the predicted starting time and the predicted processing time corresponding to each single task;
step S103: determining the predicted ending time of the task path according to the predicted starting time and the predicted processing time;
step S104: and generating a task monitoring result corresponding to each task path and the single task according to whether the predicted starting time, the predicted processing time and the predicted ending time meet a preset threshold.
In the field of financial application, it is generally required to publish the calculation result of each task path outwards at a specific time in each day, so it is particularly important whether each single task in the task paths and the whole task paths can be completed on time, and the earlier and better the result is predicted, so that when the calculation result is abnormal and cannot be published smoothly, the corresponding processing strategy is determined in time.
For the historical startup time and the historical processing time in step S102, in an alternative embodiment, the execution result is obtained according to the execution result of the single task executed multiple times in the historical period. Typically, each single task and task path needs to be executed once a day, such as the day end settlement of a settlement system, so for each single task and task path, a plurality of historical starting times and historical processing times are correspondingly stored.
In an alternative embodiment, in step S102, the process of determining the predicted start time and the predicted process time corresponding to each single task according to the historical start time and the historical process time corresponding to the single task respectively, as shown in fig. 2, specifically includes:
step S201: determining the execution date of the single task in the history period and the weight corresponding to each execution date respectively;
step S202: and determining the predicted starting time and the predicted processing time according to the weight, the historical starting time and the historical processing time.
The historical data which is closer to the current date has a reference value for the predicted value, so that the historical data which is closer to the current date has larger weight in the embodiment of the invention. For example, taking a history period of 10 days as an example, the weight of each day may be (0.5)/(0-x), where x has a value of-10, -9, -8, … …, -2, -1 represents the previous day, -10 represents the previous 10 th day, -2 to-9, and the weight of the previous day is 0.5, the weight of the previous two days is 0.25, and so on, the further the date distance is, the smaller the weight is.
In a further alternative embodiment, the predicted start time and the predicted process time may be obtained by performing weighted summation according to the weights, the historical start times, and the historical process times corresponding to each execution date.
It can be understood that when predicting according to the historical data, accuracy and rationality of the historical data need to be ensured, screening processing can be preferentially performed on the historical data, for example, abnormal values or partial maximum values and minimum values are removed, and the screened historical data is used for predicting the starting time and the processing time, so that more accurate predicted values can be obtained.
In another alternative embodiment, the process of determining the predicted start time and the predicted processing time corresponding to each single task in step S102 according to the current traffic corresponding to the single task may, as shown in fig. 3, include:
step S301: acquiring current traffic corresponding to a single task;
step S302: determining a predicted processing time according to the pre-trained business processing model and the current business quantity; the business processing model is obtained through training of a linear regression algorithm, and indicates the corresponding relation between the historical business volume and the historical processing time.
In the day-end calculation process, the service volume of the day can be directly obtained from the input data of the settlement system, so that the current service volume corresponding to each single task can be directly obtained. The historical traffic is the corresponding traffic on the historical date, and the process can directly obtain the predicted processing time after the current traffic is input through the corresponding relationship between the historical traffic and the historical processing time in the business processing model. Specifically, for the business process model, the more the historical data, the higher the accuracy of the model, so that the historical traffic and the historical process time of the last 1 year or more can be selected, and modeling is performed by adopting a linear regression algorithm to obtain a pre-trained business process model.
In an alternative embodiment of the present invention, for each task path, when there are a plurality of single tasks therein, there is necessarily an execution order (parent-child relationship) between the plurality of single tasks, so the process of determining the predicted end time of the task path in step S103 may include, as shown in fig. 4:
step S401: determining the execution sequence of a plurality of single tasks in each task path;
step S402: determining target prediction ending time corresponding to each task path respectively according to the execution sequence, the prediction starting time corresponding to each single task and the prediction processing time by using a dynamic programming algorithm;
step S403: a predicted end time is determined from a plurality of target predicted end times.
Specifically, the core idea of a dynamic programming algorithm is to break up the original problem into a series of sub-problems and solve the original problem by solving the sub-problems. The specific calculation process can be as shown in fig. 5, wherein a vertex array is constructed according to each single task in a task path, and in the process of constructing a two-dimensional edge array according to the vertex array, V0, V1, V2, V3 and V4 in the figure are respectively 5 single tasks in one task path, arrows among task nodes represent the execution sequence of the tasks, numerals beside the arrows represent the prediction processing time, and the specific calculation process is as follows:
(1) Storing the operation time map [ i ] [ j ] of the father node (the node with the previous execution sequence) by using the two-dimensional edge array, wherein the prediction processing time when the father-son relationship among the nodes is not found is set to be +Inf (infinity in the edge array);
(2) The longest path length dp [ i ] between the task node and the last executed single task, namely the longest total time length of the tasks, is calculated and stored.
Where dp [ i ] = max { dp [ i ], map [ i ] [ j ] +dp [ j ] (i, j) ϵ E }, is a state transition equation in a dynamic programming algorithm describing how to solve larger scale problems with known sub-problem solutions. Specifically, to obtain the maximum value of dp [ i ], we can decompose dp [ i ] into map [ i ] [ j ] +dp [ j ], then take the maximum value of dp [ i ] and map [ i ] [ j ] +dp [ j ], wherein map [ i ] [ j ] is the operation duration of the parent node i, if i, j is not the parent-child node relationship, then map [ i ] [ j ] = +++ infinity, when the solution of map [ i ] [ j ] +dp [ j ] is known, the solution of dp [ i ] can be obtained, and when dp [ j ] is not known, the solution of dp [ j ] = max { dp [ j ], map [ j ] [ k ] +dp [ k ] (j, k) ϵ E ] is obtained by using the state transfer equation, and the result of dp [ i ] is obtained. Wherein, dp array dividing end point is initialized to-Inf, dp [ end ] =0 to ensure that dp [ i ] stores the maximum value to end point, not affected by other nodes.
(3) The successor node is saved with next [ i ], and the next array is initialized to-1.
(4) The visual array is used to save whether the current node has been accessed, and initialized to false.
Through the process, the target prediction ending time corresponding to each task path can be determined by using a dynamic programming algorithm. In an alternative embodiment, for step S403, the target predicted end time with the longest predicted end time is taken as the predicted end time.
The calculation process of the target prediction ending time corresponding to the task path may be executed at intervals according to a preset time period, and whether a timeout abnormality occurs in the task processing process may be found in time. Specifically, in the continuous execution process of the task, when the task is executed and completed, the predicted processing time and the predicted starting time are not used, but the actual known actual starting time and the actual processing time are used. Specifically, step S402 includes: and determining target prediction ending time of the task path according to the actual ending time of the first single task, the execution sequence of the non-executed one or more second single tasks and the prediction processing time corresponding to the second single task aiming at the situation that one or more first single tasks in the task path are executed.
In an alternative embodiment of the present invention, for the preset threshold in S104, different preset thresholds may be correspondingly set according to the difference between the predicted start time, the predicted processing time, and the predicted end time. Namely, step S104 includes: and determining a task monitoring result indicating abnormal task processing when at least one of the predicted starting time, the predicted processing time and the predicted ending time exceeds a corresponding preset threshold value.
Specifically, for the setting of the preset threshold, the determination may be made using different algorithms, such as 3δ, ratio, absolute value, machine learning algorithm DBSCAN, etc. The timeout alarm may also be triggered while obtaining a task monitoring result indicating that the task is handling an exception.
In an alternative embodiment, the preset threshold may be dynamically adjusted according to the actual execution end time of the single task that has completed execution, such as task path a: task 1, task 2 and task 3, wherein the predicted starting time of task 1 calculated in step S101 to step S103 is 14 points, and the preset threshold value of the early warning is set to be 14:10, the predicted starting time of the task 2 is 15 points, and the preset threshold value of the early warning is set to be 15 points 10. With the task being continuously executed, the actual start time of task 1 is 14:15, when the preset threshold is exceeded, the early warning is triggered, but the execution time of the task 1 is delayed, which inevitably leads to the delay of the execution start time of the subsequent task, so that the preset threshold of the task 2 can be synchronously adjusted to 15 according to the delay time of the task 1: 25 I.e. 15 minutes delay, to avoid the situation where subsequent tasks trigger alarms continuously due to a timeout of previous task execution. It will be appreciated that if the actual start-up time for task 1 is 14:05, the pre-set threshold value is not exceeded, the pre-warning is not triggered, and the pre-set threshold value of the subsequent task 2 is not affected.
After determining the abnormal single task, the single task can be optimized in a targeted manner, so that the processing speed of the single task is improved, and along with the progress of the optimization, the single task which may cause the overtime of the task path execution is changed. Also in task path a: task 1→task 2→task 3 and task path B: for example, before the optimization, the execution speed of the task 2 is slow, so that the execution time of the task paths A and B is abnormal over time, after the optimization, the execution time of the task 2 is faster than the execution time of the task 4 and the task 5, then the single task affecting the task paths A and B is one of the task 4 and the task 5, namely, the task path with the slowest target prediction end time can be optimized always preferentially, and the optimization efficiency is improved.
According to the task monitoring method, the task starting time and the task executing time of each single task are predicted, the predicted ending time of the task path is determined according to the predicted starting time and the predicted processing time, whether the abnormality occurs to each single task is monitored in time, and the corresponding processing strategy is obtained in time according to the task monitoring result. Meanwhile, the single task with slower execution time can be accurately positioned according to the task monitoring result of the single task, and the optimization can be carried out in a targeted manner, so that the subsequent optimization efficiency is improved.
Fig. 6 is a schematic diagram of the main modules of an apparatus for task monitoring according to an embodiment of the present invention.
As shown in fig. 6, a task monitoring apparatus 600 according to an embodiment of the present invention includes:
an obtaining module 601, configured to obtain a plurality of task paths, where each task path includes one or more single tasks, and a first single task and a last single task in each task path are the same;
a determining module 602, configured to determine a predicted start time and a predicted process time corresponding to each single task according to a historical start time and a historical process time corresponding to the single task, and/or a current traffic corresponding to the single task;
a prediction module 603, configured to determine a predicted end time of the task path according to the predicted start time and the predicted processing time;
and a result module 604, configured to generate a task monitoring result corresponding to each task path and the single task according to whether the predicted starting time, the predicted processing time, and the predicted ending time meet a preset threshold.
In an alternative embodiment of the present invention, the historical starting time and the historical processing time are obtained according to the execution result of the single task executed multiple times in a historical period.
In an alternative embodiment of the present invention, the determining module 602 is further configured to determine an execution date of the single task in the history period and a weight corresponding to each execution date; and determining the predicted starting time and the predicted processing time according to the weight, the historical starting time and the historical processing time.
In an optional embodiment of the present invention, the determining module 602 is further configured to obtain a current traffic corresponding to the single task; determining the prediction processing time according to a pre-trained service processing model and the current traffic; the business processing model is obtained through training of a linear regression algorithm, and the corresponding relation between the historical business volume and the historical processing time is indicated.
In an alternative embodiment of the present invention, the prediction module 603 is further configured to determine an execution order of a plurality of single tasks in each of the task paths; determining target prediction ending time corresponding to each task path respectively according to the execution sequence, the prediction starting time corresponding to each single task and the prediction processing time by using a dynamic programming algorithm; and determining the predicted ending time from a plurality of target predicted ending times.
In an optional embodiment of the present invention, the prediction module 603 is further configured to perform the step of determining, according to a preset period of time, a predicted end time of the task path according to the predicted start time and the predicted processing time; and determining target prediction ending time of the task path according to the actual ending time of the first single task, the execution sequence of the non-executed one or more second single tasks and the prediction processing time corresponding to the second single task aiming at the situation that one or more first single tasks in the task path are executed.
In an optional embodiment of the present invention, the predicted start time, the predicted processing time, and the predicted end time are respectively set with different preset thresholds correspondingly; the result module 604 is further configured to determine a task monitoring result indicating that the task processing is abnormal when at least one of the predicted start time, the predicted processing time, and the predicted end time exceeds a corresponding preset threshold.
According to the task monitoring device provided by the embodiment of the invention, the task starting time and the task executing time of each single task are predicted, the predicted ending time of the task path is determined according to the predicted starting time and the predicted processing time, whether the abnormality occurs to each single task is monitored in time, and the corresponding processing strategy is obtained in time according to the task monitoring result. Meanwhile, the single task with slower execution time can be accurately positioned according to the task monitoring result of the single task, and the optimization can be carried out in a targeted manner, so that the subsequent optimization efficiency is improved.
Fig. 7 illustrates an exemplary system architecture 700 of a method of task monitoring or an apparatus of task monitoring to which embodiments of the present invention may be applied.
As shown in fig. 7, a system architecture 700 may include terminal devices 701, 702, 703, a network 704, and a server 705. The network 704 is the medium used to provide communication links between the terminal devices 701, 702, 703 and the server 705. The network 704 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with the server 705 via the network 704 using the terminal devices 701, 702, 703 to send task monitoring instructions etc. to initiate the method of task monitoring according to the embodiments of the present invention. Various communication client applications, such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients, social platform software, etc., may be installed on the terminal devices 701, 702, 703.
The terminal devices 701, 702, 703 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 705 may be a server providing various services, such as a background management server providing support for task monitoring instructions sent by the user using the terminal devices 701, 702, 703. The background management server may analyze and process the received data, such as the task path, the historical starting time, the historical processing time, and the current traffic, and feed back the processing result (for example, the task monitoring result) to the terminal device.
It should be noted that, the method for task monitoring provided in the embodiment of the present invention is generally executed by the server 705, and accordingly, the device for task monitoring is generally disposed in the server 705.
It should be understood that the number of terminal devices, networks and servers in fig. 7 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 8, there is illustrated a schematic diagram of a computer system 800 suitable for use in implementing an embodiment of the present invention. The terminal device shown in fig. 8 is only an example, and should not impose any limitation on the functions and the scope of use of the embodiment of the present invention.
As shown in fig. 8, the computer system 800 includes a Central Processing Unit (CPU) 801 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 802 or a program loaded from a storage section 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the system 800 are also stored. The CPU 801, ROM 802, and RAM 803 are connected to each other by a bus 804. An input/output (I/O) first interface 805 is also connected to the bus 804.
The following components are connected to the I/O first interface 805: an input portion 806 including a keyboard, mouse, etc.; an output portion 807 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage section 808 including a hard disk or the like; and a communication section 809 including a network first interface card such as a LAN card, a modem, or the like. The communication section 809 performs communication processing via a network such as the internet. The driver 810 is also connected to the I/O first interface 805 as needed. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as needed so that a computer program read out therefrom is mounted into the storage section 808 as needed.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section 809, and/or installed from the removable media 811. The above-described functions defined in the system of the present invention are performed when the computer program is executed by a Central Processing Unit (CPU) 801.
The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules involved in the embodiments of the present invention may be implemented in software or in hardware. The described modules may also be provided in a processor, for example, as: a processor includes an acquisition module, a determination module, a prediction module, and a result module. Where the names of the modules do not constitute a limitation on the module itself in some cases, for example, the acquisition module may also be described as "acquiring multiple task paths, each of which includes one or more modules of a single task".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to include: acquiring a plurality of task paths, wherein each task path comprises one or more single tasks, and the first single task and the last single task in each task path are the same; according to the historical starting time and the historical processing time which are respectively corresponding to the single tasks and/or the current traffic corresponding to the single tasks, determining the predicted starting time and the predicted processing time corresponding to each single task; determining the predicted ending time of the task path according to the predicted starting time and the predicted processing time; and generating each task path and a task monitoring result corresponding to the single task according to whether the predicted starting time, the predicted processing time and the predicted ending time meet a preset threshold.
According to the technical scheme of the embodiment of the invention, the task starting time and the task executing time of each single task are predicted, the predicted ending time of the task path is determined according to the predicted starting time and the predicted processing time, whether the abnormality occurs to each single task is monitored in time, and the corresponding processing strategy is obtained in time according to the task monitoring result. Meanwhile, the single task with slower execution time can be accurately positioned according to the task monitoring result of the single task, and the optimization can be carried out in a targeted manner, so that the subsequent optimization efficiency is improved.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (8)
1. A method of task monitoring, comprising:
acquiring a plurality of task paths, wherein each task path comprises one or more single tasks, and the first single task and the last single task in each task path are the same;
According to the historical starting time and the historical processing time which are respectively corresponding to the single tasks and/or the current traffic corresponding to the single tasks, determining the predicted starting time and the predicted processing time corresponding to each single task; the historical starting time and the historical processing time are obtained according to the execution results of the single task executed for a plurality of times in a historical period;
determining the predicted ending time of the task path according to the predicted starting time and the predicted processing time;
generating each task path and a task monitoring result corresponding to the single task according to whether the predicted starting time, the predicted processing time and the predicted ending time meet a preset threshold;
the determining the predicted starting time and the predicted processing time corresponding to each single task according to the historical starting time and the historical processing time corresponding to the single task and/or the current traffic corresponding to the single task respectively comprises the following steps: determining the execution date of the single task in a history period and the weight corresponding to each execution date respectively; and determining the predicted starting time and the predicted processing time according to the weight, the historical starting time and the historical processing time.
2. The method according to claim 1, wherein determining the predicted start time and the predicted process time corresponding to each single task according to the historical start time and the historical process time corresponding to the single task and/or the current traffic corresponding to the single task respectively comprises:
acquiring the current traffic corresponding to the single task;
determining the prediction processing time according to a pre-trained service processing model and the current traffic; the business processing model is obtained through training of a linear regression algorithm, and the corresponding relation between the historical business volume and the historical processing time is indicated.
3. The method of claim 1, wherein determining the predicted end time of the task path based on the predicted start time and the predicted processing time comprises:
determining the execution sequence of a plurality of single tasks in each task path;
determining target prediction ending time corresponding to each task path respectively according to the execution sequence, the prediction starting time corresponding to each single task and the prediction processing time by using a dynamic programming algorithm;
And determining the predicted ending time from a plurality of target predicted ending times.
4. A method according to claim 3, wherein said step of determining a predicted end time of said task path based on said predicted start time and said predicted processing time is performed in accordance with a predetermined time period;
the determining, by using a dynamic programming algorithm, a target prediction end time corresponding to each task path according to the execution sequence, the prediction start time corresponding to each single task, and the prediction processing time, includes:
and determining target prediction ending time of the task path according to the actual ending time of the first single task, the execution sequence of the non-executed one or more second single tasks and the prediction processing time corresponding to the second single task aiming at the situation that one or more first single tasks in the task path are executed.
5. The method according to claim 1, wherein the predicted start time, the predicted process time, and the predicted end time are respectively provided with different preset thresholds; generating a task monitoring result corresponding to each task path and the single task according to whether the predicted starting time, the predicted processing time and the predicted ending time meet a preset threshold value, including:
And determining a task monitoring result indicating abnormal task processing when at least one of the predicted starting time, the predicted processing time and the predicted ending time exceeds a corresponding preset threshold value.
6. An apparatus for task monitoring, comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a plurality of task paths, each task path comprises one or more single tasks, and the first single task and the last single task in each task path are the same;
the determining module is used for determining the predicted starting time and the predicted processing time corresponding to each single task according to the historical starting time and the historical processing time corresponding to the single task respectively and/or the current traffic corresponding to the single task; the historical starting time and the historical processing time are obtained according to the execution results of the single task executed for a plurality of times in a historical period;
the prediction module is used for determining the predicted ending time of the task path according to the predicted starting time and the predicted processing time;
the result module is used for generating a task monitoring result corresponding to each task path and the single task according to whether the predicted starting time, the predicted processing time and the predicted ending time meet a preset threshold value or not;
The determining the predicted starting time and the predicted processing time corresponding to each single task according to the historical starting time and the historical processing time corresponding to the single task and/or the current traffic corresponding to the single task respectively comprises the following steps: determining the execution date of the single task in a history period and the weight corresponding to each execution date respectively; and determining the predicted starting time and the predicted processing time according to the weight, the historical starting time and the historical processing time.
7. An electronic device for task monitoring, comprising:
one or more processors;
storage means for storing one or more programs,
when executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-5.
8. A computer readable medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311404935.1A CN117130873B (en) | 2023-10-26 | 2023-10-26 | Task monitoring method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311404935.1A CN117130873B (en) | 2023-10-26 | 2023-10-26 | Task monitoring method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117130873A CN117130873A (en) | 2023-11-28 |
| CN117130873B true CN117130873B (en) | 2024-01-26 |
Family
ID=88861397
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311404935.1A Active CN117130873B (en) | 2023-10-26 | 2023-10-26 | Task monitoring method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117130873B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111010292A (en) * | 2019-11-26 | 2020-04-14 | 苏宁云计算有限公司 | Offline task delay warning system and method and computer system |
| CN111737095A (en) * | 2020-08-05 | 2020-10-02 | 北京必示科技有限公司 | Batch processing task time monitoring method and device, electronic equipment and storage medium |
| CN113448808A (en) * | 2021-08-30 | 2021-09-28 | 北京必示科技有限公司 | Method, system and storage medium for predicting single task time in batch processing task |
| CN114037293A (en) * | 2021-11-11 | 2022-02-11 | 中国建设银行股份有限公司 | Task allocation method, device, computer system and medium |
| CN115934460A (en) * | 2022-11-30 | 2023-04-07 | 深圳市跨越新科技有限公司 | Method, device and equipment for monitoring key tasks and storage medium |
| CN116048762A (en) * | 2023-02-10 | 2023-05-02 | 长春大学旅游学院 | Computer resource scheduling system based on historical data analysis |
| CN116680063A (en) * | 2023-08-03 | 2023-09-01 | 苏州浪潮智能科技有限公司 | Task scheduling method, device, computing system, electronic equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11704151B2 (en) * | 2020-09-28 | 2023-07-18 | International Business Machines Corporation | Estimate and control execution time of a utility command |
-
2023
- 2023-10-26 CN CN202311404935.1A patent/CN117130873B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111010292A (en) * | 2019-11-26 | 2020-04-14 | 苏宁云计算有限公司 | Offline task delay warning system and method and computer system |
| CN111737095A (en) * | 2020-08-05 | 2020-10-02 | 北京必示科技有限公司 | Batch processing task time monitoring method and device, electronic equipment and storage medium |
| CN113448808A (en) * | 2021-08-30 | 2021-09-28 | 北京必示科技有限公司 | Method, system and storage medium for predicting single task time in batch processing task |
| CN114037293A (en) * | 2021-11-11 | 2022-02-11 | 中国建设银行股份有限公司 | Task allocation method, device, computer system and medium |
| CN115934460A (en) * | 2022-11-30 | 2023-04-07 | 深圳市跨越新科技有限公司 | Method, device and equipment for monitoring key tasks and storage medium |
| CN116048762A (en) * | 2023-02-10 | 2023-05-02 | 长春大学旅游学院 | Computer resource scheduling system based on historical data analysis |
| CN116680063A (en) * | 2023-08-03 | 2023-09-01 | 苏州浪潮智能科技有限公司 | Task scheduling method, device, computing system, electronic equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 一种快速的网格任务调度算法的设计;许兰;朱巧明;陈静;贡正仙;;计算机应用与软件(第05期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117130873A (en) | 2023-11-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109471783B (en) | Method and device for predicting task operation parameters | |
| CN109902446B (en) | Method and apparatus for generating information prediction model | |
| CN108810047B (en) | Method and device for determining information push accuracy rate and server | |
| CN114048931B (en) | Replenishment information generation method and device, electronic equipment and computer readable medium | |
| CN114818913A (en) | Decision generation method and device | |
| CN113760982A (en) | Data processing method and device | |
| CN116388112B (en) | Abnormal supply end power-off method, device, electronic equipment and computer readable medium | |
| CN111385150A (en) | Method and device for acquiring information | |
| CN117130873B (en) | Task monitoring method and device | |
| CN113312156A (en) | Method, apparatus and computer program product for determining execution progress of a task | |
| CN110727558A (en) | Information prompting method and device, storage medium and electronic equipment | |
| CN114500318B (en) | Batch operation monitoring method, device, equipment and medium | |
| CN116541069A (en) | Key function evaluation method, device, electronic equipment, medium and program product | |
| CN114861909A (en) | Model quality monitoring method and device, electronic equipment and storage medium | |
| CN115187364A (en) | Method and device for monitoring deposit risk under bank distributed scene | |
| CN111311014B (en) | Service data processing method, device, computer equipment and storage medium | |
| CN113537893A (en) | Order processing method, device, equipment and computer readable medium | |
| CN113391988A (en) | Method and device for losing user retention, electronic equipment and storage medium | |
| CN113052509A (en) | Model evaluation method, model evaluation apparatus, electronic device, and storage medium | |
| CN113965563A (en) | Model-based business processing method and device and server | |
| CN112348587B (en) | Information pushing method and device and electronic equipment | |
| CN115312208B (en) | Method, device, equipment and medium for displaying treatment data | |
| CN110262756B (en) | Method and device for caching data | |
| CN116663928A (en) | Operation control method, device and equipment for platform batch operation and storage medium | |
| CN114237856A (en) | Operation type identification method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |