CN115292141B - Scheduling abnormity early warning method based on sliding time window and monitoring server - Google Patents
Scheduling abnormity early warning method based on sliding time window and monitoring server Download PDFInfo
- Publication number
- CN115292141B CN115292141B CN202211197815.4A CN202211197815A CN115292141B CN 115292141 B CN115292141 B CN 115292141B CN 202211197815 A CN202211197815 A CN 202211197815A CN 115292141 B CN115292141 B CN 115292141B
- Authority
- CN
- China
- Prior art keywords
- task
- scheduling
- time window
- time
- sliding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/302—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3051—Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
Abstract
The invention discloses a scheduling abnormity early warning method based on a sliding time window and a monitoring server, wherein a task scheduling plan is generated, task scheduling time is generated according to a CRON expression corresponding to each task of a scheduling client, the task state is marked as unexecuted, and a task scheduling time point is recorded in the sliding time window; the task scheduling plan is issued to the client, and the task scheduling information uploaded when the client starts to perform at least one task scheduling is received; when the task scheduling information is received, marking the corresponding task state in the task scheduling plan as executed; and when the sliding time window is removed, if the task in the unexecuted state exists, executing abnormal early warning. Compared with the prior art, the scheduling plan is generated through the monitoring service, the scheduling information is uploaded by each scheduling service, and the monitoring service timely performs early warning when the scheduling service is abnormal, so that the problem can be quickly positioned and checked conveniently.
Description
Technical Field
The invention relates to the technical field of task scheduling, in particular to a scheduling abnormity early warning method and a monitoring server based on a sliding time window.
Background
In the usual business scenario, there are often some scenarios that require the use of timing tasks, such as: time-driven scenario: sending coupons at a certain point in time, sending short messages, etc. Processing data in batches: batch statistics of monthly bills, statistics of monthly sales data, and the like. Fixed frequency scenario: every 5 minutes. Therefore, the timing task is not uncommon in ordinary development, and for the current era of rapid consumption, various pushes need to be sent every day, messages need to be completed by relying on the timing task, and the application is very wide. However, scheduling tasks may not be performed normally for various reasons, such as exhaustion of the thread pool with no available threads, IO blocking leading to task false death, etc. Resulting in the scheduled tasks not being executed as planned, causing losses.
In the prior art, the abnormality is not found timely, and the user cannot be informed timely; the log statistical method has large resource consumption and high cost; the special scheduling plan is difficult to count the scheduling times, such as once a month or once a week, and cannot identify whether the technical defect of scheduling is performed in a statistical manner.
Disclosure of Invention
In view of this, the embodiments of the present invention aim to generate a scheduling plan through a monitoring service, each scheduling service uploads scheduling information, and when the scheduling service is not executed according to the scheduling plan due to an abnormality or does not upload the scheduling information in time, the monitoring service performs early warning in time, so that an operation and maintenance worker can quickly locate and troubleshoot problems according to corresponding early warning information, and avoid unnecessary loss.
The invention provides a scheduling abnormity early warning method based on a sliding time window, which is applied to a monitoring server side and comprises the following steps:
generating a task scheduling plan, generating task scheduling time according to a CRON expression corresponding to each task of a scheduling client, marking the task state as unexecuted, and recording the task scheduling time point in a sliding time window;
the task scheduling plan is issued to the client, and the task scheduling information uploaded when the client starts to perform at least one task scheduling is received;
when the task scheduling information is received, marking the corresponding task state in the task scheduling plan as executed;
and when the sliding time window is removed, if the task in the unexecuted state exists, executing abnormal early warning.
Further, the method further comprises: monitoring a scheduling abnormity early warning request of a client, and simultaneously establishing a sliding time window;
the creating of the sliding time window comprises: determining a size of a sliding time window; each sliding time window comprises a plurality of scheduling tasks, and each task is provided with a corresponding task scheduling plan according to a preset time interval in the sliding time window; the task scheduling plan comprises scheduling times or scheduling time of each task according to a preset time interval in a sliding time window.
Further, the sliding time window is described by adopting a double hash table set; wherein, the key corresponding to the first-level hash table of the double hash tables is a time stamp; the value is a task data hash table;
the task data hash table is used for storing corresponding task scheduling plan information in a sliding time window; the key of the task data hash table is a scheduling task code, and the value is the set of the task scheduling plan in the sliding time window; each piece of data corresponds to a task scheduling plan; the set is Treeset structure.
Further, the generating a task scheduling plan, generating a task scheduling time according to the CRON expression corresponding to each task of the scheduling client, marking the task state as unexecuted, and recording the task scheduling time point in a sliding time window includes:
step 1, generating a CRON expression corresponding to a scheduling task, and calculating a next scheduling time point by taking the current time as a base point;
step 3, judging whether the task corresponding to the time point exists or not, if not, inserting a corresponding task scheduling plan, and marking the task state as unexecuted; judging whether the task corresponding to the time point exists or not, wherein the judging comprises traversing plan scheduling set data according to the service name, the task name and the scheduling time point, and searching whether data equal to the current service name, the task name and the scheduling time point exist in the set or not;
and 4, circulating the steps 1-3, and continuously generating the next scheduling time point by using the last calculated scheduled time point as a base point until the calculated next scheduling time point exceeds the boundary of the pre-execution time window.
Further, when the task scheduling information is received, marking a corresponding task state in the task scheduling plan as executed includes:
after receiving task scheduling information of a scheduling client, searching out the earliest scheduling plan in an executed sliding time window interval according to the ascending order of time of the sliding time window through scheduling task codes, and modifying the state of the earliest scheduling plan into an executed state;
and the searching according to the ascending time sequence of the sliding window is performed according to the time boundary which is the difference between the current time and the fault-tolerant time.
Further, the sliding time window comprises a pre-execution interval and an executed interval, and the current time is acquired according to a preset time period to update the sliding time window;
the updating of the sliding time window comprises:
calculating the boundary of a current time window according to the current time, if the calculated pre-execution boundary is larger than the maximum time of the pre-execution interval of the current window, inserting a new timestamp into the pre-execution interval of the window, and updating a first-stage hash table and a task data hash table corresponding to the new timestamp;
and calculating an executed boundary, and removing window data outside the boundary from the time window if the executed interval of the current time window contains a sample window with corresponding time smaller than the executed boundary.
Further, when the sliding time window is removed, if there is a task in an unexecuted state, executing an abnormal early warning, including:
and traversing the scheduling plan task which is correspondingly removed when the sliding time window is removed, if the task which is not executed in the state is searched out, marking the scheduling task as the non-executed abnormal scheduling, recording the abnormal scheduling, and executing early warning.
The second aspect of the invention provides a monitoring server, which comprises a generating module, an issuing and receiving module, a marking module and an early warning module; wherein:
the generating module is used for generating a task scheduling plan, generating task scheduling time according to a CRON expression corresponding to each task of the scheduling client, marking the task state as unexecuted, and recording the task scheduling time point in a sliding time window;
the issuing and receiving module is used for issuing the task scheduling plan to the client and receiving the task scheduling information uploaded when the client starts to schedule at least one task;
the marking module is used for marking the corresponding task state in the task scheduling plan as executed when the task scheduling information is received;
and the early warning module is used for executing abnormal early warning if tasks in an unexecuted state exist when the sliding time window is removed.
Further, the server also comprises a monitoring module and a creating module; the monitoring module is used for monitoring scheduling abnormity early warning requests of the client;
the creation module is used for creating a sliding time window; the creating of the sliding time window comprises: determining a size of a sliding time window; each sliding time window comprises a plurality of scheduling tasks, and each task is provided with a corresponding task scheduling plan according to a preset time interval in the sliding time window; the task scheduling plan comprises scheduling times or scheduling time of each task according to a preset time interval in a sliding time window.
Further, the creating module is further configured to describe the sliding time window by using a double hash table set; wherein, the key corresponding to the first-level hash table of the double hash tables is a time stamp; the value is a task data hash table;
the task data hash table is used for storing corresponding task scheduling plan information in a sliding time window; the key of the task data hash table is a scheduling task code, and the value is the set of the task scheduling plan in the sliding time window; each piece of data corresponds to a task scheduling plan; the set is a Treeset structure.
In the scheme of the invention, a task scheduling plan is generated, task scheduling time is generated according to a CRON expression corresponding to each task of a scheduling client, the task state is marked as unexecuted, and the task scheduling time point is recorded in a sliding time window; the task scheduling plan is issued to the client, and the task scheduling information uploaded when the client starts to perform at least one task scheduling is received; when the task scheduling information is received, marking the corresponding task state in the task scheduling plan as executed; and when the sliding time window is removed, if the task in the unexecuted state exists, executing abnormal early warning. Compared with the prior art, the scheduling plan is generated through the monitoring service, each scheduling service uploads scheduling information, when the scheduling service is not executed according to the scheduling plan due to abnormity or the scheduling information is not uploaded in time, the monitoring service gives early warning in time, and operation and maintenance personnel can quickly position and troubleshoot problems according to corresponding early warning information, so that unnecessary loss is avoided.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a schematic flow chart of a scheduling anomaly early warning method based on a sliding time window disclosed in embodiment 1 of the present invention;
FIG. 2 is a diagram of a relationship between a scheduling task and a sliding time window disclosed in embodiment 1 of the present invention;
FIG. 3 is a flowchart of a thread execution timing scheduling disclosed in embodiment 1 of the present invention;
fig. 4 is a flowchart of the sliding time window performing pre-execution interval update disclosed in embodiment 1 of the present invention;
FIG. 5 is a flowchart illustrating a sliding time window for performing an executed interval update according to embodiment 1 of the present invention;
fig. 6 is a schematic structural diagram of a monitoring server disclosed in embodiment 2 of the present invention.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the subject matter of the present application can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations, or operations have not been shown or described in detail to avoid obscuring aspects of the application.
The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.
The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
It should be noted that: reference herein to "a plurality" means two or more.
The implementation details of the technical solution of the embodiment of the present application are set forth in detail below:
example 1
In the embodiment, the whole framework core of the scheduling early warning based on the sliding time window is divided into two parts, namely a timed task scheduling client service (client) and a monitoring service (monitoring server). The method comprises the steps that a timed task scheduling client serves, task calling is carried out according to a scheduling plan, and task scheduling information is uploaded to a monitoring end when calling starts; the monitoring server is responsible for generating task planning time according to the CRON expression corresponding to each task of the scheduling client, marking the task planning time as unexecuted, recording the task planning time point in a sliding time window, and marking the corresponding planning task state as executed when receiving scheduling information of the scheduling task; when the time window is removed in a sliding mode and an unexecuted planning task is found, the planning task is marked as an abnormal scheduling unexecuted task, and then notification is carried out.
Referring to fig. 1, fig. 1 is a schematic flowchart illustrating a scheduling exception early warning method based on a sliding time window according to an embodiment of the present invention. As shown in fig. 1, a scheduling exception early warning method based on a sliding time window according to an embodiment of the present invention is applied to a monitoring server.
The method comprises the following steps: s1, generating a task scheduling plan, generating task scheduling time according to a CRON expression corresponding to each task of a scheduling client, marking the task state as unexecuted, and recording task scheduling time points in a sliding time window.
In this embodiment, before generating the task scheduling plan, a step of monitoring a scheduling exception early warning request of the client and creating a sliding time window is further provided. The creating of the sliding time window comprises: determining a size of a sliding time window; each sliding time window comprises a plurality of scheduling tasks, and each task is provided with a corresponding task scheduling plan according to a preset time interval in the sliding time window; the task scheduling plan comprises scheduling times or scheduling time of each task according to a preset time interval in a sliding time window.
Specifically, in this embodiment, the client and the server may be started first. When a client is started, loading corresponding scheduling configuration, using cron expression statement description for a scheduling period corresponding to a scheduling task plan, and uploading a configuration to a scheduling monitoring service; and scheduling after the corresponding configuration is loaded by the timing task quartz component, rewriting the task execution method, and sending current scheduling information to a monitoring end before the actual scheduling of the task. When the server side is started, the interface is started to monitor the client side request, and meanwhile a sliding time window is established. The cron expression is a character string with time meaning, the character string is separated by 5~6 blank spaces and divided into 6~7 fields in the format of X X X X X X X X X X X. Where X is a placeholder for a domain. The last year-representative domain is not essential and may be omitted. When a single field has multiple values, half-angle commas are used to separate the values. Each field may be a definite value or a special character with logical meaning. Each domain supports at most one leading zero. The timed task Quartz component is an open source project in the field of Job scheduling, and Quartz can be used alone or integrated with a spring framework, and the Quartz component is generally used in actual development. One or more timed tasks may be developed using Quartz, each of which may be individually designated as a time of execution, e.g., every 1 hour, 10 am the first day of the month, 5 pm the last day of the month, etc.
Further, the sliding time window is described by adopting a double hash table set; wherein, the key corresponding to the first-level hash table of the double hash tables is a time stamp; the value is a task data hash table. The task data hash table is used for storing corresponding task scheduling plan information in a sliding time window; the key of the task data hash table is a scheduling task code, and the value is a set of the task scheduling plan in the sliding time window; each piece of data corresponds to a task scheduling plan; the set is a Treeset structure.
Specifically, in the present embodiment, for the sliding time window construction, the whole data structure thereof adopts a double hash table set. Key (Key) corresponding to the first-level hash table is a time stamp, time precision is minutes, and Value is a task data hash table. The task data hash table is used for storing task scheduling plan information corresponding to each minute, keys are scheduling task codes, values are a set of the task scheduling plan in the minute, each piece of data is a scheduling plan, time precision is seconds, namely a cron expression can describe the minimum unit second, the set is a Treeset structure, firstly, the whole scheduling plan is sorted according to a time positive sequence, and secondly, when the scheduling plan generated by operation is prevented, the same scheduling plan is generated for many times to cause repetition.
Based on this, the overall data structure of the overall sliding time window can be obtained: taking one minute as the size of the sliding time window as an example, each sliding window has zero to multiple scheduling tasks, each task has 0 to 60 execution plans within the current one minute, as shown in fig. 2, which is a structure diagram of the relationship between the scheduling tasks and the sliding time window of the embodiment, the task has 60 time windows in total, and the task includes 30 pre-execution ranges and 30 executed ranges. The middle separates pre-execution from already executed with the current time as a boundary. The task has 12 schedules per window, i.e. every 5 seconds, where the darker colored areas on the right represent to be scheduled (not scheduled) and the lighter colored areas on the left are scheduled.
Further, S1, generating a task scheduling plan, generating task scheduling time according to a CRON expression corresponding to each task of a scheduling client, marking the task state as unexecuted, and recording the task scheduling time point in a sliding time window, wherein the steps of: s11, generating a CRON expression corresponding to the scheduling task, and calculating the next scheduling time point by taking the current time as a base point; s12, searching whether a corresponding time point is included in a current sliding time window, if so, judging whether a planned dispatching set of the dispatching task exists at the time point, and if not, newly building the planned dispatching set; s13, judging whether the task corresponding to the time point exists or not, if not, inserting a corresponding task scheduling plan, and marking the task state as unexecuted; judging whether the task corresponding to the time point exists or not, wherein the judging comprises the steps of traversing plan scheduling set data according to the service name, the task name and the scheduling time point and searching whether data equal to the current service name, the task name and the scheduling time point exist in the set or not; and S14, looping the steps S11 to S13, and continuing to generate the next scheduling time point by using the last calculated scheduled time point as a base point until the calculated next scheduling time point exceeds the boundary of the pre-execution time window.
And S2, issuing the task scheduling plan to the client, and receiving the task scheduling information uploaded when the client starts to perform at least one task scheduling.
Specifically, in this embodiment, the scheduling plan is generated by the monitoring server and sent to the timed task scheduling client for service. And each client schedules the task scheduling information uploaded by the service to the monitoring server.
Further, fig. 3 is a flowchart illustrating the execution of the timed dispatch thread according to this embodiment. Executing the scheduling thread once every ten seconds through a timing scheduling thread, and generating a scheduling time point for the next time by taking the current time as a base point according to a cron expression corresponding to the task; then, searching whether the corresponding time point is included in the time window, and skipping if the corresponding time point is not included; judging whether a plan scheduling set of the scheduling task exists at the time point or not, and if not, establishing a new scheduling set; and judging whether the task at the corresponding time point exists or not, judging whether the condition is service name, task name and scheduling time point, traversing the plan scheduling set data according to the service name, the task name and the scheduling time point, and searching whether the data equal to the current service name, the task name and the scheduling time point exist in the set or not. And if the scheduling plan exists, skipping the insertion, and if the scheduling plan does not exist, inserting the corresponding scheduling plan, and setting the scheduling state as non-scheduling. And circulating the process, and continuously generating the next scheduling time point by using the last calculated scheduled time point as a base point until the calculated next scheduling time point exceeds the boundary of the pre-execution time window.
The next scheduling reservation calculation method comprises the following steps: if the scheduling cron expression is 0/5 x, and the scheduling time base point is 2022-09-05 10, then the frequency obtained by the cron expression is executed every 5 minutes from 0 seconds 2, and the next scheduling time is 2022-09-05 10 based on the scheduling base point. And so on, then the next scheduled time is 2022-09-05 10 with 2022-09-05 10 as a time base.
And S3, when the task scheduling information is received, marking the corresponding task state in the task scheduling plan as executed.
Further, when the task scheduling information is received, marking a corresponding task state in the task scheduling plan as executed includes: after receiving task scheduling information of a scheduling client, searching out the earliest scheduling plan in an executed sliding time window interval according to the ascending order of time of the sliding time window through scheduling task codes, and modifying the state of the earliest scheduling plan into an executed state; the searching according to the ascending time sequence of the sliding window is performed according to the time boundary which is the difference between the current time and the fault-tolerant time.
Specifically, in this embodiment, after receiving the task scheduling information of the scheduling client, the earliest scheduling plan is retrieved in the executed sliding time window interval according to the ascending order of the time of the sliding window by scheduling task codes, and the state of the scheduling plan is modified to be executed. The retrieval range is the past time of the linear interval, no task with a scheduled task executing time point yet is theoretically available, and the time difference between servers is not eliminated, so that the current time plus the fault-tolerant time difference is suggested to be used as a time boundary for retrieval, the time of a scheduled client is prevented from being early compared with the time of a monitoring end, when the task executes the scheduling information, the corresponding scheduling plan cannot be found, and the early warning error report is finally caused.
And S4, when the sliding time window is removed, if the task in the unexecuted state exists, executing abnormal early warning.
Further, the sliding time window includes a pre-execution interval and an executed interval, and the current time is acquired according to a predetermined time period to update the sliding time window. FIG. 4 is a flowchart illustrating the sliding time window performing the pre-execution interval update according to the present embodiment. Wherein, the updating of the sliding time window comprises: and calculating the boundary of the current time window according to the current time, if the calculated pre-execution boundary is larger than the maximum time of the pre-execution interval of the current window, inserting a new timestamp into the pre-execution interval of the window, and updating a first-stage hash table and a task data hash table corresponding to the new timestamp. Referring to FIG. 5, which is a flowchart illustrating the sliding of the time window to update the executed window in the present embodiment, by calculating the executed boundary, if the executed window of the current time window includes a sample window with a corresponding time shorter than the executed boundary, the window data outside the boundary is removed from the time window.
Specifically, in the present embodiment, the current time is obtained and the current sliding time window boundary is determined. For example, the current time is 13. Then, for the current time of 13. Similarly, an executed boundary is calculated, and the executed interval is (12. Within this time window, tasks are performed as planned. With the movement of time, the time window of the present embodiment also needs to be moved, and the present embodiment refreshes the current time every 10 seconds by setting a certain refresh period, for example, 10 seconds, and if there is no change, continues scheduling and executing the task corresponding to the time in the sliding time window corresponding to the current time.
Further, once the current time changes, for example, from 13. Specifically, when the current time becomes 13, in a manner that "the current time plus thirty minutes is a pre-execution interval and the current time minus 30 minutes is an executed interval" as described above, the calculated pre-execution boundary is 13. Likewise, the current time changes, as well as the executed boundaries of the sliding time window, and from the current new time 13.
Further, when the sliding time window is removed, if there is a task in an unexecuted state, executing an abnormal early warning, including: and traversing the scheduling plan task which is correspondingly removed when the sliding time window is removed, if the task which is not executed in the state is searched out, marking the scheduling task as the non-executed abnormal scheduling, recording the abnormal scheduling, and executing early warning.
In this embodiment, the whole server includes three threads for the sliding time window, one of the three threads is used for generating a pre-execution scheduling plan, calculating the next scheduling time of each scheduling task, and inserting the scheduling plan. One thread is used to remove dispatch plans that exceed boundaries and to notify the unexecuted dispatch plan calls to a notification service. And the other thread is used for receiving scheduling information of the scheduling client, retrieving a corresponding scheduling plan and modifying a scheduling state. When the sliding window is modified by the current three threads simultaneously, the re-entrant read-write lock is used, so that the problem of data asynchronism under a multi-thread scene is prevented. The three threads use the write lock, and the monitoring end uses the read lock to display interface display data. Generating a task scheduling plan, generating task scheduling time according to a CRON expression corresponding to each task of a scheduling client, marking the task state as unexecuted, and recording the task scheduling time point in a sliding time window; the task scheduling plan is issued to the client, and the task scheduling information uploaded when the client starts to perform at least one task scheduling is received; when the task scheduling information is received, marking the corresponding task state in the task scheduling plan as executed; and when the sliding time window is removed, if the task in the unexecuted state exists, executing abnormal early warning.
Compared with the prior art, the scheduling plan is generated through the monitoring service, each scheduling service uploads scheduling information, when the scheduling service is not executed according to the scheduling plan due to abnormity or the scheduling information is not uploaded in time, the monitoring service gives early warning in time, and operation and maintenance personnel can quickly position and troubleshoot problems according to corresponding early warning information, so that unnecessary loss is avoided. The client server mode does not influence the scheduling monitoring operation due to the shutdown of the client; the early warning real-time performance is strong, and the early warning time difference can be set according to the size of the time window; the sliding time window data can be displayed, and the whole scheduling state can be observed conveniently.
Example 2
As shown in fig. 6, the monitoring server includes a generating module 10, an issuing and receiving module 20, a marking module 30, and an early warning module 40; wherein:
the generating module 10 is configured to generate a task scheduling plan, generate a task scheduling time according to a CRON expression corresponding to each task of the scheduling client, mark a task state as unexecuted, and record a task scheduling time point in a sliding time window;
the issuing and receiving module 20 is configured to issue the task scheduling plan to the client, and receive task scheduling information uploaded when the client starts to perform at least one task scheduling;
a marking module 30, configured to mark, when receiving the task scheduling information, a corresponding task state in the task scheduling plan as executed;
and the early warning module 40 is used for executing abnormal early warning if the task in the unexecuted state exists when the sliding time window is removed.
Further, the system also includes a listening module 50 and a creating module 60; the monitoring module 50 is configured to monitor a scheduling exception early warning request of a client;
the creating module 60 is configured to create a sliding time window; the creating of the sliding time window comprises: determining a size of a sliding time window; each sliding time window comprises a plurality of scheduling tasks, and each task is provided with a corresponding task scheduling plan according to a preset time interval in the sliding time window; the task scheduling plan comprises scheduling times or scheduling time of each task according to a preset time interval in a sliding time window.
Further, the creating module 60 is further configured to describe the sliding time window by using a double hash table set; wherein, the key corresponding to the first-level hash table of the double hash tables is a time stamp; the value is a task data hash table;
the task data hash table is used for storing corresponding task scheduling plan information in a sliding time window; the key of the task data hash table is a scheduling task code, and the value is the set of the task scheduling plan in the sliding time window; each piece of data corresponds to a task scheduling plan; the set is a Treeset structure.
Further, the generating module 10 is further configured to: generating a CRON expression corresponding to the scheduling task, and calculating the next scheduling time point by taking the current time as a base point; searching whether a corresponding time point is included in a current sliding time window, if so, judging whether a planned dispatching set of the dispatching task exists at the time point, and if not, establishing the planned dispatching set; judging whether the task corresponding to the time point exists or not, if not, inserting a corresponding task scheduling plan, and marking the task state as unexecuted; judging whether the task corresponding to the time point exists or not, wherein the judging comprises the steps of traversing plan scheduling set data according to the service name, the task name and the scheduling time point and searching whether data equal to the current service name, the task name and the scheduling time point exist in the set or not; and continuously generating the next scheduling time point by using the last calculated planned time point as a base point until the calculated next scheduling time point exceeds the boundary of the pre-execution time window.
Further, when the task scheduling information is received, marking a corresponding task state in the task scheduling plan as executed includes:
after receiving task scheduling information of a scheduling client, searching out the earliest scheduling plan in an executed sliding time window interval according to the ascending order of time of the sliding time window through scheduling task codes, and modifying the state of the earliest scheduling plan into an executed state; the searching according to the ascending time sequence of the sliding window is performed according to the time boundary which is the difference between the current time and the fault-tolerant time.
Further, the sliding time window comprises a pre-execution interval and an executed interval, and the current time is acquired according to a preset time period to update the sliding time window;
the updating of the sliding time window comprises: calculating the boundary of a current time window according to the current time, if the calculated pre-execution boundary is larger than the maximum time of the pre-execution interval of the current window, inserting a new timestamp into the pre-execution interval of the window, and updating a first-stage hash table and a task data hash table corresponding to the new timestamp; and calculating an executed boundary, and removing window data outside the boundary from the time window if the executed interval of the current time window contains a sample window with corresponding time smaller than the executed boundary.
Further, the early warning module 40 is further configured to traverse the scheduling plan task that is correspondingly removed when the sliding time window is removed, mark the scheduling task as an unexecuted abnormal scheduling if the task whose state is not executed is retrieved, record the abnormal scheduling, and perform early warning.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one type of logical functional division, and other divisions may be realized in practice, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may also be an electric, mechanical or other form of connection.
The elements described as separate components may or may not be physically separate, as one of ordinary skill in the art would recognize that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the components and steps of the various examples have been described in a functional generic manner in the above description for the purpose of clearly illustrating the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention essentially or partially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a grid device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are merely exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (9)
1. A scheduling abnormity early warning method based on a sliding time window is applied to a monitoring server side, and is characterized in that the method comprises the following steps:
generating a task scheduling plan, generating task scheduling time according to a CRON expression corresponding to each task of a scheduling client, marking the task state as unexecuted, and recording the task scheduling time point in a sliding time window;
the task scheduling plan is issued to the client, and the task scheduling information uploaded when the client starts to perform at least one task scheduling is received;
when the task scheduling information is received, marking the corresponding task state in the task scheduling plan as executed;
when the sliding time window is removed, if the task in the unexecuted state exists, executing abnormal early warning;
the generating of the task scheduling plan, generating the task scheduling time according to the CRON expression corresponding to each task of the scheduling client, marking the task state as unexecuted, and recording the task scheduling time point in a sliding time window comprises the following steps:
step 1, generating a CRON expression corresponding to a scheduling task, and calculating a next scheduling time point by taking the current time as a base point;
step 2, searching whether a corresponding time point is included in a current sliding time window, if so, judging whether a planned scheduling set of the scheduling task exists at the time point, and if not, establishing the planned scheduling set;
step 3, judging whether the task corresponding to the time point exists or not, if not, inserting a corresponding task scheduling plan, and marking the task state as unexecuted; judging whether the task corresponding to the time point exists or not, wherein the judging comprises traversing plan scheduling set data according to the service name, the task name and the scheduling time point, and searching whether data equal to the current service name, the task name and the scheduling time point exist in the set or not;
and 4, circulating the steps 1-3, and continuously generating the next scheduling time point by using the last calculated scheduled time point as a base point until the calculated next scheduling time point exceeds the boundary of the pre-execution time window.
2. The method for early warning of scheduling anomalies based on a sliding time window according to claim 1, further comprising: monitoring a scheduling abnormity early warning request of a client, and simultaneously creating a sliding time window;
the creating of the sliding time window comprises:
determining a size of a sliding time window; each sliding time window comprises a plurality of scheduling tasks, and each task is provided with a corresponding task scheduling plan according to a preset time interval in the sliding time window; the task scheduling plan comprises scheduling times or scheduling time of each task according to a preset time interval in a sliding time window.
3. The scheduling anomaly early warning method based on the sliding time window according to claim 2, wherein the sliding time window is described by adopting a double hash table set; wherein, the key corresponding to the first-level hash table of the double hash tables is a time stamp; the value is a task data hash table;
the task data hash table is used for storing corresponding task scheduling plan information in a sliding time window; the key of the task data hash table is a scheduling task code, and the value is the set of the task scheduling plan in the sliding time window; each piece of data corresponds to a task scheduling plan; the set is Treeset structure.
4. The method of claim 1, wherein the step of marking the corresponding task state in the task scheduling plan as executed when the task scheduling information is received comprises:
after receiving task scheduling information of a scheduling client, searching out the earliest scheduling plan in an executed sliding time window interval according to the ascending order of time of the sliding time window through scheduling task codes, and modifying the state of the earliest scheduling plan into an executed state;
and the searching according to the ascending time sequence of the sliding window is performed according to the time boundary which is the difference between the current time and the fault-tolerant time.
5. The scheduling anomaly early warning method based on the sliding time window according to claim 4, wherein the sliding time window comprises a pre-execution interval and an executed interval, and the current time is acquired according to a preset time period to update the sliding time window;
the updating of the sliding time window comprises:
calculating the boundary of a current time window according to the current time, if the calculated pre-execution boundary is larger than the maximum time of the pre-execution interval of the current window, inserting a new timestamp into the pre-execution interval of the window, and updating a first-stage hash table and a task data hash table corresponding to the new timestamp;
calculating the executed boundary, and if the executed interval of the current time window contains a sample window with corresponding time smaller than the executed boundary, removing window data outside the boundary from the time window.
6. The method according to claim 5, wherein when the sliding time window is removed, if there is a task in an unexecuted state, performing exception early warning comprises:
and traversing the scheduling plan task which is correspondingly removed when the sliding time window is removed, if the task which is not executed in the state is searched out, marking the scheduling task as the non-executed abnormal scheduling, recording the abnormal scheduling, and executing early warning.
7. A monitoring server is characterized by comprising a generating module, a transmitting and receiving module, a marking module and an early warning module; wherein:
the generating module is used for generating a task scheduling plan, generating task scheduling time according to a CRON expression corresponding to each task of the scheduling client, marking the task state as unexecuted, and recording the task scheduling time point in a sliding time window; the issuing and receiving module is used for issuing the task scheduling plan to the client and receiving the task scheduling information uploaded when the client starts to schedule at least one task;
the marking module is used for marking the corresponding task state in the task scheduling plan as executed when the task scheduling information is received;
the early warning module is used for executing abnormal early warning if tasks in an unexecuted state exist when the sliding time window is removed;
the generation module is further used for generating a CRON expression corresponding to the scheduling task and calculating the next scheduling time point by taking the current time as a base point; searching whether a corresponding time point is included in a current sliding time window, if so, judging whether a planned dispatching set of the dispatching task exists at the time point, and if not, establishing the planned dispatching set; judging whether the task corresponding to the time point exists or not, if not, inserting a corresponding task scheduling plan, and marking the task state as unexecuted; judging whether the task corresponding to the time point exists or not, wherein the judging comprises traversing plan scheduling set data according to the service name, the task name and the scheduling time point, and searching whether data equal to the current service name, the task name and the scheduling time point exist in the set or not; and continuously generating the next scheduling time point by using the last calculated planned time point as a base point until the calculated next scheduling time point exceeds the boundary of the pre-execution time window.
8. The monitoring server of claim 7, wherein the server further comprises a listening module and a creation module; the monitoring module is used for monitoring scheduling abnormity early warning requests of the client;
the creation module is used for creating a sliding time window; the creating of the sliding time window comprises: determining a size of a sliding time window; each sliding time window comprises a plurality of scheduling tasks, and each task is provided with a corresponding task scheduling plan according to a preset time interval in the sliding time window; the task scheduling plan comprises scheduling times or scheduling time of each task according to a preset time interval in a sliding time window.
9. The monitoring server of claim 8, wherein the creating module is further configured to use a set of double hash tables to describe a sliding time window; wherein, the key corresponding to the first-level hash table of the double hash tables is a time stamp; the value is a task data hash table;
the task data hash table is used for storing corresponding task scheduling plan information in a sliding time window; the key of the task data hash table is a scheduling task code, and the value is the set of the task scheduling plan in the sliding time window; each piece of data corresponds to a task scheduling plan; the set is a Treeset structure.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211197815.4A CN115292141B (en) | 2022-09-29 | 2022-09-29 | Scheduling abnormity early warning method based on sliding time window and monitoring server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211197815.4A CN115292141B (en) | 2022-09-29 | 2022-09-29 | Scheduling abnormity early warning method based on sliding time window and monitoring server |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115292141A CN115292141A (en) | 2022-11-04 |
| CN115292141B true CN115292141B (en) | 2023-02-03 |
Family
ID=83834088
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211197815.4A Active CN115292141B (en) | 2022-09-29 | 2022-09-29 | Scheduling abnormity early warning method based on sliding time window and monitoring server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115292141B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104639388A (en) * | 2014-12-30 | 2015-05-20 | 中国科学院计算机网络信息中心 | DNS server availability detection method based on user perception |
| CN106681808A (en) * | 2016-12-01 | 2017-05-17 | 北京奇虎科技有限公司 | Task scheduling method and device |
| CN107168849A (en) * | 2017-05-02 | 2017-09-15 | 广东网金数据服务有限公司 | A kind of task scheduling operation and monitoring method and device |
| CN113760491A (en) * | 2020-10-30 | 2021-12-07 | 北京沃东天骏信息技术有限公司 | Task scheduling system, method, equipment and storage medium |
| CN114840392A (en) * | 2022-06-09 | 2022-08-02 | 深圳前海微众银行股份有限公司 | Method, apparatus, medium, and program product for monitoring task scheduling exception |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3123371A1 (en) * | 2014-03-26 | 2017-02-01 | Koninklijke Philips N.V. | Device and method for visualizing tasks |
| US10768984B2 (en) * | 2015-06-11 | 2020-09-08 | Honeywell International Inc. | Systems and methods for scheduling tasks using sliding time windows |
| CN110942254A (en) * | 2019-11-29 | 2020-03-31 | 杭州派迩信息技术有限公司 | Scheduling planning and dispatching system for boarding gate service personnel |
| CN111782390B (en) * | 2020-06-28 | 2024-03-22 | 武汉安辰鑫信息科技有限公司 | Scheduling method of timing task and related equipment |
-
2022
- 2022-09-29 CN CN202211197815.4A patent/CN115292141B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104639388A (en) * | 2014-12-30 | 2015-05-20 | 中国科学院计算机网络信息中心 | DNS server availability detection method based on user perception |
| CN106681808A (en) * | 2016-12-01 | 2017-05-17 | 北京奇虎科技有限公司 | Task scheduling method and device |
| CN107168849A (en) * | 2017-05-02 | 2017-09-15 | 广东网金数据服务有限公司 | A kind of task scheduling operation and monitoring method and device |
| CN113760491A (en) * | 2020-10-30 | 2021-12-07 | 北京沃东天骏信息技术有限公司 | Task scheduling system, method, equipment and storage medium |
| CN114840392A (en) * | 2022-06-09 | 2022-08-02 | 深圳前海微众银行股份有限公司 | Method, apparatus, medium, and program product for monitoring task scheduling exception |
Non-Patent Citations (1)
| Title |
|---|
| 面向动态需求的成像卫星滚动式重调度方法研究;习婷等;《中国管理科学》;20151115;275-280 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115292141A (en) | 2022-11-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10491490B2 (en) | Systems and methods of specifying service level criteria | |
| US9400728B2 (en) | Cross platform workflow management | |
| US9552229B2 (en) | Systems and methods for task scheduling | |
| CN108460598B (en) | Real-time automatic supervision reporting system based on double-chain architecture block chain | |
| US10911447B2 (en) | Application error fingerprinting | |
| Ligus | Effective monitoring and alerting | |
| US11954123B2 (en) | Data processing method and device for data integration, computing device and medium | |
| CN111784318A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN112561370A (en) | Software version management method and device, computer equipment and storage medium | |
| JP4309803B2 (en) | Maintenance support program | |
| US9455940B2 (en) | Information processing apparatus and information processing method | |
| US8335759B2 (en) | Work analysis device and recording medium recording work analysis program | |
| CN115292141B (en) | Scheduling abnormity early warning method based on sliding time window and monitoring server | |
| CN109284331A (en) | Accreditation information acquisition method, terminal device and medium based on business datum resource | |
| CN108900497B (en) | Data synchronization method and system between heterogeneous systems | |
| WO2019223171A1 (en) | Workflow management method and system, computer device and storage medium | |
| CN112685360B (en) | Method and device for persistence of memory data, storage medium and computer equipment | |
| CN103309929A (en) | Method and system for storing and retrieving data | |
| Rozum | Defining and understanding software measurement data | |
| JP2006226846A (en) | Data logger device and its data collection method | |
| CN113227978B (en) | Automatic anomaly detection in a computer processing pipeline | |
| JP2008299496A (en) | Operation simulator and program | |
| US20170140308A1 (en) | Tabular phase mapping for cycle time reporting | |
| US20210398084A1 (en) | Time activity processing | |
| CN116703092A (en) | Batch task scheduling method, device, equipment and medium based on resource allocation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |