CN114064230A - Method and device for scheduling offline tasks, computer equipment and storage medium - Google Patents
Method and device for scheduling offline tasks, computer equipment and storage medium Download PDFInfo
- Publication number
- CN114064230A CN114064230A CN202111171121.9A CN202111171121A CN114064230A CN 114064230 A CN114064230 A CN 114064230A CN 202111171121 A CN202111171121 A CN 202111171121A CN 114064230 A CN114064230 A CN 114064230A
- Authority
- CN
- China
- Prior art keywords
- task
- offline
- line
- time
- latest
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 239000008280 blood Substances 0.000 claims abstract description 12
- 210000004369 blood Anatomy 0.000 claims abstract description 12
- 238000004590 computer program Methods 0.000 claims description 21
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 4
- 230000003111 delayed effect Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Retry When Errors Occur (AREA)
Abstract
The application relates to a method and a device for scheduling off-line tasks, computer equipment and a storage medium, wherein historical ending time of a task instance corresponding to each off-line task and current execution time of the off-line task are obtained, the latest tolerance time of each off-line task is calculated according to the historical ending time, the dependency relationship among the off-line tasks is determined according to a blood margin link among the off-line tasks, the off-line tasks are scheduled according to the current execution time, the latest tolerance time and the dependency relationship, the scheduling of the off-line tasks is automatically completed, delay of the execution time among the off-line tasks is avoided, and timeliness of service data is guaranteed.
Description
Technical Field
The present application relates to the field of big data technologies, and in particular, to a method and an apparatus for offline task scheduling, a computer device, and a storage medium.
Background
With continuous iterative updating of a big data offline scheduling scene, more and more services need to be developed by one-time, minute, hour, day, week, month and other offline tasks depending on a big data offline scheduling platform. The multiple off-line tasks can be divided into a preposed off-line task and a postposition off-line task according to a front-back dependency relationship formed by business relationships, and the postposition off-line task can be executed only after the preposed off-line task is successful.
However, for some services whose timeliness requires more than data accuracy, the execution of the post-off-line task cannot be affected after the pre-off-line task is completed. Although the traditional scheduling platform supports users to configure various frequency tasks, the traditional scheduling platform cannot automatically execute the post-off-line task after the pre-off-line task fails, and the execution time of the post-off-line task is delayed, so that the timeliness of service data is poor.
Disclosure of Invention
Therefore, it is necessary to provide a method, an apparatus, a computer device, and a storage medium for offline task scheduling, which automatically execute a post task after a pre task fails, so that the post task is normally executed, and the timeliness of service data is ensured.
In a first aspect, a method for offline task scheduling is provided, where the method includes:
acquiring historical ending time of a task instance corresponding to each offline task and current execution time of the offline task;
calculating the latest tolerance time of each off-line task according to the historical finish time;
determining a dependency relationship between the off-line tasks according to a blood margin link between the off-line tasks;
and scheduling the off-line task according to the current execution time, the latest tolerance time and the dependency relationship.
In some possible implementations, the offline tasks include a pre-offline task and a post-offline task; scheduling the offline tasks according to the current execution time, the latest tolerance time and the dependency relationship, wherein the scheduling comprises the following steps:
and when the dependency relationship meets the condition that the output information of the preposed offline task is the input information of the postposition offline task and the current execution time is greater than the latest tolerance time, setting the preposed offline task to be in a successful state and triggering the postposition offline task.
In some possible implementations, scheduling the offline task according to the current execution time, the latest tolerated time, and the dependency relationship includes:
and when the dependency relationship meets the condition that the output information of the preposed off-line task is the input information of the postposition off-line task and the current execution time is less than the latest tolerance time, executing the preposed off-line task.
In some possible implementations, calculating the latest tolerated time for each offline task according to the historical end time includes:
calculating the normal distribution mean of each offline task according to the historical end time;
and determining the normal distribution mean of each offline task as the latest tolerated time of each offline task.
In some possible implementations, after determining the dependency relationship between the offline tasks according to the bloodline links between the offline tasks, the method further includes:
when the dependency relation does not meet the requirement that the output information of the preposed off-line task is the input information of the postposition off-line task, executing the preposed off-line task;
and triggering the post-positioned task after the pre-positioned off-line task is successfully executed.
In a second aspect, an apparatus for offline task scheduling is provided, the apparatus comprising:
the acquisition module is used for acquiring the historical ending time of the task instance corresponding to each offline task and the current execution time of the offline task;
the calculation module is used for calculating the latest tolerance time of each off-line task according to the historical finish time;
the determining module is used for determining the dependency relationship between the off-line tasks according to the blood margin links between the off-line tasks;
and the scheduling module is used for scheduling the offline tasks according to the current execution time, the latest tolerance time and the dependency relationship.
In some possible implementations, the offline tasks include a pre-offline task and a post-offline task; the scheduling module is specifically configured to:
and when the dependency relationship meets the condition that the output information of the preposed offline task is the input information of the postposition offline task and the current execution time is greater than the latest tolerance time, setting the preposed offline task to be in a successful state and triggering the postposition offline task.
In some possible implementations, the scheduling module is specifically configured to:
and when the dependency relationship meets the condition that the output information of the preposed off-line task is the input information of the postposition off-line task and the current execution time is less than the latest tolerance time, executing the preposed off-line task.
In a third aspect, a computer device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the computer program is executed by the processor, the processor implements the method for offline task scheduling in the first aspect or any one of the implementations of the first aspect.
In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, which, when being executed by a processor, implements the method of offline task scheduling in the first aspect or any one of the implementations of the first aspect.
According to the method, the device, the computer equipment and the storage medium for scheduling the off-line tasks, the historical ending time of the task instance corresponding to each off-line task and the current execution time of the off-line task are obtained, the latest tolerance time of each off-line task is calculated according to the historical ending time, the dependency relationship among the off-line tasks is determined according to the blood margin links among the off-line tasks, the off-line tasks are scheduled according to the current execution time, the latest tolerance time and the dependency relationship, the scheduling of the off-line tasks is automatically completed, the delay of the execution time among the off-line tasks is avoided, and the timeliness of service data is ensured.
Drawings
FIG. 1 is a diagram of an application environment of an offline task scheduling method according to an embodiment of the present application;
FIG. 2 is a flowchart illustrating an offline task scheduling method according to an embodiment of the present application;
FIG. 3 is a block diagram illustrating an offline task scheduling device according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
For the offline scheduling platform, the most important two points are: firstly, the integrity and the correctness of the service data are ensured, and secondly, the dependent post-positioned off-line task is automatically pulled up after the off-line task is successfully executed. Under normal conditions, the preposed off-line task is successfully operated, data is normally transmitted to the postposed off-line task, and the postposed off-line task is triggered to normally operate through the scheduling dependency relationship. However, in some special abnormal scenarios, for example, the preposed offline task has insufficient resources to run slowly, so that the postposed offline task is delayed seriously and cannot be solved in a short time. At this time, if the data of the preposed offline task is not so important, the preposed offline task is required to be subjected to degradation processing, that is, the preposed offline task is immediately ended, and the post offline task is pulled up in time.
The method for scheduling the offline task provided by the present application can be applied to the application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The terminal 102 calculates the latest tolerance time of each offline task according to the historical end time by obtaining the historical end time of the task instance corresponding to each offline task and the current execution time of the offline task, determines the dependency relationship among the offline tasks according to the bloody border links among the offline tasks, and schedules the offline tasks according to the current execution time, the latest tolerance time and the dependency relationship. The terminal 102 may be, but is not limited to, various personal computers, notebook computers, and tablet computers, and the server 104 may be implemented by an independent server or a server cluster composed of a plurality of servers.
In an embodiment, as shown in fig. 2, a method for offline task scheduling is provided, which is described by taking the method as an example applied to the terminal in fig. 1, and includes the following steps:
s210, acquiring the historical ending time of the task instance corresponding to each offline task and the current execution time of the offline task.
And summarizing the offline tasks at regular time, and counting the historical end time of the task instance corresponding to each offline task, wherein the historical end time comprises the end time of the task instance of the offline task in different periods. Meanwhile, the current execution time of the offline task is obtained to be used for judging whether the offline task affects the execution of other offline tasks.
And S220, calculating the latest tolerance time of each off-line task according to the historical finish time.
The latest tolerated time is the latest ending time of the offline task without affecting the execution of other offline tasks. And calculating the latest tolerance time of the offline task according to the ending time of the task instances of the offline task in different periods so as to judge whether the execution time of the offline task affects the execution of other offline tasks.
And S230, determining the dependency relationship between the off-line tasks according to the blood-related links between the off-line tasks.
The blood-border link represents the link relation generated by data, and the dependency relation between the off-line tasks can be clearly known through the blood-border link before the off-line tasks so as to determine the scheduling mode of the off-line tasks.
And S240, scheduling the offline task according to the current execution time, the latest tolerance time and the dependency relationship.
The current execution time, the latest tolerance time and the correlation among the dependency relationships of the offline tasks are comprehensively considered, the scheduling of the offline tasks is automatically completed, the delay of the execution time among the offline tasks is avoided, and the timeliness of the service data is ensured.
In the embodiment of the application, the latest tolerance time of each offline task is calculated according to the historical end time by obtaining the historical end time of the task instance corresponding to each offline task and the current execution time of the offline task, the dependency relationship between the offline tasks is determined according to the bloody border links between the offline tasks, the offline tasks are scheduled according to the current execution time, the latest tolerance time and the dependency relationship, the scheduling of the offline tasks is automatically completed, the delay of the execution time between the offline tasks is avoided, and the timeliness of service data is ensured.
In some embodiments, the offline tasks include a pre-offline task and a post-offline task; scheduling the offline tasks according to the current execution time, the latest tolerance time and the dependency relationship, wherein the scheduling comprises the following steps:
and when the dependency relationship meets the condition that the output information of the preposed offline task is the input information of the postposition offline task and the current execution time is greater than the latest tolerance time, setting the preposed offline task to be in a successful state and triggering the postposition offline task.
And forming a task flow by the plurality of offline tasks according to the business relationship, wherein the former offline task in the task flow is called a front offline task, and the latter offline task is called a rear offline task. The preposed offline task and the postpositional offline task are relative, and after the preposed offline task is successfully executed, the postpositional offline task is used as the preposed offline task of the next offline task in the task flow.
The dependency relationship satisfies that the output information of the preposed offline task is the input information of the post offline task, that is, the preposed offline task must be successfully executed to trigger the post offline task, and a strong dependency relationship exists between the preposed offline task and the post offline task.
The fact that the current execution time is larger than the latest tolerance time shows that the execution time of the preposed offline task can influence the triggering of the postpositional offline task.
When the dependency relationship meets the condition that the output information of the prepositive offline task is the input information of the postpositive offline task and the current execution time is greater than the latest tolerance time, the prepositive offline task is automatically degraded by a killing neglect state machine and is set to be in a successful state, the prepositive offline task is not required to be killed firstly, and then the postpositive offline task is manually set to be in the successful state, so that the delay of the postpositive offline task caused by overtime execution of the prepositive offline task is avoided, and the timeliness of service data is poor.
In some embodiments, scheduling the offline task according to the current execution time, the latest tolerated time, and the dependency relationship includes:
and when the dependency relationship meets the condition that the output information of the preposed off-line task is the input information of the postposition off-line task and the current execution time is less than the latest tolerance time, executing the preposed off-line task.
After the preposed offline task and the post offline task are determined to be in a strong dependency relationship, if the current execution time is less than the latest tolerance time, the execution of the preposed offline task does not influence the post offline task, and the preposed offline task is continuously executed.
In some embodiments, calculating the latest tolerated time for each offline task from the historical end time includes:
calculating the normal distribution mean of each offline task according to the historical end time;
and determining the normal distribution mean of each offline task as the latest tolerated time of each offline task.
And calculating the average number of normal distribution of the historical ending time of the task example, wherein the average value is also called u distribution, and summarizing the latest tolerance time executed by each off-line task according to the 'rule of thumb' of the normal distribution graph, namely that 99.7% of numerical values are distributed within 3 standard deviations from the average value. The latest tolerance time is calculated by adopting a normal distribution algorithm, so that the judgment result of whether each preposed off-line task is degraded is more accurate, and the delay of the execution time of the postposed off-line task is avoided.
In some embodiments, after determining the dependency relationships between the offline tasks from the bloodline links between the offline tasks, the method further comprises:
when the dependency relation does not meet the requirement that the output information of the preposed off-line task is the input information of the postposition off-line task, executing the preposed off-line task;
and triggering the post-positioned task after the pre-positioned off-line task is successfully executed.
When the dependency relationship does not meet the condition that the output information of the front offline task is the input information of the rear offline task, the weak dependency relationship exists between the front offline task and the rear offline task, the input information of the rear offline task does not necessarily need to be the output information of the front offline task, and can also be historical data and the like automatically acquired by the rear offline task. And triggering the post-positioned task after the pre-positioned off-line task is successfully executed.
It should be understood that, although the steps in the flowchart of fig. 2 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in fig. 2 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
In one embodiment, as shown in fig. 3, there is provided an offline task scheduling apparatus, including: an obtaining module 310, a calculating module 320, a determining module 330, and a scheduling module 340, wherein:
an obtaining module 310, configured to obtain a historical end time of a task instance corresponding to each offline task and a current execution time of the offline task;
the calculating module 320 is configured to calculate the latest tolerance time of each offline task according to the historical end time;
a determining module 330, configured to determine a dependency relationship between the offline tasks according to a blood-related link between the offline tasks;
and the scheduling module 340 is configured to schedule the offline task according to the current execution time, the latest tolerated time, and the dependency relationship.
In the embodiment of the application, the preposed offline task does not need to be killed firstly, and then the postposed offline task is manually set to be in a successful state, so that the delay of the postposed offline task caused by overtime execution of the preposed offline task is avoided, and the timeliness of the service data is poor.
In some embodiments, the offline tasks include a pre-offline task and a post-offline task; the scheduling module 340 is specifically configured to:
and when the dependency relationship meets the condition that the output information of the preposed offline task is the input information of the postposition offline task and the current execution time is greater than the latest tolerance time, setting the preposed offline task to be in a successful state and triggering the postposition offline task.
In some embodiments, the scheduling module 340 is specifically configured to:
and when the dependency relationship meets the condition that the output information of the preposed off-line task is the input information of the postposition off-line task and the current execution time is less than the latest tolerance time, executing the preposed off-line task.
In some embodiments, the calculation module 320 is specifically configured to:
calculating the normal distribution mean of each offline task according to the historical end time;
and determining the normal distribution mean of each offline task as the latest tolerated time of each offline task.
In some embodiments, after determining the dependency between the offline tasks according to the bloodline links between the offline tasks, the apparatus further comprises:
the execution module 350 is configured to execute the pre-offline task when the dependency relationship does not satisfy that the output information of the pre-offline task is input information of the post-offline task;
and triggering the post-positioned task after the pre-positioned off-line task is successfully executed.
For specific limitations of the offline task scheduling apparatus, reference may be made to the above limitations of the offline task scheduling method, which is not described herein again. The modules in the offline task scheduling device may be implemented in whole or in part by software, hardware, and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 4. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is for storing offline task scheduling data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement an offline task scheduling method.
Those skilled in the art will appreciate that the architecture shown in fig. 4 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program:
acquiring historical ending time of a task instance corresponding to each offline task and current execution time of the offline task;
calculating the latest tolerance time of each off-line task according to the historical finish time;
determining a dependency relationship between the off-line tasks according to a blood margin link between the off-line tasks;
and scheduling the off-line task according to the current execution time, the latest tolerance time and the dependency relationship.
In some embodiments, the processor, when executing the computer program, further performs the steps of: the off-line tasks comprise a preposed off-line task and a postposed off-line task; scheduling the offline tasks according to the current execution time, the latest tolerance time and the dependency relationship, wherein the scheduling comprises the following steps: and when the dependency relationship meets the condition that the output information of the preposed offline task is the input information of the postposition offline task and the current execution time is greater than the latest tolerance time, setting the preposed offline task to be in a successful state and triggering the postposition offline task.
In some embodiments, the processor, when executing the computer program, further performs the steps of: scheduling the offline tasks according to the current execution time, the latest tolerance time and the dependency relationship, wherein the scheduling comprises the following steps: and when the dependency relationship meets the condition that the output information of the preposed off-line task is the input information of the postposition off-line task and the current execution time is less than the latest tolerance time, executing the preposed off-line task.
In some embodiments, the processor, when executing the computer program, further performs the steps of: calculating the latest tolerance time of each offline task according to the historical end time, wherein the method comprises the following steps: calculating the normal distribution mean of each offline task according to the historical end time; and determining the normal distribution mean of each offline task as the latest tolerated time of each offline task.
In some embodiments, the processor, when executing the computer program, further performs the steps of: after determining the dependency relationships between the offline tasks according to the blood-related links between the offline tasks, the method further comprises: when the dependency relation does not meet the requirement that the output information of the preposed off-line task is the input information of the postposition off-line task, executing the preposed off-line task; and triggering the post-positioned task after the pre-positioned off-line task is successfully executed.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring historical ending time of a task instance corresponding to each offline task and current execution time of the offline task;
calculating the latest tolerance time of each off-line task according to the historical finish time;
determining a dependency relationship between the off-line tasks according to a blood margin link between the off-line tasks;
and scheduling the off-line task according to the current execution time, the latest tolerance time and the dependency relationship.
In one embodiment, the computer program when executed by the processor further performs the steps of: the off-line tasks comprise a preposed off-line task and a postposed off-line task; scheduling the offline tasks according to the current execution time, the latest tolerance time and the dependency relationship, wherein the scheduling comprises the following steps: and when the dependency relationship meets the condition that the output information of the preposed offline task is the input information of the postposition offline task and the current execution time is greater than the latest tolerance time, setting the preposed offline task to be in a successful state and triggering the postposition offline task.
In one embodiment, the computer program when executed by the processor further performs the steps of: scheduling the offline tasks according to the current execution time, the latest tolerance time and the dependency relationship, wherein the scheduling comprises the following steps: and when the dependency relationship meets the condition that the output information of the preposed off-line task is the input information of the postposition off-line task and the current execution time is less than the latest tolerance time, executing the preposed off-line task.
In one embodiment, the computer program when executed by the processor further performs the steps of: calculating the latest tolerance time of each offline task according to the historical end time, wherein the method comprises the following steps: calculating the normal distribution mean of each offline task according to the historical end time; and determining the normal distribution mean of each offline task as the latest tolerated time of each offline task.
In one embodiment, the computer program when executed by the processor further performs the steps of: after determining the dependency relationships between the offline tasks according to the blood-related links between the offline tasks, the method further comprises: when the dependency relation does not meet the requirement that the output information of the preposed off-line task is the input information of the postposition off-line task, executing the preposed off-line task; and triggering the post-positioned task after the pre-positioned off-line task is successfully executed.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method of offline task scheduling, the method comprising:
acquiring historical ending time of a task instance corresponding to each offline task and current execution time of the offline task;
calculating the latest tolerance time of each off-line task according to the historical end time;
determining a dependency relationship between the off-line tasks according to a blood margin link between the off-line tasks;
and scheduling the offline task according to the current execution time, the latest tolerance time and the dependency relationship.
2. The method of claim 1, wherein the offline tasks include a pre-offline task and a post-offline task; the scheduling the offline task according to the current execution time, the latest tolerated time, and the dependency relationship includes:
and when the dependency relationship meets the condition that the output information of the preposed off-line task is the input information of the postposition off-line task and the current execution time is greater than the latest tolerance time, setting the preposed off-line task to be in a successful state and triggering the postposition off-line task.
3. The method of claim 2, wherein the scheduling the offline task according to the current execution time, the latest tolerated time, and the dependency comprises:
and when the dependency relationship meets the condition that the output information of the preposed offline task is the input information of the postposition offline task and the current execution time is less than the latest tolerance time, executing the preposed offline task.
4. The method of claim 1, wherein calculating a latest tolerated time for each of the offline tasks according to the historical end time comprises:
calculating the normal distribution mean of each off-line task according to the historical end time;
and determining the normal distribution mean of each off-line task as the latest tolerance time of each off-line task.
5. The method of claim 2, wherein after determining dependencies between the offline tasks based on the bloodline links between the offline tasks, the method further comprises:
when the dependency relationship does not meet the requirement that the output information of the preposed offline task is the input information of the postposition offline task, executing the preposed offline task;
and triggering the post-positioned task after the pre-positioned off-line task is successfully executed.
6. An apparatus for offline task scheduling, the apparatus comprising:
the acquisition module is used for acquiring the historical ending time of the task instance corresponding to each offline task and the current execution time of the offline task;
the calculation module is used for calculating the latest tolerance time of each off-line task according to the historical end time;
the determining module is used for determining the dependency relationship among the off-line tasks according to the blood margin links among the off-line tasks;
and the scheduling module is used for scheduling the offline task according to the current execution time, the latest tolerance time and the dependency relationship.
7. The apparatus of claim 6, wherein the offline task comprises a pre-offline task and a post-offline task; the scheduling module is specifically configured to:
and when the dependency relationship meets the condition that the output information of the preposed off-line task is the input information of the postposition off-line task and the current execution time is greater than the latest tolerance time, setting the preposed off-line task to be in a successful state and triggering the postposition off-line task.
8. The apparatus of claim 7, wherein the scheduling module is specifically configured to:
and when the dependency relationship meets the condition that the output information of the preposed offline task is the input information of the postposition offline task and the current execution time is less than the latest tolerance time, executing the preposed offline task.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of any of claims 1 to 5 are implemented when the computer program is executed by the processor.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111171121.9A CN114064230A (en) | 2021-10-08 | 2021-10-08 | Method and device for scheduling offline tasks, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111171121.9A CN114064230A (en) | 2021-10-08 | 2021-10-08 | Method and device for scheduling offline tasks, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114064230A true CN114064230A (en) | 2022-02-18 |
Family
ID=80234177
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111171121.9A Pending CN114064230A (en) | 2021-10-08 | 2021-10-08 | Method and device for scheduling offline tasks, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114064230A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105117286A (en) * | 2015-09-22 | 2015-12-02 | 北京大学 | Task scheduling and pipelining executing method in MapReduce |
| CN107239335A (en) * | 2017-06-09 | 2017-10-10 | 中国工商银行股份有限公司 | The job scheduling system and method for distributed system |
| WO2020140683A1 (en) * | 2019-01-04 | 2020-07-09 | 深圳壹账通智能科技有限公司 | Task scheduling method and apparatus, computer device, and storage medium |
| CN112445600A (en) * | 2020-12-15 | 2021-03-05 | 北京首汽智行科技有限公司 | Method and system for issuing offline data processing task |
| CN112817721A (en) * | 2021-01-29 | 2021-05-18 | 中国平安财产保险股份有限公司 | Task scheduling method and device based on artificial intelligence, computer equipment and medium |
-
2021
- 2021-10-08 CN CN202111171121.9A patent/CN114064230A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105117286A (en) * | 2015-09-22 | 2015-12-02 | 北京大学 | Task scheduling and pipelining executing method in MapReduce |
| CN107239335A (en) * | 2017-06-09 | 2017-10-10 | 中国工商银行股份有限公司 | The job scheduling system and method for distributed system |
| WO2020140683A1 (en) * | 2019-01-04 | 2020-07-09 | 深圳壹账通智能科技有限公司 | Task scheduling method and apparatus, computer device, and storage medium |
| CN112445600A (en) * | 2020-12-15 | 2021-03-05 | 北京首汽智行科技有限公司 | Method and system for issuing offline data processing task |
| CN112817721A (en) * | 2021-01-29 | 2021-05-18 | 中国平安财产保险股份有限公司 | Task scheduling method and device based on artificial intelligence, computer equipment and medium |
Non-Patent Citations (1)
| Title |
|---|
| 桑强: "苏宁大数据离线任务开发调度平台时间：开发过程中的要点", Retrieved from the Internet <URL:https://www.infoq.cn/article/tQrEDFduTg1YF71*wEws> * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109743390B (en) | Task scheduling method and device, computer equipment and storage medium | |
| CN109039937B (en) | Dynamic current limiting method, dynamic current limiting device, computer equipment and storage medium | |
| CN110493190B (en) | Data information processing method and device, computer equipment and storage medium | |
| CN110750267A (en) | Service information updating method and device, computer equipment and storage medium | |
| CN111008188B (en) | Data migration method, device, computer equipment and storage medium | |
| CN110851159B (en) | Business rule updating method and device, computer equipment and storage medium | |
| CN110928653B (en) | Cross-cluster task execution method and device, computer equipment and storage medium | |
| CN111667334B (en) | Audit failure order processing method and device, computer equipment and storage medium | |
| CN111193609A (en) | Application abnormity feedback method and device and application abnormity monitoring system | |
| CN111538585B (en) | Js-based server process scheduling method, system and device | |
| CN110659137A (en) | Processing resource allocation method and system for offline tasks | |
| CN108509322B (en) | Method for avoiding excessive return visit, electronic device and computer readable storage medium | |
| CN112132652B (en) | Order information acquisition method and device, computer equipment and storage medium | |
| CN114385485A (en) | Interface test method, device, equipment and storage medium | |
| CN113886590A (en) | Data summarizing method and device, computer equipment and storage medium | |
| CN114064230A (en) | Method and device for scheduling offline tasks, computer equipment and storage medium | |
| CN116827898A (en) | OTA upgrade message pushing method and device, computer equipment and storage medium | |
| CN112286704B (en) | Processing method and device of delay task, computer equipment and storage medium | |
| CN110969430A (en) | Method and device for identifying suspicious user, computer equipment and storage medium | |
| CN114119085A (en) | Method and device for point consumption, computer equipment and storage medium | |
| CN109240906B (en) | Database configuration information adaptation method and device, computer equipment and storage medium | |
| CN111698266A (en) | Service node calling method, device, equipment and readable storage medium | |
| CN112241754B (en) | Online model learning method, system, device and computer readable storage medium | |
| CN111538636B (en) | Computer equipment determination method and device and storage medium | |
| CN109753420B (en) | Monitoring data acquisition method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |