CN112463318A - Timed task processing method, device and system - Google Patents
Timed task processing method, device and system Download PDFInfo
- Publication number
- CN112463318A CN112463318A CN202011285002.1A CN202011285002A CN112463318A CN 112463318 A CN112463318 A CN 112463318A CN 202011285002 A CN202011285002 A CN 202011285002A CN 112463318 A CN112463318 A CN 112463318A
- Authority
- CN
- China
- Prior art keywords
- task
- key
- timing
- redis server
- service node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
- G06F9/4887—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues involving deadlines, e.g. rate based, periodic
Abstract
The application discloses a method, a device and a system for processing a timing task. The method can be applied to a task management module and comprises the following steps: when a task to be executed exists, detecting whether the task is a timing task or not; if the task is a timing task, writing a first key into a redis server based on the task, wherein the expiration time of the first key is the timing execution time of the task, so as to indicate that when the redis server generates an expiration event of the first key based on the expiration time of the first key, a notification is sent to a service node capable of executing the task based on a publish-subscribe mode, and the service node is triggered to execute the task. By the scheme, the execution of the timing task can be realized based on redis. The scheme of the application depends on the characteristics of the redis and the publish-subscribe mode, so that the design complexity can be reduced, the task processing performance can be improved, and the possibility of data loss can be reduced.
Description
Technical Field
The present application belongs to the field of information processing technologies, and in particular, to a timed task processing method, a timed task processing apparatus, and a timed task processing system.
Background
Currently, there are two main implementations for timing tasks: one is based on mysql and the other is based on zookeeper. However, the two implementations have the problems of high design complexity, poor performance and/or easy data loss.
Disclosure of Invention
The application provides a timed task processing method, a timed task processing device and a timed task processing system, which can reduce the design complexity, improve the task processing performance and reduce the possibility of data loss.
In a first aspect, the present application provides a timed task processing method, where the timed task processing method is applied to a task management module, and the timed task processing method includes:
when a task to be executed exists, detecting whether the task is a timing task or not;
and if the task is a timed task, writing a first key into a redis server based on the task, wherein the expiration time of the first key is the timed execution time of the task, so as to indicate that when the redis server generates an expiration event of the first key based on the expiration time of the first key, a notification is sent to a service node capable of executing the task based on a publish-subscribe mode, and the service node is triggered to execute the task.
In a second aspect, the present application provides a method for processing a timed task, where the method for processing a timed task is applied to a service node, and the method for processing a timed task includes:
subscribing an overdue event of a first key on a redis server based on a publishing subscription mode, wherein the first key is a key written into the redis server based on a task to be executed, and the task is a timing task;
and executing the task when receiving the notice generated based on the expired event of the first key.
In a third aspect, the present application provides a timed task processing device, where the timed task processing device is applied to a task management module, and the timed task processing device includes:
the device comprises a detection unit, a processing unit and a processing unit, wherein the detection unit is used for detecting whether a task to be executed is a timing task or not when the task exists;
and a writing unit, configured to write a first key to a redis server based on the task if the task is a timed task, where an expiration time of the first key is a timed execution time of the task, so as to instruct, when the redis server generates an expiration event of the first key based on the expiration time of the first key, to send a notification to a service node capable of executing the task based on a publish-subscribe mode, and to trigger the service node to execute the task.
In a fourth aspect, the present application provides a timed task processing device, where the timed task processing device is applied to a service node, and the timed task processing device includes:
the system comprises a subscription unit, a processing unit and a processing unit, wherein the subscription unit is used for subscribing an expiration event of a first key on a redis server based on a publish-subscribe mode, the first key is a key written into the redis server based on a task to be executed, and the task is a timing task;
and an execution unit for executing the task when receiving a notification generated based on an expiration event of the first key.
In a fifth aspect, the present application provides a timed task processing system, where the timed task processing system includes a task management module, a redis server, and a service node;
the task management module is used for detecting whether the task is a timing task or not when the task to be executed exists; if the task is a timed task, writing a first key into a redis server based on the task, wherein the expiration time of the first key is the timed execution time of the task, so as to indicate that when the redis server generates an expiration event of the first key based on the expiration time of the first key, a notification is sent to a service node capable of executing the task based on a publish-subscribe mode, and the service node is triggered to execute the task;
the server node is used for subscribing the overdue event of the first key on the redis server based on a publish-subscribe mode; and executing the task when receiving the notice generated based on the expired event of the first key.
Compared with the prior art, the application has the beneficial effects that: when a task to be executed exists, whether the task is a timing task is detected, and only when the task is the timing task, a first key is written into a redis server based on the task. Specifically, the expiration time of the first key is a timing execution time of the task, so as to indicate that when the redis server generates an expiration event of the first key based on the expiration time of the first key, a notification is sent to a service node capable of executing the task based on a publish-subscribe mode. After the service node receives the notification, it can begin to perform the task. The above process can realize the execution of the timing task through the redis and the distributed subscription mode. Firstly, since almost any web application program uses redis as a cache to store hotspot information, tokens, authority information and the like, executing a timing task based on the redis does not increase the additional complexity and dependence of the system; secondly, the performance of the redis is very high, so that the performance problem does not exist when the redis is used for executing timing tasks; finally, since redis supports sentinel mode, trunked mode, and persistence, the likelihood of data loss may be reduced. It is understood that the beneficial effects of the second aspect to the fifth aspect can be referred to the related description of the first aspect, and are not described herein again.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a schematic flow chart of an implementation of a timing task processing method provided in an embodiment of the present application;
fig. 2 is a schematic flow chart of an implementation of another method for processing a timed task according to an embodiment of the present application;
fig. 3 is a block diagram of a timed task processing device according to an embodiment of the present application;
FIG. 4 is a block diagram of another timed task processing device provided in an embodiment of the present application;
fig. 5 is a schematic architecture diagram of a timed task processing system according to an embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
The following describes a timed task processing method provided by an embodiment of the present application, where the timed task processing method is applied to a task management module in a timed task processing system. Referring to fig. 1, the method for processing a timed task includes:
in the embodiment of the present application, the tasks initiated by the user through the client, that is, the tasks to be executed, can be divided into two types: one is a timing task; one type is non-timed tasks, i.e., tasks that are executed immediately. After the timed task processing system is started, as long as a task to be executed is reported by a user through a client side through scanning, the task management module can detect the type of the task.
And 102, if the task is a timing task, writing a first key into a redis server based on the task.
In this embodiment of the present application, if it is detected that the task is a timing task, the task management module may enter a processing flow of the timing task. Specifically, for the timing task, there are two cases as follows:
the first case is that the timed execution time of the task is not reached, i.e., the current time is earlier than the timed execution time. In this case, the first key1 may be written to a preset redis server based on the task. The first key is explained below: for a redis server, the actual written data is a key-value pair. In this scheme, only the key (key) in the written key value pair is concerned, and the value (value) in the written key value pair is not concerned. Considering that a task under a single service can be executed at the same time, the first key1 may be determined based on the service to which the task belongs. That is, the first key1 can be uniquely mapped to the business to which the task belongs. It should be noted that the keys mapped by each service are preset by the developer according to a predetermined rule and configuration. When the first key1 is written to the redis server based on the task, an expiration time of the first key1 is also set, and the expiration time is specifically a timing execution time of the task.
For service nodes capable of executing the task, the service nodes subscribe to an expiration event of a key related to a service to which the task belongs in advance based on a publish-subscribe mode; based on this, as time passes, when the expiration time is reached, the first key1 expires, thereby generating an expiration event for the first key 1. Since the service node has subscribed to the expiration event of the key related to the service to which the task belongs in advance, the expiration event of the first key1 is notified to the service node to trigger the service node to execute the task.
For example, at time T0, the Task management module scans a Task1 to be executed, and detects that the Task1 belongs to a timed Task, and the specific timing execution time is time T1. The Task management module writes a unique first key1 (the first key1 uniquely indicates the Task1) to the redis server based on the type of service to which the Task1 belongs, and sets the expiration time of the first key1 to T1. When the time reaches the time T1, the first key1 expires, and at this time, the redis server sends a notification to the service node that subscribes to the expiration event of the first key, so that the service node can know that the Task1 needs to be executed now after receiving the notification.
The second case is that the timed execution time of the task has elapsed, i.e., the current time is later than the timed execution time. At this time, the management of the task is directly ended. I.e. there is no longer a need to perform the task.
Based on the two situations, in this step, when it is determined that the task to be executed is the timed task, it is further necessary to further determine whether the timed execution time is later than the current time, and only when the timed execution time is later than the current time, the first key is written to the redis server based on the task.
In some embodiments, the timing tasks may be further subdivided into two types: one is aperiodic timing tasks (i.e., single timing tasks); the other is a periodic timing task. Considering that the periodic timing task is executed for a plurality of times later, the timing task processing method further includes:
after the task is successfully executed, detecting whether the task is a periodic timing task;
if the task is a periodic timing task, determining a new timing execution time of the task according to the period time of the task;
and returning to execute the step 102, so that the expiration time of the first key stored in the redis server is updated to the new timing execution time.
That is, for the periodic timing task, each time the service node is triggered to execute the task due to the expiration of the first key1, the timing execution time for executing the task next time is determined again according to the cycle time of the task, and the timing execution time is written into the first key1 again according to the timing execution time. The only difference between the newly written first key1 and the previously written first key1 is the expiration time, specifically: the expiration time of the newly written first key1 is the new timing execution time.
For example, at time T0, the Task management module scans a Task1 to be executed, and detects that the Task1 belongs to a timed Task, and the specific timing execution time is time T1. The Task management module writes a unique first key1 (the first key1 uniquely indicates the Task1) to the redis server based on the type of service to which the Task1 belongs, and sets the expiration time of the first key1 to T1. When the time reaches T1, the first key1 expires (when the time expires, the first key1 is deleted from the redis server), and at this time, the redis server sends a notification to the service node that subscribes to the expiration event of the first key, so that the service node can know that the Task1 needs to be executed now after receiving the notification. After the Task1 is successfully executed, it is detected that the Task1 belongs to a periodic timing Task, and the period time is T3. Based on this, the timing execution time (i.e., the new timing execution time) of the Task1 at the next time can be determined to be T1+ T3. The task management module again writes the unique first key1 and sets the expiration time of this first key1 to T1+ T3. Based on the above process, the first key is periodically written, that is, the service node can be periodically triggered to execute the task due to the expiration event of the first key, so that the periodic timing execution of the task is ensured.
In some embodiments, when the task fails to execute, the task management module may rewrite the first key1 into the redis server and set the expiration time of the newly written first key1 to the last few seconds of the current time, facilitating the server node to listen again for the expiration event of the first key, so that the service node can re-execute the task in a short time. For example, the service node receives the notification of the expiration event of the first key1 at time T1, and starts to execute Task 1; for some reason, if the Task1 fails to execute, the Task management module will write the first key1 to the redis server again, and the expiration event of the first key1 is the last seconds of time T1, for example, the last 5 seconds of time T1. That is, at the time of T1+5s, the expiration event of the first key1 is generated again to trigger the service node to execute the Task1 again, which can ensure that the Task1 can be executed successfully finally.
In some embodiments, it may also happen that the service node crashes due to a hardware failure or the like, which may cause a situation that the service node cannot provide service temporarily, which may also cause the task to be unable to be executed smoothly. Based on this, the present application also proposes a backup key, specifically:
before the service node executes the task, writing a second key into a redis server based on the task to serve as a backup of the first key, wherein the expiration time of the second key is set based on the execution time of the task and a preset buffer time so as to indicate that when the redis server generates an expiration event of the second key based on the expiration time of the second key, a notification is sent to a service cluster where the service node is located based on a publish-subscribe mode, and any service node in the service cluster is triggered to execute the task;
and if the task is successfully executed, deleting the second key.
Based on the backup key, there may be several different application scenarios as follows:
in the first application scenario, the service cluster where the service node is located only has the service node. Then the task management module will write a second key2 based on the task before the service node performs the task, which second key2 can be considered a backup of the first key 1; assuming that the service node fails to execute the task due to the crash, and is restarted quickly in a short time, and the restart time is before the expiration time of the second key2, the restarted service node may still receive the notification sent by the redis server based on the expiration event of the second key2 again, and trigger itself to execute the task again, so as to prevent the task from being lost.
In the second application scenario, there may actually be more than two service nodes in the service cluster where the service node is located, that is, more than two service nodes can execute the task currently. Considering that only 1 service node can actually execute the task, the service nodes in the service cluster may first determine only one service node that can currently execute the task by means of obtaining the distributed lock through contention. For convenience of illustration, the service node currently acquiring the distributed lock in the service cluster may be referred to as a target service node. That is, the target service node is the service node that is going to perform the task next. Before the target service node formally executes the task, in order to prevent the occurrence of a situation that the task execution fails due to the crash of the target service node, the task management module may write a second key2 to the redis server based on the task; assuming that the target service node fails to execute the task due to crash and cannot be restarted successfully within a short time, other service nodes in the service cluster except the target service node still can receive the notification sent by the redis server based on the expiration event of the second key2 again, and trigger any other service node to execute the task again in a manner of obtaining a distributed lock through competition, so as to prevent the task from being lost; or, assuming that the target service node fails to execute the task due to the crash, and is restarted quickly in a short time, and the restart time is before the expiration time of the second key2, the restarted target service node may also receive a notification sent by the redis server based on the expiration event of the second key2 together with other service nodes again, and trigger the target service node or any other service node to execute the task again by way of contention to acquire the distributed lock, so as to prevent the task from being lost.
It is noted that, similar to the first key1, the second key2 can be determined based on the business to which the task belongs. That is, the second key2 can also be uniquely mapped to the service to which the task belongs. It should be noted that the keys mapped by each service are preset by the developer according to certain rules and configurations, that is, for each service, the corresponding first key1 and the second key2 as a backup of the first key1 may be preset according to the rules and configurations. When the second key2 is written to the redis server based on the task, an expiration time of the second key2 is also set, and the expiration time is specifically set based on the execution time length of the task and a preset buffer time length.
For example, at time T1, the expiration event of the first key1 is notified to three service nodes, s1, s2, and s 3. Where the target service node s1 acquires the distributed lock, it may perform the task. Before the target service node s1 executes the task, the task management module writes a second key2 to redis based on the task. Assuming that the execution time of the task is 5 seconds and the buffer time is 3 seconds, the second key2 may be set to expire after 8 seconds from time T1; that is, the expiration time of the second key2 is the time T1+8 s.
If the target service node performs the task successfully, the target service node s1 is considered not to be broken down, and the task management module may delete the second key2, so as to prevent other service nodes (i.e., the nodes s2 and s3) from mistaking that the task is not successfully performed due to the occurrence of the expiration event of the second key, and thus, the other nodes perform the task repeatedly. On the contrary, if the target service node fails to execute the task, the target service node s1 is considered to be crashed, and the task management module does not delete the second key2 before the second key2 expires; thus, when the second key2 expires, an expiration event of the second key2 is generated and notified to other service nodes (i.e., nodes s2 and s 3); these other nodes will again contend to acquire the distributed lock and determine a new target serving node among the other nodes to perform the task again.
As can be seen from the above, in the embodiment of the present application, when there is a task to be executed, it is first detected whether the task is a timed task, and only when the task is the timed task, the first key is written to the redis server based on the task. Specifically, the expiration time of the first key is a timing execution time of the task, so as to indicate that when the redis server generates an expiration event of the first key based on the expiration time of the first key, a notification is sent to a service node capable of executing the task based on a publish-subscribe mode. After the service node receives the notification, it can begin to perform the task. The above process can realize the execution of the timing task through the redis and the distributed subscription mode. Firstly, since almost any web application program uses redis as a cache to store hotspot information, tokens, authority information and the like, executing a timing task based on the redis does not increase the additional complexity and dependence of the system; secondly, the performance of the redis is very high, so that the performance problem does not exist when the redis is used for executing timing tasks; finally, since redis supports sentinel mode, trunked mode, and persistence, the likelihood of data loss may be reduced. Furthermore, the distributed lock function is realized, so that when each timed task reaches the timed execution time, only one service node can be allowed to execute the task, and the repeated execution of the task is avoided. And moreover, a backup key is introduced, so that after a service node currently executing a task in the service cluster crashes, the service node which normally runs in the service cluster can be triggered to execute the task again based on the backup key, and the smooth execution of the task is guaranteed.
The following describes a method for processing a timed task provided in an embodiment of the present application, where the method is applied to a service node in a timed task processing system. Referring to fig. 2, the method for processing the timed task includes:
in the embodiment of the application, a service node firstly subscribes an overdue event of a first key on a redis server based on a publish-subscribe mode, wherein the first key is a key written into the redis server based on a task to be executed, the task is a timing task, and the service node can execute the timing task; that is, the service node has the capability of providing services for the service to which the task belongs.
For example, service node s1 is capable of performing task A and incapable of performing task B; the service node s1 would only subscribe to the expiration event of the first key written on the redis server based on timed task a and would not subscribe to the expiration event of the first key written on the redis server based on timed task B.
In some embodiments, with the introduction of the backup mechanism, the server node also subscribes to an expiration event for a second key (i.e., the backup key for the first key) on the redis server. It should be noted that the keys mapped by each service are preset by the research and development personnel according to a certain rule and configuration, that is, for each service, a corresponding first key and a second key serving as a backup of the first key may be preset according to the rule and configuration. The service node may subscribe to the first key and the second key mapped by the self-supported services in advance based on the self-supported condition of each service.
In the embodiment of the application, when the notification generated based on the expiration event of the subscribed first key is received, the task to be executed can be known to reach the timing execution time; at this point, the service node may go to perform the task.
For example, there is currently a service node s1 that is subscribed to an expiration event based on the first key written to the redis server by task A. At time T1, the service node receives the notification of the expiration event of the first key, and the service node knows that the task execution time of task A has been reached, and the service node s1 can start executing task A.
In some embodiments, it may happen that multiple service nodes all subscribe to an expiration event written to the first key of the redis server based on a certain task; that is, in this case, the plurality of service nodes constitute a service cluster. Considering that only one service node is needed to execute the task in the service cluster, the target service node in the service cluster, which executes the task finally, can be selected through the distributed lock to avoid the repeated execution of the task. In this case, the step 202 can be embodied as:
when receiving a notification generated based on an expiration event of the first key, attempting to acquire a distributed lock from the redis server;
and if the distributed lock is successfully acquired from the redis server, executing the task.
Wherein the server node may attempt to acquire a distributed lock by executing the setnx () method; if the return value of setnx () is 1, it represents that the distributed lock is successfully acquired; if the return value of setnx () is 0, it represents that the acquisition of the distributed lock has failed. The principle of executing the setnx () method to attempt to acquire a distributed lock described above is: attempting to write a preset third key3 to the redis server by executing the setnx () method; if the third key3 can be written into the redis server, it is known that the third key3 does not exist in the previous redis server, and the return value of setnx () is 1 at this time; if the third key cannot be written into the redis server, it can be known that the third key3 already exists in the current redis server, that is, other service nodes have acquired the distributed lock by writing the third key3, and at this time, the return value of setnx () is 0.
It should be noted that only the service node (i.e., the target service node) that acquires the distributed lock executes the corresponding task; and service nodes that do not acquire the distributed lock will not do any processing.
In some embodiments, where a backup mechanism is introduced; that is, in the case that the server node further subscribes to an expiration event of a second key (i.e., a backup key of the first key) on the redis server, the server node, regardless of receiving the notification generated based on the expiration event of the first key or receiving the notification generated based on the expiration event of the second key, attempts to acquire the distributed lock, and executes the task corresponding to the first key (or the second key) only when the distributed lock is acquired.
As can be seen from the above, in the embodiment of the present application, the service node can implement execution of the timing task through the redis and the distributed subscription mode. Firstly, since almost any web application program uses redis as a cache to store hotspot information, tokens, authority information and the like, executing a timing task based on the redis does not increase the additional complexity and dependence of the system; secondly, the performance of the redis is very high, so that the performance problem does not exist when the redis is used for executing timing tasks; finally, since redis supports sentinel mode, trunked mode, and persistence, the likelihood of data loss may be reduced. Furthermore, the distributed lock function is realized, so that when each timed task reaches the timed execution time, only one service node can be allowed to execute the task, and the repeated execution of the task is avoided. And moreover, a backup key is introduced, so that after a service node currently executing a task in the service cluster crashes, the service node which normally runs in the service cluster can be triggered to execute the task again based on the backup key, and the smooth execution of the task is guaranteed.
Based on the foregoing timing task processing method applied to the task management module, the embodiment of the present application further provides a timing task processing device applied to the task management module. Referring to fig. 3, the timed task processing apparatus 300 includes:
a detecting unit 301, configured to detect whether a task to be executed is a timing task;
a writing unit 302, configured to write a first key to a redis server based on the task if the task is a timed task, where an expiration time of the first key is a timed execution time of the task, so as to instruct the redis server to send a notification to a service node capable of executing the task based on a publish-subscribe mode when an expiration event of the first key occurs based on the expiration time of the first key, and trigger the service node to execute the task.
Optionally, the writing unit 302 includes:
a time judging subunit, configured to judge whether the timing execution time is later than a current time if the task is a timing task;
and a first key writing subunit, configured to write a first key to the redis server based on the task if the timed execution time is later than the current time.
Optionally, the timed task processing apparatus 300 further includes:
the periodic detection unit is used for detecting whether the task is a periodic timing task or not after the task is successfully executed;
a timing execution time determining unit, configured to determine a new timing execution time of the task according to a cycle time of the task if the task is a periodic timing task;
the writing unit 302 is triggered again after the execution of the timer execution time determination unit is completed, so that the expiration time of the first key stored in the redis server is updated to the new timer execution time.
Alternatively, if there are two or more service nodes capable of executing the task, the writing unit 302 further includes:
a second key writing subunit, configured to write, before the service node executes the task, a second key to a redis server based on the task, as a backup of the first key, where an expiration time of the second key is set based on an execution duration of the task and a preset buffer duration, so as to indicate that when the redis server generates an expiration event of the second key based on the expiration time of the second key, a notification is sent to a service cluster where the service node is located based on a publish-subscribe mode, and any service node in the service cluster is triggered to execute the task;
and a second key deleting subunit, configured to delete the second key if the task is successfully executed.
As can be seen from the above, in the embodiment of the present application, when there is a task to be executed, the timed task processing device may first detect whether the task is a timed task, and only when the task is the timed task, write the first key to the redis server based on the task. Specifically, the expiration time of the first key is a timing execution time of the task, so as to indicate that when the redis server generates an expiration event of the first key based on the expiration time of the first key, a notification is sent to a service node capable of executing the task based on a publish-subscribe mode. After the service node receives the notification, it can begin to perform the task. The above process can realize the execution of the timing task through the redis and the distributed subscription mode. Firstly, since almost any web application program uses redis as a cache to store hotspot information, tokens, authority information and the like, executing a timing task based on the redis does not increase the additional complexity and dependence of the system; secondly, the performance of the redis is very high, so that the performance problem does not exist when the redis is used for executing timing tasks; finally, since redis supports sentinel mode, trunked mode, and persistence, the likelihood of data loss may be reduced. Furthermore, the distributed lock function is realized, so that when each timed task reaches the timed execution time, only one service node can be allowed to execute the task, and the repeated execution of the task is avoided. And moreover, a backup key is introduced, so that after a service node currently executing a task in the service cluster crashes, the service node which normally runs in the service cluster can be triggered to execute the task again based on the backup key, and the smooth execution of the task is guaranteed.
Based on the foregoing timing task processing method applied to the service node, the embodiment of the present application further provides a timing task processing device applied to the service node. Referring to fig. 4, the timed task processing apparatus 400 includes:
a subscribing unit 401, configured to subscribe to an expiration event of a first key on a redis server based on a publish-subscribe mode, where the first key is a key written into the redis server based on a task to be executed, and the task is a timing task;
an executing unit 402, configured to execute the task when receiving a notification generated based on an expiration event of the first key.
Optionally, the executing unit 402 includes:
a distributed lock acquisition subunit configured to attempt to acquire a distributed lock from the redis server when receiving a notification generated based on an expiration event of the first key;
and the task execution subunit is used for executing the task if the distributed lock is successfully acquired from the redis server.
Optionally, the distributed lock acquiring subunit includes:
a third key trial writing subunit, configured to try to write a preset third key to the redis server;
and an obtaining result determining subunit, configured to determine that obtaining the distributed lock from the redis server is successful if the third key can be written into the redis server, and determine that obtaining the distributed lock from the redis server is failed if the third key cannot be written into the redis server.
As can be seen from the above, in the embodiment of the present application, the timed task processing device can implement execution of the timed task through redis and a distributed subscription mode. Firstly, since almost any web application program uses redis as a cache to store hotspot information, tokens, authority information and the like, executing a timing task based on the redis does not increase the additional complexity and dependence of the system; secondly, the performance of the redis is very high, so that the performance problem does not exist when the redis is used for executing timing tasks; finally, since redis supports sentinel mode, trunked mode, and persistence, the likelihood of data loss may be reduced. Furthermore, the distributed lock function is realized, so that when each timed task reaches the timed execution time, only one service node can be allowed to execute the task, and the repeated execution of the task is avoided. And moreover, a backup key is introduced, so that after a service node currently executing a task in the service cluster crashes, the service node which normally runs in the service cluster can be triggered to execute the task again based on the backup key, and the smooth execution of the task is guaranteed.
Referring to fig. 5, an embodiment of the present application further provides a timed task processing system 5, where the timed task processing system includes a task management module 51, a redis server 52, and at least one service node 53 (only one is shown in fig. 5); the task management module 51 may implement the above-mentioned timing task processing method applied to the task management module, and the service node 53 may implement the above-mentioned timing task processing method applied to the service node, which is not described herein again. The timed task processing system relies on redis and a distributed subscription schema to implement the execution of timed tasks. Firstly, since almost any web application program uses redis as a cache to store hotspot information, tokens, authority information and the like, executing a timing task based on the redis does not increase the additional complexity and dependence of the system; secondly, the performance of the redis is very high, so that the performance problem does not exist when the redis is used for executing timing tasks; finally, since redis supports sentinel mode, trunked mode, and persistence, the likelihood of data loss may be reduced.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned functions may be distributed as different functional units and modules according to needs, that is, the internal structure of the system may be divided into different functional units or modules to implement all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed system and method may be implemented in other ways. For example, the above-described system embodiments are merely illustrative, and for example, the division of the above-described modules or units is only one logical functional division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of modules or units through some interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated modules/units described above, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow in the method of the embodiments described above may be implemented by a computer program, which may be stored in a computer readable storage medium and used by a processor to implement the steps of the embodiments of the methods described above. The computer program includes computer program code, and the computer program code may be in a source code form, an object code form, an executable file or some intermediate form. The computer readable medium may include: any entity or device capable of carrying the above-mentioned computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signal, telecommunication signal, software distribution medium, etc. It should be noted that the computer readable medium described above may be suitably increased or decreased as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media excludes electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.
The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.
Claims (10)
1. A timing task processing method is applied to a task management module and comprises the following steps:
when a task to be executed exists, detecting whether the task is a timing task or not;
if the task is a timing task, writing a first key into a redis server based on the task, wherein the expiration time of the first key is the timing execution time of the task, so as to indicate that when the redis server generates an expiration event of the first key based on the expiration time of the first key, a notification is sent to a service node capable of executing the task based on a publish-subscribe mode, and the service node is triggered to execute the task.
2. The timed task processing method according to claim 1, wherein said writing a first key to a redis server based on the task if the task is a timed task comprises:
if the task is a timing task, judging whether the timing execution time is later than the current time;
and if the timing execution time is later than the current time, writing the first key into a redis server based on the task.
3. The timed task processing method according to claim 1, characterized in that it further comprises:
after the task is successfully executed, detecting whether the task is a periodic timing task;
if the task is a periodic timing task, determining the new timing execution time of the task according to the periodic time of the task;
and returning to the step of executing the writing of the first key to the redis server based on the task, so that the expiration time of the first key stored in the redis server is updated to the new timing execution time.
4. The timed task processing method according to claim 1, characterized in that it further comprises:
before the service node executes the task, writing a second key to a redis server based on the task to serve as a backup of the first key, wherein the expiration time of the second key is set based on the execution time of the task and a preset buffer time so as to indicate that when the redis server generates an expiration event of the second key based on the expiration time of the second key, a notification is sent to a service cluster where the service node is located based on a publish-subscribe mode, and any service node in the service cluster is triggered to execute the task;
and if the task is successfully executed, deleting the second key.
5. A method for processing a timing task is applied to a service node, and comprises the following steps:
subscribing an expiration event of a first key on a redis server based on a publishing subscription mode, wherein the first key is a key written into the redis server based on a task to be executed, and the task is a timing task;
executing the task upon receiving a notification generated based on an expiration event of the first key.
6. The timed task processing method according to claim 5, wherein said executing said task when receiving a notification generated on the basis of an expiration event of said first key, comprises:
attempting to acquire a distributed lock to the redis server upon receiving a notification generated based on an expiration event of the first key;
and if the distributed lock is successfully acquired from the redis server, executing the task.
7. The timed task processing method according to claim 6, wherein said attempting to acquire a distributed lock from said redis server comprises:
attempting to write a preset third key to the redis server;
if the third key can be written into the redis server, determining that the distributed lock is successfully acquired from the redis server;
and if the third key cannot be written into the redis server, determining that the distributed lock acquisition from the redis server fails.
8. A timed task processing device is applied to a task management module, and comprises:
the device comprises a detection unit and a processing unit, wherein the detection unit is used for detecting whether a task to be executed is a timing task or not when the task exists;
and a writing unit, configured to write a first key to a redis server based on the task if the task is a timed task, where an expiration time of the first key is a timed execution time of the task, so as to indicate that when the redis server generates an expiration event of the first key based on the expiration time of the first key, a notification is sent to a service node capable of executing the task based on a publish-subscribe mode, and the service node is triggered to execute the task.
9. A timed task processing device, wherein the timed task processing device is applied to a service node, and the timed task processing device comprises:
the subscription unit is used for subscribing an expiration event of a first key on a redis server based on a publish-subscribe mode, wherein the first key is a key written into the redis server based on a task to be executed, and the task is a timing task;
an execution unit for executing the task when receiving a notification generated based on an expiration event of the first key.
10. A timing task processing system is characterized by comprising a task management module, a redis server and a service node;
the task management module is used for detecting whether the task is a timing task or not when the task to be executed exists; if the task is a timing task, writing a first key into a redis server based on the task, wherein the expiration time of the first key is the timing execution time of the task, so as to indicate that when the redis server generates an expiration event of the first key based on the expiration time of the first key, a notification is sent to a service node capable of executing the task based on a publish-subscribe mode, and the service node is triggered to execute the task;
the server node is used for subscribing the overdue event of the first key on the redis server based on a publish-subscribe mode; executing the task upon receiving a notification generated based on an expiration event of the first key.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011285002.1A CN112463318A (en) | 2020-11-17 | 2020-11-17 | Timed task processing method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011285002.1A CN112463318A (en) | 2020-11-17 | 2020-11-17 | Timed task processing method, device and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112463318A true CN112463318A (en) | 2021-03-09 |
Family
ID=74837918
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011285002.1A Pending CN112463318A (en) | 2020-11-17 | 2020-11-17 | Timed task processing method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112463318A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113238862A (en) * | 2021-05-13 | 2021-08-10 | 京东数字科技控股股份有限公司 | Distributed task scheduling method and device |
| CN113282580A (en) * | 2021-04-28 | 2021-08-20 | 青岛海尔科技有限公司 | Method, storage medium and server for executing timed task |
| CN115048207A (en) * | 2022-08-17 | 2022-09-13 | 恒丰银行股份有限公司 | Task execution method, device and medium based on fixed time interval |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105677536A (en) * | 2016-01-08 | 2016-06-15 | 上海斐讯数据通信技术有限公司 | Implementing method for task messages and task system for implementing task messages |
| CN109582466A (en) * | 2017-09-29 | 2019-04-05 | 北京金山软件有限公司 | A kind of timed task executes method, distributed server cluster and electronic equipment |
| CN109710394A (en) * | 2018-12-27 | 2019-05-03 | 出门问问信息科技有限公司 | Timing task processing system and method |
| CN110196765A (en) * | 2019-05-30 | 2019-09-03 | 苏州浪潮智能科技有限公司 | Timed task processing method, device, equipment and computer readable storage medium |
| CN110837409A (en) * | 2018-08-16 | 2020-02-25 | 北京京东尚科信息技术有限公司 | Method and system for executing task regularly |
| CN111190732A (en) * | 2019-12-27 | 2020-05-22 | 成都欧珀通信科技有限公司 | Timed task processing system and method, storage medium and electronic device |
| CN111210270A (en) * | 2020-01-02 | 2020-05-29 | 广州虎牙科技有限公司 | Business processing method and device, service cluster, computer equipment and storage medium |
| CN111767128A (en) * | 2020-06-23 | 2020-10-13 | 北京字节跳动网络技术有限公司 | Method and device for executing timing task |
-
2020
- 2020-11-17 CN CN202011285002.1A patent/CN112463318A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105677536A (en) * | 2016-01-08 | 2016-06-15 | 上海斐讯数据通信技术有限公司 | Implementing method for task messages and task system for implementing task messages |
| CN109582466A (en) * | 2017-09-29 | 2019-04-05 | 北京金山软件有限公司 | A kind of timed task executes method, distributed server cluster and electronic equipment |
| CN110837409A (en) * | 2018-08-16 | 2020-02-25 | 北京京东尚科信息技术有限公司 | Method and system for executing task regularly |
| CN109710394A (en) * | 2018-12-27 | 2019-05-03 | 出门问问信息科技有限公司 | Timing task processing system and method |
| CN110196765A (en) * | 2019-05-30 | 2019-09-03 | 苏州浪潮智能科技有限公司 | Timed task processing method, device, equipment and computer readable storage medium |
| CN111190732A (en) * | 2019-12-27 | 2020-05-22 | 成都欧珀通信科技有限公司 | Timed task processing system and method, storage medium and electronic device |
| CN111210270A (en) * | 2020-01-02 | 2020-05-29 | 广州虎牙科技有限公司 | Business processing method and device, service cluster, computer equipment and storage medium |
| CN111767128A (en) * | 2020-06-23 | 2020-10-13 | 北京字节跳动网络技术有限公司 | Method and device for executing timing task |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113282580A (en) * | 2021-04-28 | 2021-08-20 | 青岛海尔科技有限公司 | Method, storage medium and server for executing timed task |
| CN113238862A (en) * | 2021-05-13 | 2021-08-10 | 京东数字科技控股股份有限公司 | Distributed task scheduling method and device |
| CN113238862B (en) * | 2021-05-13 | 2024-04-16 | 京东科技控股股份有限公司 | Distributed task scheduling method and device |
| CN115048207A (en) * | 2022-08-17 | 2022-09-13 | 恒丰银行股份有限公司 | Task execution method, device and medium based on fixed time interval |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112463318A (en) | Timed task processing method, device and system | |
| CN109101341B (en) | Distribution method and equipment of distributed lock | |
| US6687847B1 (en) | Failure detector with consensus protocol | |
| US9778998B2 (en) | Data restoration method and system | |
| CN109788027B (en) | Data synchronization method, device, server and computer storage medium | |
| JP2022013854A (en) | Method and device for updating database by using two-phase commit distributed transaction | |
| CN112367149B (en) | Message acquisition method, device, equipment and storage medium | |
| CN113342507B (en) | Distributed lock service realization method and device and computer equipment | |
| JP7250121B2 (en) | Method and Apparatus for Continuously Ensuring Device Operation Reliability in Cloud Degraded Mode | |
| US7171410B1 (en) | Fault tolerant network element | |
| WO2017080362A1 (en) | Data managing method and device | |
| CN111708775A (en) | Self-adding ID generation method, device and system | |
| CN111092956A (en) | Resource synchronization method, device, storage medium and equipment | |
| CN114679464B (en) | Data rollback method, device, equipment and storage medium based on distributed lock | |
| CN113032021B (en) | System switching and data processing method, device, equipment and storage medium thereof | |
| CN109602413B (en) | Heartbeat detection method, heartbeat detection device, storage medium and server | |
| CN111625402A (en) | Data recovery method and device, electronic equipment and computer readable storage medium | |
| CN111385356B (en) | Distributed session processing method, device and system | |
| CN114979153B (en) | Load balancing method, computer equipment and storage device | |
| EP4333469A1 (en) | User data disaster tolerance method and apparatus, network element device, and storage medium | |
| EP4170518A1 (en) | Distributed transaction processing method, terminal and computer readable storage medium | |
| WO2023029837A1 (en) | Request processing method and apparatus, computing device and storage medium | |
| CN111639089B (en) | Transaction processing method, transaction processing device, electronic equipment and computer readable storage medium | |
| WO2021168654A1 (en) | Method and apparatus for detecting synchronized data, server, and data synchronization system | |
| WO2023005484A1 (en) | User information registration method and apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |