CN110737526A - method and device for managing timed tasks under Redis-based distributed cluster - Google Patents
method and device for managing timed tasks under Redis-based distributed cluster Download PDFInfo
- Publication number
- CN110737526A CN110737526A CN201911003752.2A CN201911003752A CN110737526A CN 110737526 A CN110737526 A CN 110737526A CN 201911003752 A CN201911003752 A CN 201911003752A CN 110737526 A CN110737526 A CN 110737526A
- Authority
- CN
- China
- Prior art keywords
- task
- timing
- redis
- timed
- timing task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000012545 processing Methods 0.000 claims abstract description 16
- 230000001960 triggered effect Effects 0.000 claims abstract description 11
- 238000007726 management method Methods 0.000 claims abstract description 8
- 230000008569 process Effects 0.000 claims abstract description 7
- 230000004048 modification Effects 0.000 claims description 14
- 238000012986 modification Methods 0.000 claims description 14
- 238000012217 deletion Methods 0.000 claims description 9
- 230000037430 deletion Effects 0.000 claims description 9
- 230000004083 survival effect Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003862 health status Effects 0.000 description 1
- 210000001503 joint Anatomy 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
Abstract
The invention provides timed task management methods and devices under a Redis-based distributed cluster, which are applied to the technical field of timed task processing, and the methods comprise that at least application programs send timed tasks, a task monitoring program acquires timed task information, processes the timed tasks and a timer according to the type of the timed tasks, writes the timed tasks into a Redis task execution queue when the timer of the task monitoring program is triggered, monitors the Redis task execution queue by the application program, reads and consumes the timed tasks in the Redis task execution queue according to the preemption characteristic of the queue.
Description
Technical Field
The invention relates to the technical field of timed task processing, in particular to timed task management methods and devices based on a Redis distributed cluster.
Background
The timed task system is a service system for executing tasks periodically or periodically according to a specified periodic strategy.
The conventional task is that the configuration is appointed, the configuration is read when the application program is started, the timer is started according to the timing task information in the configuration, and after the timing task in the configuration is changed, the application program needs to be restarted to read the timing task information again. With the increase of the traffic, the program is usually deployed in a cluster manner, a large number of timing tasks need to be run, timing task information in configurations in each cluster needs to be modified for any change of the timing tasks, the application programs in the whole cluster need to be restarted, each application program reads the timing tasks, and the tasks are repeatedly executed.
Disclosure of Invention
In view of the above drawbacks of the prior art, an object of the present invention is to provide methods and apparatuses for managing timed tasks in a Redis-based distributed cluster, which are to read all information of timed tasks by a task listener and respectively start a timer, place the timed tasks into a Redis task execution queue to wait for consumption when the timed tasks are triggered, monitor the Redis task execution queue by all applications in the cluster, and execute the tasks when the tasks are consumed.
To achieve the above and other related objects, the present invention provides methods for managing timed tasks in a Redis-based distributed cluster, the method comprising:
at least applications send timed tasks;
the task monitoring program acquires timing task information, processes the timing task and a timer according to the type of the timing task, and writes the timing task into a redis task execution queue when the timer of the task monitoring program is triggered;
and the application program monitors a redis task execution queue and reads and consumes the timing task in the redis task execution queue according to the preemption characteristic of the queue.
, the step of sending the timing task by at least applications includes:
when every application programs determine that the timing task is the creation timing task, the timing task is sent to a redis newly-added task queue;
when every application programs determine that the timing task is the creation and deletion timing task, sending the timing task to a redis deletion task queue;
every applications send timed tasks to the redis modified task queue upon determining that the timed task is an create modify time task.
, the step of the task listener acquiring the information of the timed task and processing the timed task and the timer according to the type of the timed task includes:
a task monitoring program acquires timing task information;
when the timing task is determined to be the created timing task, setting a timer for the timing task to be created;
or;
when the timing task is determined to be a deleted timing task, deleting a timer corresponding to the timing task;
or;
and when the timing task is determined to be the modification timing task, a timer corresponding to the timing task is established, and a new timer is established according to the timing task information after the current modification.
, the step of sending the timing task by at least applications includes:
at least applications send timed tasks to the database;
and adding, modifying and deleting the timing task information in the database through the application program.
, the method further comprises:
the task monitoring program reads the timing task information from the database after being started;
the task listener directly receives at least application sending timed tasks after starting.
, the task monitor includes a main task monitor and a standby task monitor;
starting a main task monitor, and not starting a standby task monitor;
deploying keepAlive, setting connection frequency, and configuring node information of a main task monitor and node information of a standby task monitor in the keepAlive;
keepAlive starts, data are sent to the main task monitoring program according to a preset time interval, and if the data are available, the main task monitoring program is in a survival state;
if the data is not reachable, continuing to send the data and canceling the connection after the data is overtime, and determining that the main task monitoring program is unavailable;
and starting a task monitoring program in the standby node, and processing the timing task and the timer according to the type of the timing task.
In addition, the invention also discloses kinds of timed task management devices based on the Redis distributed cluster, and the device comprises:
at least applications for sending timed tasks;
the task monitoring program is used for acquiring timing task information, processing a timing task and a timer according to the type of the timing task, and writing the timing task into a redis task execution queue when the timer of the task monitoring program is triggered; and enabling the application program to monitor the redis task execution queue and read and consume the timing task in the redis task execution queue according to the preemption characteristic of the queue.
As described above, in the timed task management methods and apparatuses under the Redis-based distributed cluster according to embodiments of the present invention, task monitoring programs are started to read all timed task information and respectively start timers, the timed task is placed in a Redis task execution queue to wait for consumption when the timed task is triggered, all applications in the cluster monitor the Redis task execution queue, and the tasks are executed when the tasks are consumed, and the timed task information is completely recorded in a database.
Therefore, the embodiment of the invention changes the timing task by directly establishing, deleting or modifying the timer in the task monitor program without restarting the application program, thereby realizing a method that the application program does not need to be restarted, reducing the times of restarting the application program in the production environment, reducing the maintenance cost, saving manpower and time, providing the capability of the application program for competing tasks by the preemption characteristic of the queue among every application programs, ensuring that the tasks are not repeatedly executed, and increasing the high availability of the system by setting the main and standby schemes of the task monitor program.
Drawings
Fig. 1 is a flowchart illustration of methods for managing timed tasks in a Redis-based distributed cluster according to an embodiment of the present invention.
Fig. 2 is a schematic diagram of specific applications of methods for managing timed tasks in a Redis-based distributed cluster according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of a second specific application of methods for managing timed tasks in a Redis-based distributed cluster according to an embodiment of the present invention.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention.
It should be noted that, referring to fig. 1-3, the drawings provided in the present embodiment are only schematic and illustrate the basic concept of the present invention, and the elements related to the present invention are not drawn according to the number, shape and size of the elements in actual implementation, and the type, number and ratio of the elements in actual implementation may be random variations, and the layout of the elements may be more complex.
As shown in fig. 1, an embodiment of the present invention provides methods for managing timed tasks in a Redis-based distributed cluster, where the methods include:
s101, sending timing tasks by at least application programs.
It can be understood that, when a user needs to perform task processing, the user can customize a task through the operation of the application program, for example, the creation of a timed task or the modification of the timed task is performed through the application program, and when the task no longer needs to be performed, the user needs to perform the deletion of the timed task.
Therefore, the type of the timing task can be the creation, modification and deletion of the pair of timing tasks, and therefore, the task sent by the application program can be any of the three or any combination of the three, but the type can only be of the tasks, for example, the creation of the task.
It can be understood that the number of the applications under the cluster may be multiple, as shown in fig. 2, the number of the applications includes application 1, application 2, and application 3, and the number of the applications sent at the same time as may be or multiple, the timing task sent by the application needs to have a receiving party, and the receiving party may directly perform processing arrangement of the task or may be a buffer device of the task.
The receiving of the timing task is realized by the interface butt joint of the database and the application program, in addition, the application program and the database can be directly communicated, the adding operation, the modifying operation and the deleting operation of the timing task are realized, and the processing of the timing task is realized.
Specifically, according to different types of the timing tasks, classified storage can be performed so as to improve the processing efficiency of the subsequent timing tasks, and the step of sending the timing tasks by at least application programs includes:
the method comprises the steps that every application programs send timing tasks to a redis newly-added task queue when the timing tasks are determined to be creation timing tasks, every application programs send the timing tasks to a redis deletion task queue when the timing tasks are determined to be creation deletion timing tasks, and every application programs send the timing tasks to a redis modification task queue when the timing tasks are determined to be creation deletion modification tasks.
It will be appreciated that every task queues may work in the same manner as a FIFO, i.e. first-in-first-out, the first timed task to enter the queue will be stored at the very front of the queue for the first read and process.
S102, the task monitor program obtains the timing task information, processes the timing task and the timer according to the type of the timing task, and writes the timing task into a redis task execution queue when the timer of the task monitor program is triggered.
It should be understood that in the embodiment of the present invention, the task listener performs a process of associating the timing task with the timer, that is, timers are set for every timing tasks, so that when the timer is triggered, the next processing tasks for the timing task are triggered.
In order to not miss the timing task, all task information is received by the database, and simultaneously the task information is written into a task operation (such as adding, modifying and deleting tasks) queue, when the task monitor is started, the tasks in the task operation queue are emptied, all task information in the database is read simultaneously, a timer is started for the tasks, and then the timing task is continuously received directly through the task operation queue.
Thus, in the event that the task listener is not started, the timed task in the database is modified (e.g., any or more of delete, modify, or add modifications) and the modified timed task is received after the task listener is started.
Therefore, the embodiment of the invention provides two processing modes, namely that the task monitoring program reads the timing task information from the database after being started, and the task monitoring program directly receives at least application sending timing tasks after being started.
As can be understood from the above, after the classification storage of the timed tasks has been performed and the processing of the tasks is required to be performed next, the task listener obtains the timed task information and then obtains the types of every timed tasks, including:
when the timing task is determined to be the created timing task, setting a timer for the timing task to be created;
or;
when the timing task is determined to be a deleted timing task, deleting a timer corresponding to the timing task;
or;
and when the timing task is determined to be the modification timing task, a timer corresponding to the timing task is established, and a new timer is established according to the timing task information after the current modification.
Therefore, as shown in FIG. 3, the task listener processes the newly changed timed task, creates a new timer for the timed task from the time when the redis newly adds the task queue to consume the timed task information, deletes the timers before the timed task and creates timers according to the currently modified timed task information from the time when the redis modifies the task queue to consume the timed task information, and deletes the timer corresponding to the timed task from the time when the redis deletes the task queue to consume the timed task information.
Any timers started in the task listener will immediately write task information into the redis task execution queue upon triggering, waiting to be consumed.
S103, the application program monitors a redis task execution queue, and reads and consumes the timing task in the redis task execution queue according to the preemption characteristic of the queue.
All applications monitor the redis task execution queue and, based on the preemptive nature of the queue, only applications can successfully consume the timed task information and execute the task.
In another implementation modes of the present invention, the task monitor is designed to use a main task monitor and a standby task monitor, and the health status of the main node is continuously detected through keepAlive, which is a mechanism that nodes in TCP can detect dead connection, if the service of the task monitor on the main task monitor or the main task monitor does not exist, the specific implementation mode of the standby task monitor is started as follows:
1. starting a main task monitor program, and not starting a standby task monitor program;
2. deploying keepAlive, setting connection frequency, and configuring the information of the main task monitoring program in the keepAlive;
3, keepAlive starts, and sends data to the main task monitor program every time intervals, if the data can be reached, the main task monitor program is in a survival state;
4. if the data is not reachable, the data is continuously sent and the connection is cancelled after time-out, the main task monitor is considered to be unavailable, and then the script is called to start the standby task monitor.
The high availability of the system is increased through the main and standby scheme of the task monitor.
In addition, the invention also discloses kinds of timed task management devices based on the Redis distributed cluster, and the device comprises:
at least applications for sending timed tasks;
the task monitoring program is used for acquiring timing task information, processing a timing task and a timer according to the type of the timing task, and writing the timing task into a redis task execution queue when the timer of the task monitoring program is triggered; and enabling the application program to monitor the redis task execution queue and read and consume the timing task in the redis task execution queue according to the preemption characteristic of the queue.
It will be appreciated by those skilled in the art that modifications and variations can be made to the disclosed embodiments without departing from the spirit and scope of the invention, and therefore, is equivalent to modifications and variations that would be apparent to those skilled in the art without departing from the spirit and scope of the invention as disclosed in the appended claims.
Claims (7)
1, timing task management method under Redis-based distributed cluster, characterized in that, the method includes:
at least applications send timed tasks;
the task monitoring program acquires timing task information, processes the timing task and a timer according to the type of the timing task, and writes the timing task into a redis task execution queue when the timer of the task monitoring program is triggered;
and the application program monitors a redis task execution queue and reads and consumes the timing task in the redis task execution queue according to the preemption characteristic of the queue.
2. The method for managing timed tasks under Redis-based distributed cluster according to claim 1, wherein said step of sending timed tasks by at least applications comprises:
when every application programs determine that the timing task is the creation timing task, the timing task is sent to a redis newly-added task queue;
when every application programs determine that the timing task is the creation and deletion timing task, sending the timing task to a redis deletion task queue;
every applications send timed tasks to the redis modified task queue upon determining that the timed task is an create modify time task.
3. The method for managing the timed task under the Redis-based distributed cluster according to claim 2, wherein the step of the task listener acquiring the information of the timed task and processing the timed task and the timer according to the type of the timed task comprises:
a task monitoring program acquires timing task information;
when the timing task is determined to be the created timing task, setting a timer for the timing task to be created;
or;
when the timing task is determined to be a deleted timing task, deleting a timer corresponding to the timing task;
or;
and when the timing task is determined to be the modification timing task, a timer corresponding to the timing task is established, and a new timer is established according to the timing task information after the current modification.
4. The method for managing timed tasks under Redis-based distributed cluster according to claim 1, wherein said step of sending timed tasks by at least applications comprises:
at least applications send timed tasks to the database;
and adding, modifying and deleting the timing task information in the database through the application program.
5. The method for timed task management under Redis-based distributed cluster according to claim 4, characterized in that it further comprises:
the task monitoring program reads the timing task information from the database after being started;
the task listener directly receives at least application sending timed tasks after starting.
6. The method for managing timed tasks under Redis-based distributed cluster according to any of the claims 1-5 and , wherein the task monitor comprises a main task monitor, a standby task monitor;
starting a main task monitor, and not starting a standby task monitor;
deploying keepAlive, setting connection frequency, and configuring node information of a main task monitor and node information of a standby task monitor in the keepAlive;
keepAlive starts, data are sent to the main task monitoring program according to a preset time interval, and if the data are available, the main task monitoring program is in a survival state;
if the data is not reachable, continuing to send the data and canceling the connection after the data is overtime, and determining that the main task monitoring program is unavailable;
and starting a task monitoring program in the standby node, and processing the timing task and the timer according to the type of the timing task.
7, A timed task management device under Redis-based distributed cluster, the device comprising:
at least applications for sending timed tasks;
the task monitoring program is used for acquiring timing task information, processing a timing task and a timer according to the type of the timing task, and writing the timing task into a redis task execution queue when the timer of the task monitoring program is triggered; and enabling the application program to monitor the redis task execution queue and read and consume the timing task in the redis task execution queue according to the preemption characteristic of the queue.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911003752.2A CN110737526A (en) | 2019-10-22 | 2019-10-22 | method and device for managing timed tasks under Redis-based distributed cluster |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911003752.2A CN110737526A (en) | 2019-10-22 | 2019-10-22 | method and device for managing timed tasks under Redis-based distributed cluster |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110737526A true CN110737526A (en) | 2020-01-31 |
Family
ID=69270703
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911003752.2A Pending CN110737526A (en) | 2019-10-22 | 2019-10-22 | method and device for managing timed tasks under Redis-based distributed cluster |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110737526A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112711474A (en) * | 2021-01-12 | 2021-04-27 | 百果园技术(新加坡)有限公司 | Timer management method and device and electronic equipment |
| CN113407175A (en) * | 2021-06-24 | 2021-09-17 | 上海中通吉网络技术有限公司 | Timed task management system |
| CN114063936A (en) * | 2022-01-18 | 2022-02-18 | 苏州浪潮智能科技有限公司 | Method, system, equipment and storage medium for optimizing timing task |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1873615A (en) * | 2006-01-20 | 2006-12-06 | 华为技术有限公司 | Method for servicing task of timer |
| CN104536809A (en) * | 2014-11-26 | 2015-04-22 | 上海瀚之友信息技术服务有限公司 | Distributed timing task scheduling system based on client and server system |
| CN106993037A (en) * | 2017-03-31 | 2017-07-28 | 山东超越数控电子有限公司 | A kind of method that load-balanced server based on distributed system realizes high availability |
| CN109101370A (en) * | 2018-08-30 | 2018-12-28 | 重庆富民银行股份有限公司 | The date storage method of principal and subordinate's bi-directional synchronization |
-
2019
- 2019-10-22 CN CN201911003752.2A patent/CN110737526A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1873615A (en) * | 2006-01-20 | 2006-12-06 | 华为技术有限公司 | Method for servicing task of timer |
| CN104536809A (en) * | 2014-11-26 | 2015-04-22 | 上海瀚之友信息技术服务有限公司 | Distributed timing task scheduling system based on client and server system |
| CN106993037A (en) * | 2017-03-31 | 2017-07-28 | 山东超越数控电子有限公司 | A kind of method that load-balanced server based on distributed system realizes high availability |
| CN109101370A (en) * | 2018-08-30 | 2018-12-28 | 重庆富民银行股份有限公司 | The date storage method of principal and subordinate's bi-directional synchronization |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112711474A (en) * | 2021-01-12 | 2021-04-27 | 百果园技术(新加坡)有限公司 | Timer management method and device and electronic equipment |
| CN112711474B (en) * | 2021-01-12 | 2023-12-08 | 百果园技术(新加坡)有限公司 | Timer management method and device and electronic equipment |
| CN113407175A (en) * | 2021-06-24 | 2021-09-17 | 上海中通吉网络技术有限公司 | Timed task management system |
| CN114063936A (en) * | 2022-01-18 | 2022-02-18 | 苏州浪潮智能科技有限公司 | Method, system, equipment and storage medium for optimizing timing task |
| CN114063936B (en) * | 2022-01-18 | 2022-03-22 | 苏州浪潮智能科技有限公司 | Method, system, equipment and storage medium for optimizing timing task |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110737526A (en) | method and device for managing timed tasks under Redis-based distributed cluster | |
| US10091307B2 (en) | Application management method and terminal | |
| JP5191062B2 (en) | Storage control system, operation method related to storage control system, data carrier, and computer program | |
| CN113656157B (en) | Distributed task scheduling method and device, storage medium and electronic equipment | |
| CN109766194B (en) | Method and system for realizing low-coupling plan task component based on message | |
| CN105786603B (en) | Distributed high-concurrency service processing system and method | |
| JPS61500751A (en) | Method for stopping program processes in multiprocessing systems | |
| WO2020238989A1 (en) | Method and apparatus for scheduling task processing entity | |
| JP7161560B2 (en) | Artificial intelligence development platform management method, device, medium | |
| CN110071855A (en) | Equipment linkage control method, device, system, gateway and storage medium | |
| CN110209654A (en) | A kind of text file data storage method, system and electronic equipment and storage medium | |
| CN111913793A (en) | Distributed task scheduling method, device, node equipment and system | |
| EP3438847A1 (en) | Method and device for duplicating database in distributed system | |
| US20060282524A1 (en) | Apparatus, system, and method for facilitating communication between an enterprise information system and a client | |
| CN113157411B (en) | Celery-based reliable configurable task system and device | |
| CN106843890A (en) | Sensor network, node and its operation method based on intelligent decision | |
| CN112448977A (en) | System, method, apparatus and computer readable medium for assigning tasks | |
| CN107908370A (en) | Date storage method and device | |
| CN113032139A (en) | Request processing method and device, computer readable storage medium and electronic equipment | |
| CN109842674A (en) | A kind of delay queue implementation method that multilingual interaction is cross-platform | |
| JP2012113558A (en) | Interruption control device, interruption control method and integrated circuit | |
| CN111400003A (en) | Task processing method and device | |
| CN111400060A (en) | Equipment linkage method, device, server and medium | |
| CN111767113A (en) | Method and device for realizing container eviction | |
| CN113296892B (en) | Dormancy method and device of development environment, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200131 |
|
| RJ01 | Rejection of invention patent application after publication |