CN109901918B - Method and device for processing overtime task - Google Patents
Method and device for processing overtime task Download PDFInfo
- Publication number
- CN109901918B CN109901918B CN201711292379.8A CN201711292379A CN109901918B CN 109901918 B CN109901918 B CN 109901918B CN 201711292379 A CN201711292379 A CN 201711292379A CN 109901918 B CN109901918 B CN 109901918B
- Authority
- CN
- China
- Prior art keywords
- task
- task data
- timeout
- data
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 130
- 238000000034 method Methods 0.000 title claims abstract description 48
- 230000015654 memory Effects 0.000 claims abstract description 96
- 230000002159 abnormal effect Effects 0.000 claims description 45
- 230000005856 abnormality Effects 0.000 claims description 37
- 238000004590 computer program Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 description 16
- 238000010586 diagram Methods 0.000 description 13
- 238000004891 communication Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000008901 benefit Effects 0.000 description 4
- 238000013500 data storage Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012384 transportation and delivery Methods 0.000 description 3
- 239000000835 fiber Substances 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 238000012550 audit Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Abstract
The invention discloses a method and a device for processing overtime tasks, and relates to the technical field of computers. One embodiment of the method comprises the following steps: storing the first task data in a database, and storing the second task data in the memory of each instance; wherein the information attribute of the second task data belongs to a part of the information attribute of the first task data; acquiring and processing a timeout task according to the second task data; and updating the first task data and the second task data corresponding to the overtime task. The implementation mode can solve the problems of low task processing efficiency and high resource expense.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and apparatus for processing a timeout task.
Background
In many real life scenarios, a task overtime situation is encountered, for example, a user places an order, but does not pay for a specified time, which requires a related service module to process the order, and the order is automatically cancelled.
In the process of implementing the present invention, the inventor finds that at least the following problems exist in the prior art: aiming at scattered global overtime data to be processed, a global unified scheduling center firstly scans whether all tasks to be processed meet overtime conditions or not, so that overtime tasks are collected, and the overtime tasks are redistributed and processed. Therefore, a single machine instance needs to process a large number of timeout tasks at the same time, and when the timeout tasks reach a certain level and the service system is numerous, some unnecessary resource overhead is increased. When the process of processing the overtime task is abnormal, the abnormal retry mechanism is thrown out to wait for the next retry, and unnecessary waste is brought to the resources of each service system.
Disclosure of Invention
In view of the above, the embodiments of the present invention provide a method and an apparatus for processing a task overtime, which can solve the problems of low task processing efficiency and large resource overhead.
To achieve the above object, according to one aspect of the embodiments of the present invention, there is provided a method for processing a timeout task, including:
storing the first task data in a database, and storing the second task data in the memory of each instance; wherein the information attribute of the second task data belongs to a part of the information attribute of the first task data;
acquiring and processing a timeout task according to the second task data;
updating the first task data and the second task data corresponding to the overtime task;
if an abnormality occurs in any step of storing second task data in the memory of each instance, acquiring and processing a timeout task according to the second task data and updating first task data corresponding to the timeout task, storing the abnormality information into the database;
obtaining the abnormal information in the database, selecting a strategy corresponding to the abnormal information according to a strategy library, and processing the abnormality according to the strategy.
Optionally, the first task data stored in the database includes second task data stored in each instance memory, and the second task data stored in each instance memory is different.
Optionally, storing the first task data in a database and the second task data in the memory of each instance includes:
receiving a task processing request;
generating first task data according to the task processing request, wherein the information attribute of the first task data at least comprises task timeout time and task state;
storing the first task data in a database;
generating second task data, wherein the information attribute of the second task data comprises task timeout time;
and storing the second task data in the memory of each instance.
Optionally, acquiring and processing a timeout task according to the second task data includes:
scanning the memory to acquire second task data stored in the memory;
comparing the current time with the task timeout time of the second task data, judging whether the current time exceeds the task timeout time, and if so, processing the timeout task corresponding to the task timeout time.
Optionally, updating the first task data and the second task data corresponding to the timeout task includes:
and updating the task state of the first task data corresponding to the overtime task according to the processing result of the overtime task, and deleting the second task data corresponding to the overtime task.
In addition, according to another aspect of the embodiment of the present invention, there is provided an apparatus for processing a timeout task, including:
the storage module is used for storing the first task data in the database and storing the second task data in the memory of each instance; wherein the information attribute of the second task data belongs to a part of the information attribute of the first task data;
the processing module is used for acquiring and processing a timeout task according to the second task data;
the updating module is used for updating the first task data and the second task data corresponding to the overtime task;
the exception storage module is used for storing exception information into the database if an exception occurs in any step of storing second task data into the memory of each instance, acquiring and processing a timeout task according to the second task data and updating first task data corresponding to the timeout task;
The exception handling module is used for acquiring the exception information in the database, selecting a strategy corresponding to the exception information according to a strategy library, and handling the exception according to the strategy.
Optionally, the first task data stored in the database includes second task data stored in each instance memory, and the second task data stored in each instance memory is different.
Optionally, the storage module is configured to:
receiving a task processing request;
generating first task data according to the task processing request, wherein the information attribute of the first task data at least comprises task timeout time and task state;
storing the first task data in a database;
generating second task data, wherein the information attribute of the second task data comprises task timeout time;
and storing the second task data in the memory of each instance.
Optionally, the processing module is configured to:
scanning the memory to acquire second task data stored in the memory;
comparing the current time with the task timeout time of the second task data, judging whether the current time exceeds the task timeout time, and if so, processing the timeout task corresponding to the task timeout time.
Optionally, the updating module is configured to:
and updating the task state of the first task data corresponding to the overtime task according to the processing result of the overtime task, and deleting the second task data corresponding to the overtime task.
According to another aspect of an embodiment of the present invention, there is also provided an electronic device including:
one or more processors;
storage means for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the methods of any of the embodiments described above.
According to another aspect of an embodiment of the present invention, there is also provided a computer readable medium having stored thereon a computer program which, when executed by a processor, implements the method according to any of the embodiments described above.
One embodiment of the above invention has the following advantages or benefits: because the technical means that the first task data is stored in the database, the second task data is stored in the memory of each instance, and the overtime task is acquired and processed according to the second task data is adopted, the technical problems of low task processing efficiency and high resource expenditure are solved, the original global task is decomposed into a plurality of subtasks to be respectively processed in each instance, and the performance pressure of processing the overtime task in a large batch at the same time by a single instance is greatly reduced. When the task processing or the system of each instance in the cluster is abnormal, the abnormal can be classified and screened from the global angle through the step of abnormal retry, and different strategies are selected for abnormal processing. And the strategy library can be enriched by means of manual input, so that the strategy library strategy can be dynamically classified and processed, and the processing efficiency is improved.
Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of the main flow of a method of handling timeout tasks according to an embodiment of the invention;
FIG. 2 is a schematic diagram of the main flow of a method of handling timeout tasks according to one referenceable embodiment of the invention;
FIG. 3 is a schematic diagram of the main flow of a method of handling timeout tasks according to another referenceable embodiment of the invention;
FIG. 4 is a schematic diagram of the major modules of an apparatus for handling timeout tasks according to an embodiment of the present invention;
FIG. 5 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;
fig. 6 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
FIG. 1 is a method of handling timeout tasks according to an embodiment of the invention. As an embodiment of the present invention, as shown in fig. 1, the method for processing a timeout task may include:
step 101, storing first task data in a database, and storing second task data in memories of all instances; wherein the information attribute of the second task data belongs to a part of the information attribute of the first task data.
In this step, the first task data is stored in a database, and the second task data is stored in the memory of each instance of the cluster, providing the basic data for the subsequent steps. And, the information attribute of the second task data belongs to a part of the information attribute of the first task data; the information attribute may include task timeout time and task state. As an embodiment of the present invention, the information attribute of the first task data includes at least a task timeout time and a task state, and the information attribute of the second task data includes the task timeout time, so as to simplify data storage in each instance memory, and provide basic data for subsequent steps.
As still another embodiment of the present invention, the information attribute of the first task data includes at least an order number, a task timeout time of the order, and a task status, and the information attribute of the second task data includes the order number and the task timeout time of the order. It should be noted that each order has a globally unique number, and each order may correspond to a plurality of tasks to be processed, such as payment, shipping, return, exchange, cancellation of orders, complaints, approval, and the like. Alternatively, the order may be a shopping order, a shipping order, an audit order, or the like.
It should be noted that, the database stores global order numbers, each task corresponding to each order, time-out time of each task, and task status of each task (for example, new order, to-be-paid, payment completed, payment timeout, to-be-delivered, delivery completed, delivery timeout, to-be-returned, delivery completed, approval timeout, etc.). Further, the information attribute of the first task data at least further includes task states of each task. Optionally, the information attribute of the first task data may further include classification coding when the task is abnormal; time stamp information for the task to be stored in the database may also be included.
Taking payment tasks of orders as an example, a database is used as a global resource, and an order data structure in the database is as follows:
optionally, the information attribute of the second task data does not include: task state, classification code when the task is abnormal, time stamp information of the task stored in the memory, and the like. In an embodiment of the present invention, the information attribute of the second task data may only include an order number and a timeout time of each task corresponding to each order. Alternatively, the information attribute of the second task data may consist of an order number and a task timeout time.
Taking payment task of order as an example, the order data structure in the cluster instance memory can adopt HashSet structure as follows:
for example, for 120 tasks (e.g., payment task and complaint task of 60 orders, shipment task of 120 orders, payment task of 30 orders, shipment task, order cancellation task) and 10 instances, 120 tasks may be equally distributed to 10 instances (10 tasks and timeout time of each task are stored in each instance memory). Different numbers of tasks can be allocated to each instance according to the processing capacity of each instance, so that resource allocation is optimized, and resource overhead is saved.
It is also possible that: tasks are assigned according to the task content (e.g., payment, shipping, return, cancellation of orders, complaints, etc.). Specifically, tasks having the same task content are assigned to the same instance, which processes these tasks, for example. For tasks with complex processing procedures or longer processing time, the tasks with the same task content can be distributed to a plurality of examples, so that resource distribution is optimized, and processing efficiency is improved. Multiple tasks may also be assigned to the same instance for processing, as the invention is not limited in this regard.
Optionally, each instance of the cluster adopts distributed deployment (such as ZooKeeper), and task data is guided to all instances in the cluster for parallel processing while the horizontal expansion capability is considered, so that the processing efficiency is improved, and the fault risk is reduced.
As one embodiment of the present invention, the first task data stored in the database includes second task data stored in each instance memory, and the second task data stored in each instance memory is different. That is, the order numbers stored in the respective instance memories may be the same, and the tasks may be the same, but the same order numbers and the same task data of the same order numbers are not generally stored in the respective instance memories.
Therefore, in the embodiment of the invention, the data storage in the local memory of each example can be simplified, and the data reading efficiency can be improved by adopting the data storage mode provided by the embodiment of the invention, so that the time expenditure is reduced.
As another embodiment of the present invention, the step 101 includes: the first task data is stored in a database, and the second task data is stored in a memory. In this embodiment, after the first task data is successfully stored in the database, the second task data is stored in the memory, so as to ensure the data validity of the database, and in the subsequent step, the task can be executed based on the first task data in the database.
As another embodiment of the present invention, step 101 may include: receiving a task processing request of a user; and generating first task data and second task data according to the task processing request.
It should be noted that the timeout time refers to a time when a task is timed out, that is, a time (for example, 2017, 11, 20, 01, 10 minutes, 20 seconds) until which the task belongs to a timeout task if the task is left unprocessed.
In particular, different timeout thresholds may be set for different task content. Taking a payment task as an example, the timeout threshold may be 24 hours or 0.5 hours, and when a task request of a user is received, the current time is added with the time threshold, that is, the timeout time of the payment task. Taking a shipping task as an example, the timeout threshold may be 2 days or 7 days, and when a task request of a user is received, the current time is added with the time threshold, which is the timeout time of the shipping task. The timeout period may thus be determined based on the time of the task request and a timeout threshold, which is calculated in step 101 in order to quickly determine if the task has timed out in a subsequent step.
As yet another embodiment of the present invention, step 101 may include: firstly, receiving a task processing request, then generating first task data according to the task processing request, and then storing the first task data in a database; and generating second task data, and finally storing the second task data in the memory of each instance. The information attribute of the first task data at least comprises task timeout time and task state, and the information attribute of the second task data comprises task timeout time.
In this embodiment, a task request of a user is received, first task data is generated according to the task request, after the first task data is successfully stored in a database, second task data is generated according to the task request, and then the second task data is stored in a memory, so that the data validity of the database is ensured. If the first task data fails to be stored in the database, ending. It should be noted that the second task data may be generated at the same time as the first task data is generated; however, after the first task data is successfully stored in the database, the second task data is stored in the memory of each instance.
And 102, acquiring and processing a timeout task according to the second task data.
In the step, firstly, a memory is scanned to acquire second task data, then, overtime tasks are screened out according to task overtime time, and then, the overtime tasks are processed.
As another embodiment of the present invention, step 102 includes: firstly, scanning the memory to acquire second task data stored in the memory; then, comparing the current time with the task timeout time of the second task data, and judging whether the current time exceeds the task timeout time; if yes, processing the overtime task corresponding to the task overtime time; if not, ending and waiting for the next scanning.
Optionally, the memory is scanned periodically to obtain second task data. Alternatively, the timeout task is processed in a single-threaded or multi-threaded manner.
It should be noted that, because the information attribute of the second task data includes the task timeout time, when the memory is scanned, it may be determined whether each task is timeout at the current time. If yes, the task is a timeout task; if not, the task is not a timeout task.
It should be noted that, while processing the task that is timed out, the task is processed in the normal time, and if the task is processed in the normal time, the task is deleted from the memory. It can be seen that the second task data stored in the memory is updated from time to time. In practice, the second task data is task data to be processed.
Alternatively, all the instances in the cluster may be configured to start in a spring+quantiz manner, where each instance obtains a timeout order from its own local memory. The embodiment of the invention decomposes the original global task into a plurality of subtasks to be respectively processed in each instance, thus greatly reducing the performance pressure of a single machine instance to process a large batch of overtime tasks at the same time.
It should be noted that, in the prior art, when each timeout task is processed, task basic data (such as task information, user information, commodity information, etc.) needs to be collected from each service system (such as order system, user system, commodity information system, etc.) on an instance, then these task basic data are associated and assembled to generate task data, and then the timeout task is processed by a new starting process. In the embodiment of the invention, the overtime task can be determined only by the overtime time of the task without association and assembly on the example, and then the overtime task is processed. Therefore, the embodiment of the invention fully utilizes the advantages of the distributed processing technology and the memory processing efficiency, and improves the processing efficiency of the overtime task.
And step 103, updating the first task data and the second task data corresponding to the overtime task.
Specifically, in this step, the task state of the first task data corresponding to the timeout task processed in step 102 is updated, and the second task data corresponding to the timeout task is deleted.
Optionally, after the processing of the overtime task is successful, updating the task state of the first task data; and deleting the second task data after the updating is successful so as to ensure the data validity of the database.
Taking payment task as an example, after the overtime task is successfully processed, modifying the task state orderState of the overtime task from 'to pay' to 'pay complete', and deleting the overtime task in the memory. Therefore, the number of timeout tasks stored in each instance memory can be kept within a certain order of magnitude, avoiding occupying a large amount of memory.
Fig. 2 is a schematic diagram of the main flow of a method of handling timeout tasks according to one referenceable embodiment of the invention. As yet another embodiment of the present invention, step 102 and step 103 may specifically include:
regularly scanning an instance memory;
acquiring second task data from the memory;
comparing the current time with the task timeout time of the second task data, and judging whether a timeout task exists or not;
if the overtime task does not exist, ending;
if the overtime task exists, adopting a single-thread/multi-thread data processing mode to process the overtime task;
after successful processing of the overtime task, the task state of the first task data corresponding to the overtime task is updated; and deleting the second task data corresponding to the overtime task after the updating is successful.
According to the various embodiments described above, it can be seen that the present invention solves the problems of low task processing efficiency and large resource overhead by adopting the technical means of storing the first task data in the database, storing the second task data in the memory of each instance, and acquiring and processing the timeout task according to the second task data. That is, the prior art is that a single machine instance needs to process a large number of timeout tasks at the same time, and when the timeout tasks reach a certain level and the service system is numerous, some unnecessary resource overhead is increased. According to the invention, the first task data is stored in the database, the second task data is stored in the memory of each instance, and the overtime task is acquired and processed according to the second task data, so that the original global task is decomposed into a plurality of subtasks to be respectively processed in each instance, and the performance pressure of processing a large amount of overtime tasks at the same time by a single instance is greatly reduced.
Furthermore, the overtime task can be determined through the overtime time of the task without the association and assembly on the instance, and then the processing of the overtime task is realized. Therefore, the embodiment of the invention fully utilizes the advantages of the distributed processing technology and the memory processing efficiency, and improves the processing efficiency of the overtime task.
As still another embodiment of the present invention, the method may further include:
step 104, if an abnormality occurs in any one of the steps of storing the second task data in the memory of each instance, acquiring and processing the overtime task according to the second task data and updating the first task data corresponding to the overtime task, storing the abnormality information into the database; the method comprises the steps of,
step 105, obtaining the abnormal information in the database, selecting a strategy corresponding to the abnormal information according to a strategy library, and processing the abnormality according to the strategy.
Specifically, in step 101, if an abnormality occurs when the second task data is stored in the memory of each instance, the abnormality information is stored in the database; in step 102, if an abnormality occurs in any one of the steps of scanning at regular time, acquiring second task data, determining a timeout task, and processing the timeout task, storing the abnormality information into a database; in step 103, if an abnormality occurs in updating the task state of the first task data corresponding to the timeout task, the abnormality information is stored in the database. And then, performing abnormal retry according to the abnormal information.
For convenience of description, the present invention classifies the anomalies occurring in the above steps 101, 102, 103 as "flow anomalies".
It should be noted that fig. 2 is merely an exemplary illustration of storing abnormality information in the database for performing an abnormal retry if an abnormality occurs at the time of "update task state" and the update fails. Similarly, when an abnormality occurs in the execution of other steps, the abnormality information is also stored in the database, and an abnormal retry is performed, which will not be described again.
In yet another embodiment of the present invention, the information attribute of the first task data may further include classification coding (error type) at the time of task processing exception. Optionally, after storing the anomaly information in the database, the first task data, such as an order handling anomaly time division code (error type), is updated to determine the corresponding strategy based on the order handling anomaly time division code.
Alternatively, the anomaly information is periodically acquired from the database. E.g. regularly retrieving an abnormal order for order timeout processing from the database (isfinish=0 and error type is not null).
Optionally, the method may further comprise triggering by an event, acquiring exception information in the database when the event is triggered, selecting a policy corresponding to the exception information according to a policy library, and then processing the exception according to the policy. For example, in an emergency situation, an abnormal order may go through a separate interface, using an event trigger mechanism, to a dedicated channel to handle the abnormal order.
It should be noted that, policies corresponding to various exceptions are stored in the policy repository, so that corresponding processing can be performed according to policies selected from the policy repository.
Optionally, the anomalies may also include system anomalies and/or data anomalies. Optionally, the system exception includes an instance hang-up, a network connection exception, an instance memory full, a database access exception, a cache read-write exception, and the like. Alternatively, the data anomalies may include information of the order being altered, resulting in anomalies. Likewise, for system anomalies and data anomalies, policies corresponding to those anomalies may also be selected from a policy library.
FIG. 3 is a schematic diagram of the main flow of a method of handling a timeout task according to another referenceable embodiment of the invention, which may include:
acquiring abnormal information from a database through timing triggering or event triggering, and judging whether the abnormal information exists or not;
if no abnormal information exists, ending;
if the abnormality information exists, determining the type of the abnormality, such as flow abnormality, system abnormality, data abnormality or other unknown abnormality, according to the abnormality information;
Selecting a strategy corresponding to the abnormal type according to a strategy library;
if the anomaly type is abnormal in flow, calling first task data in a database and processing a timeout task; if the processing is unsuccessful, the first task data in the database is called again, and the overtime task is processed until the processing is successful; then updating the task state of the overtime task in the first task data (namely updating the record of the abnormal information in the database);
in the process anomaly, whether to continue the retry or not can be determined according to the retry number threshold and/or the retry time threshold. If the retry time threshold and/or the retry time threshold are not reached, the call time is continuously increased, rescheduling is performed, and if the retry time threshold and/or the retry time threshold are exceeded, reminding is given to a responsible person. Specifically, a retry number threshold and a retry time threshold may be preset, and if the retry number threshold is exceeded, no retry is performed, and a reminder is sent to notify a responsible person, so that a timeout task processing may be performed manually or a next trigger may be waited. It should be noted that the following may be further aimed at: and respectively making different strategies according to any abnormal occurrence in the steps of acquiring and processing the overtime task and updating the first task data according to the second task data, and storing the strategies in a strategy library.
If the abnormality type is system abnormality, the system is not retried, and a reminder is sent to inform a responsible person, so that the abnormality can be handled manually. After the manual processing, a record of the anomaly information in the database is updated. And classification statistics can be further carried out according to the detailed information of system abnormality. After a period of time, the system anomalies and the data anomalies and the flow anomalies are formed into a statistical table together, and the statistical table is mainly used for analyzing the ratio of various anomalies when the overtime task is pulled back.
If the anomaly type is data anomalies, discarding, and not retrying, and updating the state value of the order in the database, so that the order with the data anomalies is not used as anomalies when the next round of triggering occurs.
If the anomaly type is other unknown anomalies, the retry is not performed, and a reminder is sent out to inform a responsible person, so that the anomaly processing can be performed manually. Furthermore, the corresponding strategy can be added into the strategy library according to the manual processing result, and the strategy library is updated for the next processing of the same abnormality.
It should be noted that if an anomaly occurs in updating the database record, resulting in a failure, the retry is not performed again, and the next round of triggering is awaited, so that an abnormal retry is performed in the next round.
Optionally, for an abnormal order triggered by an event, whether to continue retry or not can be determined according to a retry number threshold and/or a retry time threshold, and a prompt can be sent to a responsible person.
Therefore, when the task processing or the system of each instance in the cluster is abnormal, the abnormal can be classified and screened from the global angle through the abnormal retry step, and different strategies are selected for abnormal processing. And the strategy library can be enriched by means of manual input, so that the strategy library strategy can be dynamically classified and processed, and the processing efficiency is improved.
In addition, in another embodiment of the present invention, reference may be made to the implementation of the method for processing a timeout task described in the embodiment, and the method for processing a timeout task described in the foregoing is already described in detail, so that the description will not be repeated here.
Fig. 4 is an apparatus for processing a timeout task according to an embodiment of the present invention, and as shown in fig. 4, the apparatus 400 for processing a timeout task includes a storage module 401, a processing module 402, an updating module 403, an exception storage module 404, and an exception processing module 405. The storage module 401 stores the first task data in a database and the second task data in the memory of each instance; wherein the information attribute of the second task data belongs to a part of the information attribute of the first task data; the processing module 402 acquires and processes a timeout task according to the second task data; the updating module 403 updates the first task data and the second task data corresponding to the timeout task. If an abnormality occurs in any step of storing second task data in the memory of each instance, acquiring and processing a timeout task according to the second task data, and updating the first task data corresponding to the timeout task, the abnormality storage module 404 stores abnormality information in the database; the exception handling module 405 obtains the exception information in the database, selects a policy corresponding to the exception information according to a policy repository, and handles the exception according to the policy.
The storage module 401 stores the first task data in a database, and stores the second task data in the memory of each instance of the cluster, providing basic data for the operation of the subsequent module. And, the information attribute of the second task data belongs to a part of the information attribute of the first task data; the information attribute may include task timeout time and task state. As an embodiment of the present invention, the information attribute of the first task data includes at least a task timeout time and a task state, and the information attribute of the second task data includes the task timeout time, so as to simplify data storage in each instance memory, and provide basic data for subsequent steps.
Optionally, the information attribute of the first task data may further include classification coding when the task is abnormal; time stamp information for the task to be stored in the database may also be included. Optionally, the information attribute of the second task data does not include: task state, classification code when the task is abnormal, time stamp information of the task stored in the memory, and the like.
In some embodiments of the present invention, the information attribute of the first task data includes at least an order number, a task timeout time of the order, and a task status, and the information attribute of the second task data includes the order number and the task timeout time of the order. Optionally, the information attribute of the second task data may include only an order number and a timeout time of each task corresponding to each order. Optionally, the information attribute of the second task data is composed of an order number and a task timeout time.
Optionally, the first task data stored in the database includes second task data stored in each instance memory, and the second task data stored in each instance memory is different.
Optionally, the storage module 401 stores the first task data in the database and then stores the second task data in the memory. After the first task data is successfully stored in the database, the second task data is stored in the memory so as to ensure the data validity of the database, and other modules can execute tasks based on the first task data in the database.
Optionally, the storage module 401 first receives a task processing request, then generates first task data according to the task processing request, and then stores the first task data in a database; and generating second task data, and finally storing the second task data in the memory of each instance.
If the first task data fails to be stored in the database, ending. It should be noted that the second task data may be generated at the same time as the first task data is generated; however, after the first task data is successfully stored in the database, the second task data is stored in the memory of each instance.
Optionally, the processing module 402 scans the memory to obtain second task data stored in the memory; then, comparing the current time with the task timeout time of the second task data, and judging whether the current time exceeds the task timeout time; if yes, processing the overtime task corresponding to the task overtime time; if not, ending and waiting for the next scanning.
Optionally, the memory is scanned periodically to obtain second task data. Alternatively, the timeout task is processed in a single-threaded or multi-threaded manner.
Optionally, the updating module 403 updates the task state of the first task data corresponding to the timeout task, and deletes the second task data corresponding to the timeout task.
Optionally, after the overtime task is successfully processed, updating the task state of the first task data corresponding to the overtime task; and deleting the second task data corresponding to the overtime task after the updating is successful so as to ensure the data validity of the database.
If the second task data is abnormal when being stored in the memory of each instance, the abnormal information is stored in a database; if any process of regular scanning, second task data acquisition, overtime task determination, overtime task processing and task updating state is abnormal, the abnormal information is stored in a database; if the task state of the first task data corresponding to the overtime task is updated, the abnormal information is stored into the data. And then, performing abnormal retry according to the abnormal information.
Optionally, the anomalies may also include system anomalies and/or data anomalies. Optionally, the system exception includes an instance hang-up, a network connection exception, an instance full memory, and the like. Alternatively, the data anomalies may include information of the order being altered, resulting in anomalies. Likewise, for system anomalies and data anomalies, policies corresponding to those anomalies may also be selected from a policy library.
According to the various embodiments described above, it can be seen that the present invention solves the problems of low task processing efficiency and large resource overhead by adopting the technical means of storing the first task data in the database, storing the second task data in the memory of each instance, and acquiring and processing the timeout task according to the second task data. That is, the prior art is that a single machine instance needs to process a large number of timeout tasks at the same time, and when the timeout tasks reach a certain level and the service system is numerous, some unnecessary resource overhead is increased. According to the invention, the first task data is stored in the database, the second task data is stored in the memory of each instance, and the overtime task is acquired and processed according to the second task data, so that the original global task is decomposed into a plurality of subtasks to be respectively processed in each instance, and the performance pressure of processing a large amount of overtime tasks at the same time by a single instance is greatly reduced.
The specific implementation of the device for processing the timeout task according to the present invention is described in detail in the method for processing the timeout task described above, and thus the description thereof will not be repeated here.
Fig. 5 illustrates an exemplary system architecture 500 to which a method of processing a timeout task or an apparatus for processing a timeout task of an embodiment of the present invention may be applied.
As shown in fig. 5, the system architecture 500 may include terminal devices 501, 502, 503, a network 404, and a server 405. The network 504 is used as a medium to provide communication links between the terminal devices 501, 502, 503 and the server 505. The network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with the server 505 via the network 504 using the terminal devices 501, 502, 503 to receive or send messages or the like. Various communication client applications may be installed on the terminal devices 501, 502, 503, such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).
The terminal devices 501, 502, 503 may be a variety of electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 505 may be a server providing various services, such as a background management server (by way of example only) providing support for shopping-type websites browsed by users using the terminal devices 501, 502, 503. The background management server may analyze and process the received data such as the product information query request, and feedback the processing result (e.g., the target push information, the product information—only an example) to the terminal device.
It should be noted that, the method for processing the timeout task according to the embodiment of the present invention is generally executed by the server 505, and accordingly, the device for processing the timeout task is generally disposed in the server 505.
It should be understood that the number of terminal devices, networks and servers in fig. 5 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 6, there is illustrated a schematic diagram of a computer system 600 suitable for use in implementing an embodiment of the present invention. The terminal device shown in fig. 6 is only an example, and should not impose any limitation on the functions and the scope of use of the embodiment of the present invention.
As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, ROM 602, and RAM603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, mouse, etc.; an output portion 607 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The drive 610 is also connected to the I/O interface 605 as needed. Removable media 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on drive 610 so that a computer program read therefrom is installed as needed into storage section 608.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication portion 609, and/or installed from the removable medium 611. The above-described functions defined in the system of the present invention are performed when the computer program is executed by a Central Processing Unit (CPU) 601.
The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules involved in the embodiments of the present invention may be implemented in software or in hardware. The described modules may also be provided in a processor, for example, as: a processor includes a memory module, a processing module, an updating module, an exception memory module, and an exception processing module, wherein the names of these modules do not constitute a limitation on the module itself in some cases.
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to include: storing the first task data in a database, and storing the second task data in the memory of each instance; wherein the information attribute of the second task data belongs to a part of the information attribute of the first task data; acquiring and processing a timeout task according to the second task data; updating the first task data and the second task data corresponding to the overtime task; if an abnormality occurs in any step of storing second task data in the memory of each instance, acquiring and processing a timeout task according to the second task data and updating first task data corresponding to the timeout task, storing the abnormality information into the database; obtaining the abnormal information in the database, selecting a strategy corresponding to the abnormal information according to a strategy library, and processing the abnormality according to the strategy.
According to the technical scheme of the embodiment of the invention, the technical means that the first task data is stored in the database, the second task data is stored in the memory of each instance, and the overtime task is acquired and processed according to the second task data is adopted, so that the technical problems of low task processing efficiency and high resource expenditure are solved, the original global task is decomposed into a plurality of subtasks to be respectively processed in each instance, and the performance pressure of processing the overtime tasks in a large quantity at the same time by a single instance is greatly reduced.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (8)
1. A method of processing a timeout task, comprising:
storing the first task data in a database, and storing the second task data in the memory of each instance; wherein the information attribute of the second task data belongs to a part of the information attribute of the first task data;
Acquiring and processing a timeout task according to the second task data;
updating the first task data and the second task data corresponding to the overtime task;
if an abnormality occurs in any step of storing second task data in the memory of each instance, acquiring and processing a timeout task according to the second task data and updating first task data corresponding to the timeout task, storing the abnormality information into the database;
acquiring abnormal information in the database, selecting a strategy corresponding to the abnormal information according to a strategy library, and processing the abnormality according to the strategy;
storing the first task data in a database and the second task data in the memory of each instance, comprising:
receiving a task processing request;
generating first task data according to the task processing request, wherein the information attribute of the first task data at least comprises task timeout time and task state;
storing the first task data in a database;
generating second task data, wherein the information attribute of the second task data comprises task timeout time;
storing the second task data in the memory of each instance;
Acquiring and processing a timeout task according to the second task data, including:
scanning the memory to acquire second task data stored in the memory;
comparing the current time with the task timeout time of the second task data, judging whether the current time exceeds the task timeout time, and if so, processing the timeout task corresponding to the task timeout time.
2. The method of claim 1, wherein the first task data stored in the database includes second task data stored in each instance memory, and wherein the second task data stored in each instance memory is different.
3. The method of claim 1, wherein updating the first task data and the second task data corresponding to the timeout task comprises:
and updating the task state of the first task data corresponding to the overtime task according to the processing result of the overtime task, and deleting the second task data corresponding to the overtime task.
4. An apparatus for processing a timeout task, comprising:
the storage module is used for storing the first task data in the database and storing the second task data in the memory of each instance; wherein the information attribute of the second task data belongs to a part of the information attribute of the first task data;
The processing module is used for acquiring and processing a timeout task according to the second task data;
the updating module is used for updating the first task data and the second task data corresponding to the overtime task;
the exception storage module is used for storing exception information into the database if an exception occurs in any step of storing second task data into the memory of each instance, acquiring and processing a timeout task according to the second task data and updating first task data corresponding to the timeout task;
the exception handling module is used for acquiring exception information in the database, selecting a strategy corresponding to the exception information according to a strategy library, and handling the exception according to the strategy;
the storage module is used for:
receiving a task processing request;
generating first task data according to the task processing request, wherein the information attribute of the first task data at least comprises task timeout time and task state;
storing the first task data in a database;
generating second task data, wherein the information attribute of the second task data comprises task timeout time;
storing the second task data in the memory of each instance;
The processing module is used for:
scanning the memory to acquire second task data stored in the memory;
comparing the current time with the task timeout time of the second task data, judging whether the current time exceeds the task timeout time, and if so, processing the timeout task corresponding to the task timeout time.
5. The apparatus of claim 4, wherein the first task data stored in the database comprises second task data stored in each instance memory, and wherein the second task data stored in each instance memory is different.
6. The apparatus of claim 4, wherein the update module is configured to:
and updating the task state of the first task data corresponding to the overtime task according to the processing result of the overtime task, and deleting the second task data corresponding to the overtime task.
7. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs,
when executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-3.
8. A computer readable medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711292379.8A CN109901918B (en) | 2017-12-08 | 2017-12-08 | Method and device for processing overtime task |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711292379.8A CN109901918B (en) | 2017-12-08 | 2017-12-08 | Method and device for processing overtime task |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109901918A CN109901918A (en) | 2019-06-18 |
| CN109901918B true CN109901918B (en) | 2024-04-05 |
Family
ID=66940149
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711292379.8A Active CN109901918B (en) | 2017-12-08 | 2017-12-08 | Method and device for processing overtime task |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109901918B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111641678A (en) * | 2020-04-29 | 2020-09-08 | 深圳壹账通智能科技有限公司 | Task scheduling method and device, electronic equipment and medium |
| CN112035164A (en) * | 2020-09-03 | 2020-12-04 | 中国银行股份有限公司 | Method for updating node configuration of distributed system, application node and control node |
| CN112163810A (en) * | 2020-09-28 | 2021-01-01 | 杭州拼便宜网络科技有限公司 | Method and device for sending mail |
| CN112732460A (en) * | 2021-01-05 | 2021-04-30 | 卓望数码技术（深圳）有限公司 | Data state management method, system and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103581225A (en) * | 2012-07-25 | 2014-02-12 | 中国银联股份有限公司 | Distributed system node processing task method |
| CN104182283A (en) * | 2014-08-22 | 2014-12-03 | 北京京东尚科信息技术有限公司 | Task synchronization method |
| CN104407917A (en) * | 2014-11-13 | 2015-03-11 | 中国建设银行股份有限公司 | JMS (Java messaging service)-based transaction processing method and JMS-based transaction processing device |
| CN106293971A (en) * | 2016-08-15 | 2017-01-04 | 张家林 | A kind of method and apparatus of distributed task dispatching |
| CN106789431A (en) * | 2016-12-26 | 2017-05-31 | 中国银联股份有限公司 | A kind of overtime monitoring method and device |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150293953A1 (en) * | 2014-04-11 | 2015-10-15 | Chevron U.S.A. Inc. | Robust, low-overhead, application task management method |
| US20170236188A1 (en) * | 2014-12-30 | 2017-08-17 | NetSuite Inc. | System and method for automating business processes throughout the life cycle of an order by using a publish-subscriber pattern |
-
2017
- 2017-12-08 CN CN201711292379.8A patent/CN109901918B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103581225A (en) * | 2012-07-25 | 2014-02-12 | 中国银联股份有限公司 | Distributed system node processing task method |
| CN104182283A (en) * | 2014-08-22 | 2014-12-03 | 北京京东尚科信息技术有限公司 | Task synchronization method |
| CN104407917A (en) * | 2014-11-13 | 2015-03-11 | 中国建设银行股份有限公司 | JMS (Java messaging service)-based transaction processing method and JMS-based transaction processing device |
| CN106293971A (en) * | 2016-08-15 | 2017-01-04 | 张家林 | A kind of method and apparatus of distributed task dispatching |
| CN106789431A (en) * | 2016-12-26 | 2017-05-31 | 中国银联股份有限公司 | A kind of overtime monitoring method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109901918A (en) | 2019-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10320623B2 (en) | Techniques for tracking resource usage statistics per transaction across multiple layers of protocols | |
| CN109901918B (en) | Method and device for processing overtime task | |
| CN107729139B (en) | Method and device for concurrently acquiring resources | |
| US8818940B2 (en) | Systems and methods for performing record actions in a multi-tenant database and application system | |
| US10133797B1 (en) | Distributed heterogeneous system for data warehouse management | |
| CN109214741B (en) | Early warning method and early warning system for monitoring indexes | |
| US20190138375A1 (en) | Optimization of message oriented middleware monitoring in heterogenenous computing environments | |
| CN112100034A (en) | Service monitoring method and device | |
| CN111767127A (en) | Service data processing method and device | |
| CN113190558A (en) | Data processing method and system | |
| US11243979B1 (en) | Asynchronous propagation of database events | |
| CN113127225A (en) | Method, device and system for scheduling data processing tasks | |
| CN116701020A (en) | Message delay processing method, device, equipment, medium and program product | |
| CN113535439B (en) | Service request processing method, device, equipment and storage medium | |
| CN113138772B (en) | Construction method and device of data processing platform, electronic equipment and storage medium | |
| CN114827157A (en) | Cluster task processing method, device and system, electronic equipment and readable medium | |
| CN112463616A (en) | Chaos testing method and device for Kubernetes container platform | |
| CN116450622B (en) | Method, apparatus, device and computer readable medium for data warehouse entry | |
| CN111179097B (en) | Method, device, electronic equipment and storage medium for modifying warranty | |
| CN115981910B (en) | Method, apparatus, electronic device and computer readable medium for processing exception request | |
| CN112148705A (en) | Data migration method and device | |
| CN111008202A (en) | Distributed transaction processing method and framework | |
| CN109933459A (en) | A kind of execution method and apparatus of multitask | |
| CN112269672B (en) | File downloading exception handling method and device and electronic equipment | |
| CN116167835A (en) | Service processing method, device, electronic equipment and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |