CN110109986B - Task processing method, system, server and task scheduling system - Google Patents
Task processing method, system, server and task scheduling system Download PDFInfo
- Publication number
- CN110109986B CN110109986B CN201810041018.4A CN201810041018A CN110109986B CN 110109986 B CN110109986 B CN 110109986B CN 201810041018 A CN201810041018 A CN 201810041018A CN 110109986 B CN110109986 B CN 110109986B
- Authority
- CN
- China
- Prior art keywords
- task
- service
- target
- information
- target task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The embodiment of the specification discloses a task processing method, a system, a server and a task scheduling system, wherein the method comprises the following steps: when a business system creates an object, receiving a task which is created by the business system and bound with the object; receiving a query instruction sent by a task scheduling system, and acquiring a target task matched with the query instruction; and calling back the service system corresponding to the target task, so that the service system corresponding to the target task executes the target task, and receiving an execution result fed back by the service system corresponding to the target task. The technical scheme provided by the description can lighten the burden of a service system, thereby improving the processing efficiency of tasks.
Description
Technical Field
The present disclosure relates to the field of internet technologies, and in particular, to a task processing method, a task processing system, a task scheduling server, and a task scheduling system.
Background
In the current business system, after an object needing to be triggered at fixed time is created, a task for monitoring the timeliness of the object is generally created because the object triggered at fixed time has timeliness. For example, a ticketing system in an online travel platform may set up a payment for a user within half an hour after creating a Beijing air ticket order for the user to fly to the sea, otherwise the order may fail. At this point, the ticketing system may create a half-hour post-execution task for the order that, when executed, would fail the ticket order.
In the prior art, after creating tasks associated with objects, a business system typically accesses a database storing tasks continuously according to a fixed period, and then can determine which tasks can be triggered according to the access time. Therefore, the service system processes normal service and simultaneously needs to initiate polling to the database according to a fixed period, which brings great burden to the service system. When there are fewer idle resources in the service system, the efficiency of service processing and task processing is lower.
Disclosure of Invention
The embodiment of the specification aims to provide a task processing method, a task processing system, a task scheduling system and a server, which can reduce the burden of a service system and improve the task processing efficiency.
To achieve the above object, an embodiment of the present specification provides a task processing method, including: when a business system creates an object, receiving a task which is created by the business system and bound with the object; receiving a query instruction sent by a task scheduling system, and acquiring a target task matched with the query instruction from the received task; and calling back the service system corresponding to the target task, so that the service system corresponding to the target task executes the target task, and the task scheduling system receives an execution result fed back by the service system corresponding to the target task.
The embodiment of the specification also provides a server, which comprises a management module, an interface module and a scheduling module, wherein: the management module is used for providing a specified interface for the service system in the interface module, so that when the service system creates an object, the specified interface is called to create a task bound with the object, and the task created by the service system is stored in a database; the interface module is used for receiving the task which is created by the service system and is bound with the object; the scheduling module is used for receiving a query instruction sent by a task scheduling system and acquiring a target task matched with the query instruction from the database; and calling back the service system corresponding to the target task, so that the service system corresponding to the target task executes the target task, and the task scheduling system receives an execution result fed back by the service system corresponding to the target task.
The embodiment of the specification also provides a task processing method, which comprises the following steps: sending a data table splitting instruction to a task management system so that the task management system splits a data table in a database into a plurality of table units; receiving unit information of the table unit fed back by the task management system, wherein the unit information is used for representing a database and a data table to which data in the table unit belong; providing the unit information of the table unit to a node server, and sending a query instruction to the node server, so that the node server obtains a target task matched with the query instruction from the table unit, enables the node server to call back a service system corresponding to the target task, and receives an execution result for the target task fed back by the service system corresponding to the target task.
The embodiment of the specification also provides a task scheduling system, which comprises a network communication port, a processor and a memory, wherein: the network communication port is used for carrying out network data interaction; the memory is used for storing the unit information of the table unit fed back by the task management system; the processor is used for controlling the network communication port to send a data table splitting instruction to the task management system so that the task management system splits a data table in a database into a plurality of table units; receiving unit information of the table unit fed back by the task management system through the network communication port, wherein the unit information is used for representing a database and a data table to which data in the table unit belong; providing the unit information of the table unit to a node server, and sending a query instruction to the node server, so that the node server obtains a target task matched with the query instruction from the table unit, enables the node server to call back a service system corresponding to the target task, and receives an execution result for the target task fed back by the service system corresponding to the target task.
The embodiment of the specification also provides a task processing system, which comprises a service subsystem, a task scheduling subsystem and a task management subsystem, wherein: the business subsystem is used for creating a task bound with the object when the object is created, and sending the task to the task management subsystem; executing the target task to be processed when being called back by the task management subsystem; the task scheduling subsystem is used for sending a query instruction to the task management subsystem and receiving an execution result of the target task fed back by the service subsystem; the task management subsystem is used for receiving the task sent by the service subsystem and the query instruction sent by the task scheduling subsystem, and acquiring a target task matched with the query instruction from the received task; and calling back the service subsystem corresponding to the target task, so that the service subsystem corresponding to the target task executes the target task, and the task scheduling subsystem receives an execution result fed back by the service subsystem corresponding to the target task.
As can be seen from the technical solutions provided in the embodiments of the present specification, in the present specification, when a service system creates an object, a task bound to the object may be created in a task management system. After the business system has created a task, it is not necessary to query the task. The process of querying the task may be accomplished by a task scheduling system. The task scheduling system may send a query instruction to the task management system to query whether the created task needs to be executed. After receiving the query instruction, the task management system can acquire the target task which should be executed currently. The task management system may create a business system for the target task via a callback to cause the business system to execute the target task. After the execution result is obtained, the service system can feed back the execution result to the task scheduling system. Therefore, after the task is created, the service system only needs to wait for a callback instruction of the task management system, and the created task is not required to be inquired, so that the burden of the service system is reduced, and further, the service and the task can be executed efficiently.
Drawings
In order to more clearly illustrate the embodiments of the present description or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present description, and that other drawings may be obtained according to these drawings without inventive effort to a person skilled in the art.
FIG. 1 is a schematic diagram of a system structure in an embodiment of the present disclosure;
FIG. 2 is a flow chart of a task processing method in an embodiment of the present disclosure;
FIG. 3 is a schematic diagram illustrating the operation of the instruction in the embodiment of the present specification;
fig. 4 is a schematic diagram of a server structure in an embodiment of the present disclosure;
FIG. 5 is a flow chart of a task processing method in an embodiment of the present disclosure;
fig. 6 is a schematic diagram illustrating an internal configuration of a task scheduling system according to an embodiment of the present disclosure.
Detailed Description
In order to make the technical solution in the present specification better understood by those skilled in the art, the technical solution in the present specification embodiment will be clearly and completely described with reference to the drawings in the present specification embodiment, and it is apparent that the described embodiment is only a part of the embodiment of the present specification, not all the embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are intended to be within the scope of the present disclosure.
The specification provides a task processing method which can be applied to a task management system. The task management system may be deployed on a server, which may be a computer or workstation with computing capabilities. The server may be an independent server or may be a server cluster formed by a plurality of servers. Referring to fig. 1, in this embodiment of the present disclosure, the task management system may be connected to a business system, a task scheduling system, and a database. The business system can be a ticketing system, a hotel reservation system, a travel reservation system and the like, and can be used for creating objects needing to be triggered at fixed time. The task scheduling system may be a system responsible for task querying and task scheduling. The database can be provided with a plurality of data tables, wherein the data tables can store tasks created by a service system.
Referring to fig. 2, a task processing method provided in one embodiment of the present disclosure may include the following steps.
S11: and when the business system creates the object, receiving the task which is created by the business system and is bound with the object.
In this embodiment, the service system may create a corresponding object in response to an operation instruction of the user. The object may be a task that requires timing triggers. For example, the object may be a ticket order, an air ticket order, a trip order, a hotel room order, or the like. Typically, the object will have an effective time limit within which the object can be validated if the user completes the payment; if the user does not complete payment within the validity period, the object is revoked. Based on this, the business system, when creating an object, can create a task that is bound to the object. The task may be a task for monitoring the timeliness of the object. When the validity period expires and the user has not completed the payment operation, the task may be performed and the result of the execution may be to revoke the bound object.
In this embodiment, when the task management system establishes a connection with the service system, a specific interface may be provided in advance for the service system. The designated interface may be invoked when the business system creates an object, such that the business system may invoke the designated interface to create a task in the task management system that is bound to the object.
In this embodiment, the specified interface corresponding to the service system may be used as callback information of the service system, and the callback information may be stored in the task management system. In this way, the task management system can query the corresponding designated interface of the service system through locally stored callback information, so that the corresponding service system is called back through the queried designated interface, and the service system can process the corresponding task. In an actual application scenario, the service system and the corresponding designated interface may be stored in a key-value pair (key-value) manner. The identifier of the service system may be a key (key), and the number of the designated interface corresponding to the service system may be a value (value). In this way, the key value pair formed by the identification of the service system and the number of the appointed interface can be used as callback information of the service system.
In this embodiment, after receiving a task created by the service system, the task management system may store the task in a data table of the database. In practical application, the number of the database and the data table is multiple because the number of tasks created by the service system is huge. For example, there may be 4 databases storing tasks, and 256 data tables may be stored in total in the 4 databases, where each database may store 64 data tables. In order to enable the tasks to be uniformly written into the data tables, in one embodiment of the present disclosure, after receiving the tasks created by the service system, the task management system may determine the feature values of the tasks, and write the tasks into the data tables corresponding to the feature values. The characteristic value can be used as a basis for balanced storage. Specifically, in determining the feature value of the task, each character in the identification of the task may be traversed, and then the binary value of the character in the identification of the task may be determined. For example, the task is identified as qaz123, then one binary value can be obtained for each character, and thus 6 binary values can be obtained. In this embodiment, the determined binary values may be superimposed, so as to obtain a superimposed value. To enable the balanced storage of tasks in the various data tables, modulo arithmetic may be used to determine which data table the current task should write to. Specifically, the number of data tables may be obtained, and the superimposed value and the number of data tables may be subjected to modulo operation, and the obtained modulo value may be used as the characteristic value of the task. For example, the number of the data tables is 256, and the result obtained by performing modulo operation on the superimposed values and 256 is 15, so that 15 can be used as the characteristic value of the task. In practical applications, the data tables may be numbered sequentially. For example, 256 data tables may be numbered sequentially from 0 to 255, and thus each data table may be provided with a unique number. Because the range of the modulus value after the modulus operation is also from 0 to 255, the modulus value and the number of the data table can be in one-to-one correspondence, so that the data table corresponding to the task can be obtained. Specifically, a target data table with a number consistent with the modulus value in the data table can be determined, and the task is written into the target data table. For example, the calculated modulo value is 15, and the task can be written into the data table numbered 15. Therefore, the balanced storage of the tasks can be realized, and the resources of the data table can be utilized in a balanced mode.
In one embodiment, when a service system accesses the task management system or, the task management system may set thread pool information for the service system, in addition to assigning a specific interface to the task management system and storing callback information of the service system. The thread pool information may characterize threads of the business system executing tasks. For example, the thread pool information may include the number of threads. In this embodiment, the callback information and the thread pool information may be used as cache information of the service system. The cache information may be stored in a task management system. The cache information may be stored in association with an identifier of the service system, where the identifier of the service system may indicate a service type of the service system. Thus, based on the service type, the corresponding cache information can be queried.
In this embodiment, the task management system may update the cache information of the service system periodically. Specifically, the task management system can detect the configuration information of the designated interface and the thread related to the cache information according to a fixed period, and if the configuration information of the designated interface or the thread is changed, the task management system can update the cache information in time.
S13: and receiving a query instruction sent by a task scheduling system, and acquiring a target task matched with the query instruction.
In this embodiment, the task scheduling system may send a query instruction to the task management system according to a specified time period. When the task management system receives the inquiry command, the task acquisition process can be triggered. Specifically, the query instruction may include a specific service type, where the specific service type is used to indicate a service system corresponding to a task that needs to be processed. For example, the query is issued for a ticketing system, and then the specified service type contained in the query may be a ticketing type. Further, the tasks created by the business system may include specified fields therein. The specified field may be used to characterize the type of traffic to which the task belongs. For example, the specified field may be a biztype field. The content contained in the biztype field is also different according to different traffic types. For example, the biztype of the object task may be ttp_order, and the biztype of the travel task may be journ ey.
In this embodiment, the task management system may further set a specified number of acquisition tasks in advance. The specified number may define a number of tasks that the task management system obtains at most once. For example, the specified number is 10, and then the task management system may acquire a maximum of 10 tasks each time a task is acquired from the database. In this way, when the target tasks matched with the query instruction are acquired, the task types and the number can be limited, the tasks with the appointed number of service types consistent with the appointed service types are acquired, and the acquired appointed number of tasks are used as the target tasks matched with the query instruction, so that the appointed number can be adjusted according to the task quantity of the service system, and the scheduling capability of the task system can be dynamically adjusted.
In another embodiment, since tasks created by the business system are generally time-efficient, time nodes used to characterize the tasks are also included in the tasks. The time node may be automatically generated when a task is created. For example, an object is generated at 17 hours and 58 minutes and 30 seconds, the effective time of the object is half an hour, and then the time node of the task bound to the object may be 18 hours and 28 minutes and 30 seconds. When the task scheduling system sends a query instruction, a current time node may be included in the query instruction. Thus, the task management system, upon receiving the query instruction, can determine which tasks need to be performed. Specifically, the task management system may take a task that belongs to the specified service type and is earlier than the current time node as a target task that matches the query instruction, and acquire the target task. For example, the query instruction includes an object type, the current time node is 18 hours and 28 minutes and 50 seconds, then when the task management system acquires the target task, the service type of a certain task is the object type, and the time node is 18 hours and 28 minutes and 30 seconds, then the time node is earlier than the current time node, the task should be executed, and then the task management system can acquire the task.
S15: and calling back the service system corresponding to the target task, so that the service system corresponding to the target task executes the target task, and the service scheduling system receives an execution result fed back by the service system corresponding to the target task.
In this embodiment, the service system corresponding to the target task may refer to a service system that creates the target task. When the target task is acquired, the task management system can call back the business system for creating the target task, so that the business system corresponding to the target task executes the target task. Specifically, the task management system may store cache information of each service system. The cache information may be stored in association with a service type of the service system. Thus, after the task management system acquires the target task, callback information and thread pool information corresponding to the service type can be acquired from the cache according to the service type of the target task. For example, the task management system may read the biztype field of the target task, and then determine, from the content contained in the biztype field, the service type to which the target task belongs. The associated callback information and thread pool information are then read from the local cache based on the traffic type. In this way, the task management system can call back the service system through the designated interface characterized by the callback information, and the service system can execute the target task through the thread characterized by the thread pool information. After the business system finishes executing the target task, the execution result can be fed back to the task scheduling system. In this way, the task scheduling system may modify the state of the target task according to the execution result. For example, when a target task is executed normally, the task scheduling system may modify the state of the target task to be executed, and then when a task that needs to be executed is acquired, the target task is not acquired any more. In this embodiment, when the execution result indicates that the execution fails, the task scheduling system may recall the service system through the task management system, so that the service system executes the target task again.
In one embodiment of the present disclosure, the instructions sent by the task scheduling system may have different forms. The specification may include a data table splitting instruction in addition to the above-described query instruction. The data table splitting instruction may cause the task management system to split a data table in a database. The splitting aim is to divide a plurality of tasks into different table units and distribute the split table units to different node servers, so that the execution efficiency of the tasks is improved by a method of parallel processing of a plurality of node servers. Specifically, referring to fig. 3, when the task management system receives an instruction sent by the task scheduling system, the type of the instruction may be determined. When the instruction is a data table splitting instruction, a data table in a database may be split into a plurality of table units in response to the data table splitting instruction. For example, 256 data tables are currently provided, each 16 data tables can be split into one table unit, and finally 16 table units can be obtained by splitting, wherein each table unit comprises 16 data tables. After splitting to obtain a plurality of table units, the task management system may generate unit information of the table units according to data included in each table unit. The unit information may be data having a fixed format. For example, the element information may be a json string. In the unit information, it can be noted in which data table of which database the data contained in the table unit is located. In this way, the cell information can be used to characterize the database and the data table to which the data in the table cell belongs. The unit information can be used as index information of the query and acquisition task subsequently, and the database and the data table where the task in the table unit is located can be obtained by analyzing the unit information, so that the task can be read from the corresponding position.
In this embodiment, the task management system may generate the unit information of each table unit and then feed back the unit information to the task scheduling system. The task scheduling system can learn the number of currently existing table elements. To increase the efficiency of task execution, the task scheduling system may assign these table elements to different node servers for synchronous processing. For example, currently there are 16 table elements in total, then the task scheduling system may allocate these 16 table elements to 4 node server processes, thereby providing the processing efficiency of the task by way of parallel processing. In particular, the task management system may be deployed in the node server, and the node server may access the database. In this way, the task scheduling system can send the unit information to the node server. After receiving the unit information, the node server analyzes the unit information, so that the tasks which the node server needs to process can be known. In this way, based on the element information, access can be initiated to the data tables in the database, thereby performing the process of acquiring and executing tasks.
As can be seen from fig. 3, when the instruction is a query instruction, callback information and thread pool information corresponding to the service type may be obtained based on the service type included in the query instruction, and the specified number corresponding to the service type may be obtained. Then, the current time node in the service type can be combined to acquire a specified number of adaptive target tasks from the database. And then, the corresponding service system can be called back through the designated interface represented by the callback information, and the service system feeds back the execution result to the task scheduling system.
The present specification also provides a server in which the task management system may be deployed. Referring to fig. 4, the server may include a management module, an interface module, and a scheduling module.
The management module is used for providing a specified interface for the service system in the interface module, so that the service system calls the specified interface to create tasks bound with the object when creating the object, and the tasks created by the service system are stored in a database.
And the interface module is used for receiving the task which is created by the service system and is bound with the object.
The scheduling module is used for receiving a query instruction sent by a task scheduling system and acquiring a target task matched with the query instruction from the database; and calling back the service system corresponding to the target task, so that the service system corresponding to the target task executes the target task, and the task scheduling system receives an execution result fed back by the service system corresponding to the target task.
In one embodiment, the management module includes:
and the callback information management unit is used for storing callback information of the service system, and the callback information is used for characterizing a designated interface provided for the service system.
And the balance storage unit is used for determining the characteristic value of the task and writing the task into a data table corresponding to the characteristic value.
In one embodiment, the scheduling module includes:
and the access unit is used for accessing the task scheduling system.
And the task acquisition unit is used for acquiring the specified number of target tasks from the database.
And the thread pool management unit is used for storing thread pool information corresponding to the callback information of the service system.
And the task callback unit is used for callback the service system corresponding to the target task according to callback information of the service system.
In one embodiment, the server further comprises a monitoring module, and the monitoring module comprises:
and the alarm unit is used for sending out an alarm prompt when the difference value between the expected scheduling time and the actual scheduling time of the task is greater than or equal to the specified time difference threshold value.
And the interface monitoring unit is used for counting the calling times of the interfaces in the interface module and determining the average value of the calling times of the interfaces in unit time.
The log unit is used for recording the number of tasks created in a specified period and the number of tasks called, and recording the execution result fed back by the service system.
The specific functions of the management module, the interface module, the scheduling module and the database of the server provided in the embodiment of the present disclosure may be explained in comparison with the foregoing method embodiment in the present disclosure, and may achieve the technical effects of the foregoing method embodiment.
Referring to fig. 5, the present disclosure further provides a task processing method, where the task processing method may be applied to the task scheduling system. The method may comprise the following steps.
S21: and sending a data table splitting instruction to a task management system so that the task management system splits the data table in the database into a plurality of table units.
In this embodiment, the task scheduling system may send a data table splitting instruction to the task management system. The data table splitting instruction may cause the task management system to split a data table in a database. The splitting aim is to divide a plurality of tasks into different table units and distribute the split table units to different node servers, so that the execution efficiency of the tasks is improved by a method of parallel processing of a plurality of node servers. Specifically, when the task management system receives a data table splitting instruction sent by the task scheduling system, a data table in the database can be split into a plurality of table units in response to the data table splitting instruction. For example, 256 data tables are currently provided, each 16 data tables can be split into one table unit, and finally 16 table units can be obtained by splitting, wherein each table unit comprises 16 data tables. After splitting to obtain a plurality of table units, the task management system may generate unit information of the table units according to data included in each table unit. The unit information may be data having a fixed format. For example, the element information may be a json string. In the unit information, it can be noted in which data table of which database the data contained in the table unit is located. In this way, the cell information can be used to characterize the database and the data table to which the data in the table cell belongs. The unit information can be used as index information of the query and acquisition task subsequently, and the database and the data table where the task in the table unit is located can be obtained by analyzing the unit information, so that the task can be read from the corresponding position.
S23: and receiving unit information of the table unit fed back by the task management system, wherein the unit information is used for representing a database and a data table to which the data in the table unit belong.
S25: providing the unit information of the table unit to a node server, and sending a query instruction to the node server, so that the node server obtains a target task matched with the query instruction from the table unit, and enables the node server to call back a business system for creating the target task, and receives an execution result for the target task, which is fed back by the business system corresponding to the target task.
In this embodiment, the task management system may generate the unit information of each table unit and then feed back the unit information to the task scheduling system. The task scheduling system can learn the number of currently existing table elements. To increase the efficiency of task execution, the task scheduling system may assign these table elements to different node servers for synchronous processing. For example, currently there are 16 table elements in total, then the task scheduling system may allocate these 16 table elements to 4 node server processes, thereby providing the processing efficiency of the task by way of parallel processing. In particular, the task management system may be deployed in the node server, and the node server may access the database. In this way, the task scheduling system can send the unit information to the node server. After receiving the unit information, the node server analyzes the unit information, so that the tasks which the node server needs to process can be known. In this way, based on the element information, access can be initiated to the data tables in the database, thereby performing the process of acquiring and executing tasks.
In this embodiment, the task scheduling system may send a query instruction to the node server according to a specified time period. When the node server receives the query instruction, the flow of acquiring the task can be triggered. Specifically, the query instruction may include a specific service type, where the specific service type is used to indicate a service system corresponding to a task that needs to be processed. For example, the query is issued for a ticketing system, and then the specified service type contained in the query may be a ticketing type. Further, the tasks created by the business system may include specified fields therein. The specified field may be used to characterize the type of traffic to which the task belongs. For example, the specified field may be a biztype field. The content contained in the biztype field is also different according to different traffic types. For example, the biztype of the object task may be ttp_order, and the biztype of the travel task may be journ ey.
In this embodiment, the specified number of acquisition tasks may be preset in the node server. The specified number may define a number of tasks that the node server obtains at most once. For example, the specified number is 10, and then the node server may acquire a maximum of 10 tasks each time it acquires tasks from the database. In this way, when the target tasks matched with the query instruction are acquired, the service types and the number of the tasks can be limited, the designated number of the tasks with the service types consistent with the designated service types are acquired, and the acquired designated number of the tasks are used as the target tasks matched with the query instruction.
In this embodiment, since the task created by the service system is generally time-efficient, the task may further include a time node for representing that the task is executed. The time node may be automatically generated when a task is created. For example, an object is generated at 17 hours and 58 minutes and 30 seconds, the effective time of the object is half an hour, and then the time node of the task bound to the object may be 18 hours and 28 minutes and 30 seconds. When the task scheduling system sends a query instruction, a current time node may be included in the query instruction. Thus, the node server, upon receiving the query instruction, can determine which tasks need to be performed. Specifically, the node server may take a task that belongs to the specified service type and is earlier than the current time node as a target task that matches the query instruction, and acquire the target task. For example, the query instruction includes an object type, the current time node is 18 hours and 28 minutes and 50 seconds, then when the node server acquires the target task, the service type of a certain task is the object type, the time node is 18 hours and 28 minutes and 30 seconds, then the time node is earlier than the current time node, the task state in the database is to be executed, and then the node server can acquire the task.
In this embodiment, after the target task is acquired, the node server may call back the corresponding service system to execute the target task. Specifically, the node server may store cache information of each service system. The cache information may be stored in association with a service type of the service system. Thus, after the node server obtains the target task, callback information and thread pool information corresponding to the service type can be obtained from the cache according to the service type of the target task. For example, the node server may read the biztype field of the target task, and then determine, from the content contained in the biztype field, the service type to which the target task belongs. The associated callback information and thread pool information are then read from the local cache based on the traffic type. In this way, the node server can call back the service system through the appointed interface characterized by the callback information, and the service system can execute the target task through the thread characterized by the thread pool information. After the business system finishes executing the target task, the execution result can be fed back to the task scheduling system.
Referring to fig. 6, the present disclosure also provides a task scheduling system that includes a network communication port, a processor, and a memory.
The network communication port is used for carrying out network data interaction.
And the memory is used for storing the unit information of the table unit fed back by the task management system.
The processor is used for controlling the network communication port to send a data table splitting instruction to the task management system so that the task management system splits a data table in a database into a plurality of table units; receiving unit information of the table unit fed back by the task management system through the network communication port, wherein the unit information is used for representing a database and a data table to which data in the table unit belong; providing the unit information of the table unit to a node server, and sending a query instruction to the node server, so that the node server obtains a target task matched with the query instruction from the table unit, and enables the node server to call back a business system for creating the target task, and receives an execution result for the target task, which is fed back by the business system corresponding to the target task.
In this embodiment, the network communication port may be a virtual port that binds with different communication protocols, so that different data may be sent or received. For example, the network communication port may be an 80 # port responsible for performing web data communication, a 21 # port responsible for performing FTP data communication, or a 25 # port responsible for performing mail data communication. The network communication port may also be an entity's communication interface or a communication chip. For example, it may be a wireless mobile network communication chip, such as GSM, CDMA, etc.; it may also be a Wifi chip; it may also be a bluetooth chip.
In this embodiment, the memory may include physical means for storing information, typically by digitizing the information and then storing the information in a medium using electrical, magnetic, or optical methods. The memory according to the present embodiment may further include: means for storing information by means of electrical energy, such as RAM, ROM, etc.; devices for storing information by magnetic energy, such as hard disk, floppy disk, magnetic tape, magnetic core memory, bubble memory, and USB flash disk; devices for storing information optically, such as CDs or DVDs. Of course, there are other ways of storing, such as quantum storing, graphene storing, etc.
In this embodiment, the processor may be implemented in any suitable manner. For example, the processor may take the form of, for example, a microprocessor or processor, and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), a programmable logic controller, and an embedded microcontroller, among others.
The specific functions implemented by the network communication port, the processor and the memory of the task scheduling system provided in the embodiment of the present specification can be explained in comparison with the foregoing method embodiment in the present specification, and the technical effects of the foregoing method embodiment can be achieved.
In one embodiment of the present disclosure, the service system, the task scheduling system, and the task management system may be integrated into one system. Thus, the three systems can be used as subsystems in the integrated system. Specifically, the embodiment also provides a task processing system, which comprises a service subsystem, a task scheduling subsystem and a task management subsystem.
The business subsystem is used for creating a task bound with the object when the object is created, and sending the task to the task management subsystem; and executing the target task to be processed when being called back by the task management subsystem.
The task scheduling subsystem is used for sending a query instruction to the task management subsystem and receiving the execution result of the target task fed back by the service subsystem.
The task management subsystem is used for receiving the task sent by the service subsystem and the query instruction sent by the task scheduling subsystem, and acquiring a target task matched with the query instruction from the received task; and calling back the service subsystem corresponding to the target task, so that the service subsystem corresponding to the target task executes the target task, and the task scheduling subsystem receives an execution result fed back by the service subsystem corresponding to the target task.
In one embodiment, the task management subsystem is further configured to provide a specified interface for the service subsystem, so that when the service subsystem creates an object, the specified interface is called to create a task bound to the object.
In one embodiment, the task management subsystem is further configured to set cache information for the service subsystem, where the cache information at least includes callback information and thread pool information of the service subsystem; the callback information is used for representing a specified interface corresponding to the service subsystem, and the thread pool information is used for representing threads for acquiring and executing tasks.
In one embodiment, the task scheduling subsystem is further configured to send a data table splitting instruction to the task management subsystem, so that the task management subsystem splits the data table in the database into a plurality of table units; receiving unit information of the table unit fed back by the task management subsystem, wherein the unit information is used for representing a database and a data table to which data in the table unit belong; providing the unit information of the table unit to a node server, and sending a query instruction to the node server, so that the node server obtains a target task matched with the query instruction from the table unit, enables the node server to call back a service subsystem corresponding to the target task, and receives an execution result for the target task, which is fed back by the service subsystem corresponding to the target task.
The specific functions implemented by the service subsystem, the task scheduling subsystem and the task management subsystem in the foregoing embodiments may refer to the descriptions in the foregoing embodiments, and will not be repeated here.
As can be seen from the technical solutions provided in the embodiments of the present specification, in the present specification, when a service system creates an object, a task bound to the object may be created in a task management system. After the business system has created a task, it is not necessary to query the task. The process of querying the task may be accomplished by a task scheduling system. The task scheduling system may send a query instruction to the task management system to query whether the created task needs to be executed. After receiving the query instruction, the task management system can acquire the target task which should be executed currently. The task management system may create a business system for the target task via a callback to cause the business system to execute the target task. After the execution result is obtained, the service system can feed back the execution result to the task scheduling system. Therefore, after the task is created, the service system only needs to wait for a callback instruction of the task management system, and the created task is not required to be inquired, so that the burden of the service system is reduced, and further, the service and the task can be executed efficiently.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are referred to each other, and each embodiment is mainly described as different from other embodiments.
In the 90 s of the 20 th century, improvements to one technology could clearly be distinguished as improvements in hardware (e.g., improvements to circuit structures such as diodes, transistors, switches, etc.) or software (improvements to the process flow). However, with the development of technology, many improvements of the current method flows can be regarded as direct improvements of hardware circuit structures. Designers almost always obtain corresponding hardware circuit structures by programming improved method flows into hardware circuits. Therefore, an improvement of a method flow cannot be said to be realized by a hardware entity module. For example, a programmable logic device (Programmable Logic Device, PLD) (e.g., field programmable gate array (Field Programmable Gate Array, FPGA)) is an integrated circuit whose logic function is determined by the programming of the device by a user. A designer programs to "integrate" a digital system onto a PLD without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Moreover, nowadays, instead of manually manufacturing integrated circuit chips, such programming is mostly implemented with "logic compiler" software, which is similar to the software compiler used in program development and writing, and the original code before the compiling is also written in a specific programming language, which is called hardware description language (Hardware Description Language, HDL), but HDL is not only one, but a plurality of kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), lava, lola, myHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog2 are most commonly used at present. It will also be apparent to those skilled in the art that a hardware circuit implementing the logic method flow can be readily obtained by merely slightly programming the method flow into an integrated circuit using several of the hardware description languages described above.
Those skilled in the art will also appreciate that, in addition to servers and systems implemented as pure computer readable program code, it is well possible to cause servers and systems to be implemented in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc., by logic programming the method steps. Such servers and systems can therefore be considered as a hardware component, and the means for performing the various functions included therein can also be considered as structure within the hardware component. Or even means for achieving the various functions may be regarded as either software modules implementing the methods or structures within hardware components.
From the above description of embodiments, it will be apparent to those skilled in the art that the present description may be implemented in software plus a necessary general purpose hardware platform. Based on this understanding, the technical solution of the present specification may be embodied in essence or a part contributing to the prior art in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform the method described in the various embodiments or some parts of the embodiments of the present specification.
Although the present description has been described by way of embodiments, those of ordinary skill in the art will recognize that there are many variations and modifications to the present description without departing from the spirit of the present description, and it is intended that the appended claims encompass such variations and modifications without departing from the spirit of the present description.
Claims (20)
1. A method of task processing, the method comprising:
when a business system creates an object, a task management system receives a task which is created by the business system and bound with the object, and stores the task in a database; the business system is used for creating an object needing to be triggered at fixed time in real time and creating a task bound with the object;
receiving a query instruction sent by a task scheduling system, acquiring a target task matched with the query instruction from a database, and further screening out a target task which needs to be executed in time from the matched target task; the inquiry instruction comprises a designated number of designated service types and/or tasks;
and calling back the service system corresponding to the target task, so that the service system corresponding to the target task executes the target task, and the task scheduling system receives an execution result fed back by the service system corresponding to the target task.
2. The method of claim 1, wherein prior to the business system creating the object, the method further comprises:
a designated interface is provided for a business system, so that the business system calls the designated interface to create a task bound with an object when the object is created.
3. The method of claim 1, wherein after receiving the task created by the business system that is bound to the object, the method further comprises:
and determining the characteristic value of the task, and writing the task into a data table corresponding to the characteristic value.
4. A method according to claim 3, wherein determining the feature value of the task comprises:
determining the binary value of the character in the identification of the task, and superposing the determined binary value;
obtaining the number of data tables, performing modular operation on the superposed values and the number of the data tables, and taking the obtained modular values as the characteristic values of the task;
correspondingly, writing the task into the data table corresponding to the characteristic value comprises the following steps:
and determining a target data table with the serial number consistent with the modulus value in the data table, and writing the task into the target data table.
5. The method of claim 1, wherein prior to receiving the query instruction from the task scheduling system, the method further comprises:
receiving a data table splitting instruction sent by a task scheduling system;
responding to the data table splitting instruction, splitting a data table in a database into a plurality of table units, and feeding back unit information of the table units to the task scheduling system; the unit information is used for representing a database and a data table to which the data in the table unit belong.
6. The method according to claim 1, wherein the method further comprises:
setting cache information for the service system, wherein the cache information at least comprises callback information and thread pool information of the service system; the callback information is used for representing a specified interface corresponding to the service system, and the thread pool information is used for representing threads for acquiring and executing tasks.
7. The method according to claim 1, wherein the task includes a specified field, the specified field is used for characterizing a service type to which the task belongs, and the query instruction includes a specified service type; correspondingly, acquiring the target task matched with the query instruction comprises:
Reading a designated field of a task, and determining a service type corresponding to the task based on the designated field;
and acquiring the tasks with the appointed number of service types consistent with the appointed service types, and taking the acquired appointed number of tasks as target tasks matched with the query instruction.
8. The method of claim 7, wherein the task further comprises a time node for characterizing the task being executed, and wherein the query instruction further comprises a current time node; accordingly, after determining the service type corresponding to the task, the method further includes:
and taking the task which belongs to the appointed service type and is earlier than the current time node as a target task matched with the query instruction, and acquiring the target task.
9. The method of claim 1, wherein recalling the business system corresponding to the target task to cause the business system corresponding to the target task to execute the target task comprises:
according to the service type of the target task, callback information corresponding to the service type is obtained from a cache;
and calling back the service system corresponding to the target task through the interface represented by the callback information so that the service system corresponding to the target task executes the target task.
10. The method according to claim 9, wherein after receiving the execution result of the service system feedback corresponding to the target task, the method further comprises:
and when the execution result represents the execution failure, the service system corresponding to the target task is recalled again, so that the service system executes the target task again.
11. A server, comprising a management module, an interface module, and a scheduling module, wherein:
the management module is used for providing a designated interface for a service system in the interface module, so that when the service system creates an object, the designated interface is called to create a task bound with the object, and the task created by the service system is stored in a database; the business system is used for creating an object needing to be triggered at fixed time in real time and creating a task bound with the object;
the interface module is used for receiving the task which is created by the service system and is bound with the object;
the scheduling module is used for receiving a query instruction sent by a task scheduling system and acquiring a target task matched with the query instruction from the database; callback the business system corresponding to the target task, so that the business system corresponding to the target task executes the target task, and the task scheduling system receives an execution result fed back by the business system corresponding to the target task; the inquiry instruction comprises a designated number of designated service types and/or tasks;
The task management system is provided with thread pool information for the service system, wherein the thread pool information is used for characterizing threads of the service system for executing tasks and is used for the service system for executing target tasks; and the callback information and the thread pool information are used as cache information of the service system and are stored in the task management system in a correlated mode with the identification of the service system, and the cache information is updated periodically.
12. The server according to claim 11, wherein the management module includes:
the callback information management unit is used for storing callback information of the service system, and the callback information is used for characterizing a designated interface provided for the service system;
and the balance storage unit is used for determining the characteristic value of the task and writing the task into a data table corresponding to the characteristic value.
13. The server according to claim 11, wherein the scheduling module includes:
the access unit is used for accessing the task scheduling system;
the task acquisition unit is used for acquiring a specified number of target tasks from the database;
the thread pool management unit is used for storing thread pool information corresponding to callback information of the service system;
And the callback execution unit is used for callback the business system corresponding to the target task according to callback information of the business system.
14. The server of claim 11, further comprising a monitoring module, wherein the monitoring module comprises:
the alarm unit is used for sending out an alarm prompt when the difference value between the expected scheduling time and the actual scheduling time of the task is greater than or equal to a specified time difference threshold;
the interface monitoring unit is used for counting the calling times of the interface in the interface module and determining the average value of the calling times of the interface in unit time;
the log unit is used for recording the number of tasks created in a specified period and the number of tasks called, and recording the execution result fed back by the service system.
15. A method of task processing, the method comprising:
sending a data table splitting instruction to a task management system so that the task management system splits a data table in a database into a plurality of table units;
receiving unit information of the table unit fed back by the task management system, wherein the unit information is used for representing a database and a data table to which data in the table unit belong;
Providing the unit information of the table unit to a node server, and sending a query instruction to the node server, so that the node server obtains a target task matched with the query instruction from the table unit, further screens out a target task which needs to be executed in time from the matched target task, enables the node server to call back a service system corresponding to the target task, and receives an execution result which is fed back by the service system corresponding to the target task and is aimed at the target task; wherein, the liquid crystal display device comprises a liquid crystal display device,
the task management system is deployed in the node server, and the node server can access the database;
the business system is used for creating an object needing to be triggered at fixed time in real time and creating a task bound with the object;
the query instruction comprises a designated number of designated service types and/or tasks.
16. A task scheduling system, the system comprising a network communication port, a processor, and a memory, wherein:
the network communication port is used for carrying out network data interaction;
the memory is used for storing the unit information of the table unit fed back by the task management system;
The processor is used for controlling the network communication port to send a data table splitting instruction to the task management system so that the task management system splits a data table in a database into a plurality of table units; receiving unit information of the table unit fed back by the task management system through the network communication port, wherein the unit information is used for representing a database and a data table to which data in the table unit belong; providing the unit information of the table unit to a node server, and sending a query instruction to the node server, so that the node server obtains a target task matched with the query instruction from the table unit, and enables the node server to call back a service system corresponding to the target task, and receives an execution result for the target task fed back by the service system corresponding to the target task;
wherein the task management system is deployed in the node server, and the node server has access to the database; the business system is used for creating an object needing to be triggered at fixed time in real time and creating a task bound with the object; the query instruction comprises a designated number of designated service types and/or tasks.
17. The task processing system is characterized by comprising a service subsystem, a task scheduling subsystem and a task management subsystem, wherein:
the business subsystem is used for creating a task bound with the object when the object is created in real time and sending the task to the task management subsystem; executing the target task to be processed when being called back by the task management subsystem;
the task scheduling subsystem is used for sending a query instruction to the task management subsystem and receiving an execution result of the target task fed back by the service subsystem; the inquiry instruction comprises a designated number of designated service types and/or tasks;
the task management subsystem is used for receiving the task sent by the service subsystem and the query instruction sent by the task scheduling subsystem, and acquiring a target task matched with the query instruction from the received task; and calling back the service subsystem corresponding to the target task, so that the service subsystem corresponding to the target task executes the target task, and the task scheduling subsystem receives an execution result fed back by the service subsystem corresponding to the target task.
18. The system of claim 17, wherein the task management subsystem is further configured to provide a specified interface to the business subsystem such that the business subsystem, when creating an object, invokes the specified interface to create a task bound to the object.
19. The system of claim 17, wherein the task management subsystem is further configured to set cache information for the service subsystem, where the cache information includes at least callback information and thread pool information of the service subsystem; the callback information is used for representing a specified interface corresponding to the service subsystem, and the thread pool information is used for representing threads for acquiring and executing tasks.
20. The system of claim 17, wherein the task scheduling subsystem is further configured to send a data table splitting instruction to the task management subsystem to cause the task management subsystem to split a data table in a database into a plurality of table units; receiving unit information of the table unit fed back by the task management subsystem, wherein the unit information is used for representing a database and a data table to which data in the table unit belong; providing the unit information of the table unit to a node server, and sending a query instruction to the node server, so that the node server obtains a target task matched with the query instruction from the table unit, enables the node server to call back a service subsystem corresponding to the target task, and receives an execution result for the target task, which is fed back by the service subsystem corresponding to the target task.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810041018.4A CN110109986B (en) | 2018-01-16 | 2018-01-16 | Task processing method, system, server and task scheduling system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810041018.4A CN110109986B (en) | 2018-01-16 | 2018-01-16 | Task processing method, system, server and task scheduling system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110109986A CN110109986A (en) | 2019-08-09 |
| CN110109986B true CN110109986B (en) | 2023-08-11 |
Family
ID=67483446
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810041018.4A Active CN110109986B (en) | 2018-01-16 | 2018-01-16 | Task processing method, system, server and task scheduling system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110109986B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111611058A (en) * | 2020-05-21 | 2020-09-01 | 北京明略软件系统有限公司 | Task execution method and device and electronic equipment |
| CN112995306B (en) * | 2021-02-05 | 2023-10-20 | 建信金融科技有限责任公司 | Real-time accounting information processing method and system based on storm |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101894348A (en) * | 2010-07-20 | 2010-11-24 | 中兴通讯股份有限公司 | Self-expanded online transaction system and implementing method thereof |
| US20120158451A1 (en) * | 2010-12-16 | 2012-06-21 | International Business Machines Corporation | Dispatching Tasks in a Business Process Management System |
| CN104765639A (en) * | 2015-04-30 | 2015-07-08 | 北京奇虎科技有限公司 | Timed task scheduling system and method |
| CN107092523A (en) * | 2017-04-14 | 2017-08-25 | 浙江数链科技有限公司 | The dispatching method and system of timed task |
-
2018
- 2018-01-16 CN CN201810041018.4A patent/CN110109986B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101894348A (en) * | 2010-07-20 | 2010-11-24 | 中兴通讯股份有限公司 | Self-expanded online transaction system and implementing method thereof |
| US20120158451A1 (en) * | 2010-12-16 | 2012-06-21 | International Business Machines Corporation | Dispatching Tasks in a Business Process Management System |
| CN104765639A (en) * | 2015-04-30 | 2015-07-08 | 北京奇虎科技有限公司 | Timed task scheduling system and method |
| CN107092523A (en) * | 2017-04-14 | 2017-08-25 | 浙江数链科技有限公司 | The dispatching method and system of timed task |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110109986A (en) | 2019-08-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11836533B2 (en) | Automated reconfiguration of real time data stream processing | |
| CN109690524B (en) | Data serialization in a distributed event processing system | |
| CN111767143B (en) | Transaction data processing method, device, equipment and system | |
| US10216584B2 (en) | Recovery log analytics with a big data management platform | |
| US10715460B2 (en) | Opportunistic resource migration to optimize resource placement | |
| US10338958B1 (en) | Stream adapter for batch-oriented processing frameworks | |
| CN109002362A (en) | A kind of method of servicing, device, system and electronic equipment | |
| CN105786603B (en) | Distributed high-concurrency service processing system and method | |
| US10922357B1 (en) | Automatically mapping natural language commands to service APIs | |
| CN105824957A (en) | Query engine system and query method of distributive memory column-oriented database | |
| CN109033109B (en) | Data processing method and system | |
| US11176086B2 (en) | Parallel copying database transaction processing | |
| CN107766343B (en) | Data storage method and device and storage server | |
| CN111782365B (en) | Timed task processing method, device, equipment and storage medium | |
| CN110532074A (en) | A kind of method for scheduling task and system of multi-tenant Mode S aaS service cluster environment | |
| US10394775B2 (en) | Order constraint for transaction processing with snapshot isolation on non-transactional NoSQL servers | |
| EP3049959A1 (en) | Processing a hybrid flow associated with a service class | |
| CN114090580A (en) | Data processing method, device, equipment, storage medium and product | |
| CN110109986B (en) | Task processing method, system, server and task scheduling system | |
| EP3422186A1 (en) | Method, device and system for preventing memory data loss | |
| US10348814B1 (en) | Efficient storage reclamation for system components managing storage | |
| CN113704355A (en) | Data synchronization method, electronic device and storage medium | |
| WO2017050177A1 (en) | Data synchronization method and device | |
| CN110019296B (en) | Database query script generation method and device, storage medium and processor | |
| CN111797070A (en) | Ticket data processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |