CN117615014A - Rectifying scheduling method, device, equipment and medium for mass tasks - Google Patents

Rectifying scheduling method, device, equipment and medium for mass tasks Download PDF

Info

Publication number
CN117615014A
CN117615014A CN202311369840.0A CN202311369840A CN117615014A CN 117615014 A CN117615014 A CN 117615014A CN 202311369840 A CN202311369840 A CN 202311369840A CN 117615014 A CN117615014 A CN 117615014A
Authority
CN
China
Prior art keywords
task
tasks
time
interval
center
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311369840.0A
Other languages
Chinese (zh)
Inventor
魏鹏
余伟杰
张良标
汪沁婷
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Send Intelligent Technology Co ltd
Original Assignee
Zhejiang Send Intelligent Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Send Intelligent Technology Co ltd filed Critical Zhejiang Send Intelligent Technology Co ltd
Priority to CN202311369840.0A priority Critical patent/CN117615014A/en
Publication of CN117615014A publication Critical patent/CN117615014A/en
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4812—Task transfer initiation or dispatching by interrupt, e.g. masked
    • G06F9/4831—Task transfer initiation or dispatching by interrupt, e.g. masked with variable priority
    • G06F9/4837—Task transfer initiation or dispatching by interrupt, e.g. masked with variable priority time dependent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/50—Queue scheduling
    • H04L47/62—Queue scheduling characterised by scheduling criteria
    • H04L47/625—Queue scheduling characterised by scheduling criteria for service slots or service orders
    • H04L47/6255—Queue scheduling characterised by scheduling criteria for service slots or service orders queue load conditions, e.g. longest queue first
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The invention discloses a rectification scheduling method, device, equipment and medium for mass tasks, wherein the method comprises the following steps: creating an access party application under which a task management service registers a scene; the access party builds and registers a task to a task center; scanning inquiry is carried out on all tasks at each preset time interval of a task center so as to acquire a minimum task ID and a maximum task ID in all tasks when the task execution time is earlier than any task execution time in the current scanning starting time; setting a closed interval by using the two IDs, and segmenting to obtain a task list; the task center sends the task list to a task callback service, the task callback service extracts tasks from the task list, and the extracted tasks are randomly distributed to each time point of a scanning interval to generate a delay queue; and the task in the queue wakes up on schedule, and a corresponding executive party is selected to execute the task according to the scene name and the application information of the task. The load can be reduced, the resources can be saved, and the dispatching reliability can be improved.

Description

Rectifying scheduling method, device, equipment and medium for mass tasks
Technical Field
The invention relates to the technical field of data processing, in particular to a rectification scheduling method, device, equipment and medium for mass tasks.
Background
In consumer-oriented internet software systems, consumer-oriented massive tasks are executed in the background and some business operations are completed. For example: closing an order that is unpaid over time, timing synchronizing merchant product information to the consumer, or synchronizing order information for a large number of short time predetermined tickets for the consumer to the merchant, etc. Because these tasks are largely executed in a short time, the task executor system can face instantaneous high-flow requests, if the processing is improper, the task processing is delayed, and if the processing is heavy, the executor system can be crashed, finally the task is delayed or not executed, and the use of consumers is affected.
In view of the above problems, the task scheduling method adopted in the prior art schedules the task in a set scheduling time period, however, the scheduling method has the following drawbacks:
(1) If there are a huge number of tasks to be scheduled suddenly in the set scheduling period, the problem of overload of the working node resources is caused, for example: in the set scheduling time period, nearly hundred thousand tasks are scheduled to the working nodes, and the working nodes are high in concurrency and resource overload due to the fact that the working nodes are limited;
(2) If a task scheduling flow flood peak occurs in a set scheduling time period, an excessive number of working nodes need to be deployed for bearing the situation, however, the redundant working nodes can cause resource waste;
(3) If current limiting is used for controlling the load of the working node, the instantaneous overload flow is refused, and a large number of updated tasks exist in the task scheduling system for a short time, so that the load is too high.
Therefore, aiming at the problems of overload of working node resources, resource waste or excessive load of a dispatching system when dispatching and executing massive tasks in the prior art, no effective solution has been proposed yet.
Disclosure of Invention
The embodiment of the invention provides a rectifying and dispatching method, device, equipment and medium for massive tasks, which aim to solve the problems of overload of working node resources, resource waste or overload of a dispatching system when massive tasks are dispatched and executed in the prior art.
In a first aspect, an embodiment of the present invention provides a method for rectifying and scheduling a massive task, where the method is applied to a rectifying and scheduling device for a massive task, where the device includes an access party cluster, an execution party cluster, a task registration service, a task center, a task management service, and a task callback service, and the method includes:
Creating an access party application through an API or a page, and registering different set scenes by the task management service under the access party application;
the method comprises the steps that an access party working node in an access party cluster builds a task of the same type under each registered scene, and the task of the same type built under each scene is registered to a task center through a task registration service, wherein information in the task comprises scene names, service parameters and task execution time;
setting current scanning starting time, wherein the task center starts from the current scanning starting time, scans and inquires all tasks in a plurality of scenes once every interval preset time interval to acquire all tasks with task execution time earlier than any task execution time in the current scanning starting time, and determines a minimum task ID and a maximum task ID in all tasks, wherein the starting time point of the interval preset time interval is the ending time of the current scanning, and the ending time point is the starting time of the next scanning;
setting a closed interval by using the minimum task ID and the maximum task ID, and segmenting the closed interval according to a preset step length to obtain a task list;
The task center sends the task list to the task callback service, the task callback service traverses the task list, extracts tasks with task execution time earlier than the current scanning starting time, randomly distributes the extracted tasks to each time point of a scanning interval, and generates a delay queue, wherein the scanning interval is a preset time interval of each scanning interval;
the task in the delay queue triggers wake-up according to different time points, and a task callback triggering service selects a corresponding executive working node from the executive cluster to execute the task according to the scene name of the awakened task and the application information of the access party;
and the executive party returns an executive result to the task center.
In a second aspect, an embodiment of the present invention provides a rectifying and scheduling device for a massive task, where the device includes an access party cluster, an execution party cluster, a task registration service, a task center, a task management service, and a task callback service;
creating an access party application through an API or a page, and registering different set scenes by the task management service under the access party application;
The method comprises the steps that an access party working node in an access party cluster builds a task of the same type under each registered scene, and the task of the same type built under each scene is registered to a task center through a task registration service, wherein information in the task comprises scene names, service parameters and task execution time;
setting current scanning starting time, wherein the task center starts from the current scanning starting time, scans and inquires all tasks in a plurality of scenes once every interval preset time interval to acquire all tasks with task execution time earlier than any task execution time in the current scanning starting time, and determines a minimum task ID and a maximum task ID in all tasks, wherein the starting time point of the interval preset time interval is the ending time of the current scanning, and the ending time point is the starting time of the next scanning;
setting a closed interval by using the minimum task ID and the maximum task ID, and segmenting the closed interval according to a preset step length to obtain a task list;
the task center sends the task list to the task callback service, the task callback service traverses the task list, extracts tasks with task execution time earlier than the current scanning starting time, randomly distributes the extracted tasks to each time point of a scanning interval, and generates a delay queue, wherein the scanning interval is a preset time interval of each scanning interval;
The task in the delay queue triggers wake-up according to different time points, and a task callback triggering service selects a corresponding executive working node from the executive cluster to execute the task according to the scene name of the awakened task and the application information of the access party;
and the executive party returns an executive result to the task center.
In a third aspect, an embodiment of the present invention further provides a computer apparatus, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the method described in the first aspect.
In a fourth aspect, embodiments of the present invention also provide a computer readable storage medium, wherein the computer readable storage medium stores a computer program, which when executed by a processor causes the processor to perform the method according to the first aspect.
The embodiment of the invention provides a method and a device for rectifying and dispatching mass tasks, wherein the device comprises an access party cluster, an execution party cluster, a task registration service, a task center, a task management service and a task callback service. Creating an access party application through an API or a page, and registering different set scenes by a task management service under the access party application; the method comprises the steps that an access party working node in an access party cluster constructs tasks of the same type in each registered scene, and registers the tasks of the same type constructed in each scene to a task center through task registration service, wherein information in the tasks comprises scene names, service parameters and task execution time; setting current scanning starting time, wherein a task center starts from the current scanning starting time, each interval preset time interval scans and inquires all tasks in a plurality of scenes once to acquire all tasks with task execution time earlier than any task execution time in the current scanning starting time, and determining a minimum task ID and a maximum task ID in all tasks, wherein the starting time point of the interval preset time interval is the ending time of the current scanning, and the ending time point is the starting time of the next scanning; setting a closed interval by using a minimum task ID and a maximum task ID, and segmenting the closed interval according to a preset step length to obtain a task list; the task center sends the task list to the task callback service, the task callback service traverses the task list, extracts tasks with task execution time earlier than the current scanning starting time, randomly distributes the extracted tasks to each time point of a scanning interval, and generates a delay queue, wherein the scanning interval is a preset time interval of each scanning interval; triggering awakening of tasks in the delay queue according to different time points, and selecting corresponding executive working nodes from the executive cluster by a task callback triggering service according to scene names of the awakened tasks and access party application information to execute the tasks; and the executive returns the executive result to the task center.
Since the prior art schedules tasks in a set scheduling period, it is uncertain what tasks are specifically performed and how many tasks appear in the period, and short-term traffic peaks are likely to appear. In the method, the execution time of each task in a mass task list to be executed is set in advance, and the tasks are introduced into a delay rectifying queue in a task callback service so as to ensure that the execution of each task is controllable, thereby flexibly releasing the scheduling flow to an executive work node, avoiding short-time flow flood peak, improving the utilization rate of the work node and reducing the overload problem of the work node when facing the mass scheduling task. In addition, the method can also ensure the reliability of task scheduling execution, for example, if the network times out or responds to scheduling failure, the task center can restart scheduling in the next scheduling period, and each task is ensured to be executed.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic block diagram of a rectification scheduling device for massive tasks, which is provided by an embodiment of the invention;
fig. 2 is a flow chart of a rectification scheduling method for massive tasks, which is provided by the embodiment of the invention;
fig. 3 is a schematic block diagram of a computer device according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be understood that the terms "comprises" and "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
The embodiment of the application provides a rectifying and dispatching device for massive tasks, and fig. 1 is a schematic block diagram of the rectifying and dispatching device for massive tasks, provided by the embodiment of the invention, and as shown in fig. 1, the device comprises an access party cluster, an execution party cluster, a task registration service, a task center, a task management service and a task callback service;
the access party cluster is used for accessing scene and application information, and a plurality of tasks can be performed in one scene.
The task registration service is used for task registration.
The task management service is used for processing registration, enabling, disabling and deleting of the scene of the access party, and is a service provided for a developer or an administrator.
The task center is used for acquiring a large number of tasks to be executed from the storage system at regular time and submitting the tasks to the task callback service.
The task callback service is used for processing the task submitted by the task center and scheduling the task to any working node in the executive cluster for executing the task.
The executive party cluster is used for executing the task and feeding back an execution result to the task center.
Specifically, the flow of rectification scheduling by the device is as follows:
step one, an access party application is created through an API or a page, and under the access party application, the task management service registers different set scenes.
In this embodiment, the access party application is created through an API or a page, and it is to be noted that the access party application is related to service content, for example, an order application, a vote application, and the like. Then, under the application of different access parties, after the manual input or the system automatically obtains the scene name and remark information through the communication interface, the task management service registers the set different scenes, thereby completing scene registration. For example, under an order application, the task management service registers an order closing scenario that has been set.
Preferably, before the task management service registers different set scenes, detection and judgment are performed on the set scenes according to a detection strategy, if the scenes do not meet the detection requirement, disabling or deleting the scenes which do not meet the detection requirement, and if the scenes which meet the detection requirement can be registered. Wherein, the detection strategy can be adaptively adjusted according to the actual detection content. For example, if the configuration information of the set scene is detected, the task management service disables the scene if the configuration information is incomplete or has an error. If the scene is duplicated, the task management service deletes the duplicated scene. In addition, if a scene with error in certain configuration information is corrected, the task management service can enable the scene.
Step two, the access party working node in the access party cluster builds the tasks of the same type under each registered scene, and registers the tasks of the same type built under each scene to a task center through task registration service, wherein the information in the tasks comprises scene names, service parameters and task execution time.
In this embodiment, the working node of the access party under the access party cluster constructs the tasks of the same type under each scene registered in step S110, where the information in the constructed tasks includes scene name, service parameter and task execution time. For example, the access side working node builds an order closing task for a customer at a time and builds an order closing task for a customer at a time in an order closing scenario, and the like. It should be noted that the service parameters include personal information of the consumer, order number, order time, etc.
And registering the tasks of the same type constructed under each scene to a task center through a task registration service. For example, the task registration service registers the order closing task of A, B consumer in the order closing scenario to the task center.
Preferably, before registering the tasks of the same type constructed under each scene to the task center through the task registration service, the task center checks whether the current task is registered before according to information in the task, such as scene names and service parameters; if the current task is not registered before, the current task is registered in a task center, otherwise, if the current task is registered before, the task information data of the registered task is modified according to the information of the current task.
After the task registration is completed, the task center returns the task ID successfully registered to the access party.
Setting the current scanning starting time, wherein the task center starts from the current scanning starting time, and scans and inquires all tasks in a plurality of scenes once in each preset time interval to acquire all tasks with task execution time earlier than any task execution time in the current scanning starting time, and determining the minimum task ID and the maximum task ID in all tasks, wherein the starting time point of the preset time interval at intervals is the ending time of the current scanning, and the ending time point is the starting time of the next scanning.
In this embodiment, a current scan start time T is set, a task center starts scanning and querying all tasks in multiple scenes from the current scan start time T, and performs a second scan at preset time intervals after the scanning is completed, that is, each preset time interval performs one scan and querying all tasks in multiple scenes, so as to obtain a minimum task ID and a maximum task ID in all tasks in any task execution time earlier than the current scan start time T, and the minimum task ID and the maximum task ID are respectively recorded as Min and Max, wherein a start time point of the preset time interval of the interval is an end time of the current scan, and a termination time point is a start time of the next scan. For example, the task center scans and inquires all tasks in a plurality of scenes, and if the execution time of 1 ten thousands of tasks with the execution time of the task being earlier than the current scanning start time T is respectively T (T < T), all the tasks at the T moment are acquired, and the minimum task ID and the maximum task ID are acquired and respectively recorded as Min and Max.
And step four, setting a closed section by using the minimum task ID and the maximum task ID, and segmenting the closed section according to a preset step length to obtain a task list.
In this embodiment, the task center sets a closed section with a minimum task ID as a starting point and a maximum task ID as a node, and segments the closed section according to a step length set by a user to obtain a task list.
And fifthly, the task center sends the task list to the task callback service, the task callback service traverses the task list, extracts tasks with execution time of the tasks being earlier than the current scanning starting time, and randomly distributes the extracted tasks to each time point of a scanning interval to generate a delay queue, wherein the scanning interval specifically refers to a preset time interval of each scanning interval.
In this embodiment, the task center sends the task list to the task callback service, the task callback service traverses the task list, extracts a task whose task execution time is earlier than the current scanning start time T, and the extracted task requests the delay generation service, so that the delay generation service randomly distributes the extracted task to each time point of the task scanning interval to generate a delay queue, so as to complete rectification of the task. The scheduling traffic can be flexibly released to the working node of the executive party through the delay queue so as to avoid abnormal load of the working node.
Preferably, after the task callback service traverses the task list and extracts the task with the task execution time earlier than the current scanning start time, the type of the extracted task can be judged; if the extracted task is a time sensitive task, preferentially distributing the extracted task to the earliest time point of a preset time interval of a scanning interval so as to preferentially execute the task; if the extracted task is a throughput priority task, the extracted task is only required to be randomly allocated to each time point of a preset time interval of the scanning interval.
And step six, triggering the awakening of the tasks in the delay queue according to different time points, and selecting a corresponding executive working node from the executive cluster by the task callback triggering service according to the scene name of the awakened task and the application information of the access party to execute the tasks.
In this embodiment, the tasks in the delay queue trigger wakeup according to different time points, and the task callback triggering service selects a corresponding executive working node from the executive cluster to execute the task according to the scene name of the awakened task and the application information of the access party.
And step seven, the executive party returns the executive result to the task center.
Preferably, when the execution party returns the execution result to the task center, if the execution result does not return to the task center after exceeding the designated time, the task center performs scheduling execution again on the current task in the next scheduling period, so as to ensure that the current task is executed, thereby improving the reliability of task scheduling execution.
Preferably, after the executive returns the executive result to the task center, if the executive result is normal, the current task is finished, for example, the order closes the task. Alternatively, the current task continues to execute in the next scheduling period, e.g., a product update task, a product sync task, etc.
Preferably, after the executive returns the executive result to the task center, if the executive result is abnormal, the task center performs scheduling execution again on the current task in the next scheduling period.
In the rectifying and dispatching device for massive tasks, which is provided by the embodiment of the invention, the execution time of each task in the massive task list to be executed is set in advance, and the tasks are introduced into the delay rectifying queue in the task callback service, so that the execution of each task is controllable, the dispatching flow can be flexibly released to the working node of an executive party, short-time flow flood peak is avoided, the utilization rate of the working node is improved, and the overload problem of the working node facing the massive dispatching tasks is reduced. In addition, the method can also ensure the reliability of task scheduling execution, for example, if the network times out or responds to scheduling failure, the task center can restart scheduling in the next scheduling period, and each task is ensured to be executed.
The embodiment of the application also provides a mass task-oriented rectification scheduling method, so as to be applied to the mass task-oriented rectification scheduling device, and fig. 2 is a schematic flow diagram of the mass task-oriented rectification scheduling method provided by the embodiment of the invention, and as shown in fig. 2, the method comprises steps S110-S170.
S110, creating an access party application through an API or a page, and registering different set scenes by the task management service under the access party application.
In this embodiment, the access party application is created through an API or a page, and it is to be noted that the access party application is related to service content, for example, an order application, a vote application, and the like. Then, under the application of different access parties, after the manual input or the system automatically obtains the scene name and remark information through the communication interface, the task management service registers the set different scenes, thereby completing scene registration. For example, under an order application, the task management service registers an order closing scenario that has been set.
Preferably, before the task management service registers different set scenes, detection and judgment are performed on the set scenes according to a detection strategy, if the scenes do not meet the detection requirement, disabling or deleting the scenes which do not meet the detection requirement, and if the scenes which meet the detection requirement can be registered. Wherein, the detection strategy can be adaptively adjusted according to the actual detection content. For example, if the configuration information of the set scene is detected, the task management service disables the scene if the configuration information is incomplete or has an error. If the scene is duplicated, the task management service deletes the duplicated scene. In addition, if a scene with error in certain configuration information is corrected, the task management service can enable the scene.
S120, constructing the same type of tasks under each registered scene by the access party working node under the access party cluster, and registering the same type of tasks constructed under each scene to a task center through a task registration service, wherein the information in the tasks comprises scene names, service parameters and task execution time.
In this embodiment, the working node of the access party under the access party cluster constructs the tasks of the same type under each scene registered in step S110, where the information in the constructed tasks includes scene name, service parameter and task execution time. For example, the access side working node builds an order closing task for a customer at a time and builds an order closing task for a customer at a time in an order closing scenario, and the like. It should be noted that the service parameters include personal information of the consumer, order number, order time, etc.
And registering the tasks of the same type constructed under each scene to a task center through a task registration service. For example, the task registration service registers the order closing task of A, B consumer in the order closing scenario to the task center.
Preferably, before registering the tasks of the same type constructed under each scene to the task center through the task registration service, the task center checks whether the current task is registered before according to information in the task, such as scene names and service parameters; if the current task is not registered before, the current task is registered in a task center, otherwise, if the current task is registered before, the task information data of the registered task is modified according to the information of the current task.
After the task registration is completed, the task center returns the task ID successfully registered to the access party.
S130, setting current scanning starting time, wherein a task center starts from the current scanning starting time, scanning and inquiring all tasks in a plurality of scenes once in each preset time interval to acquire all tasks with task execution time earlier than any task execution time in the current scanning starting time, and determining a minimum task ID and a maximum task ID in all tasks, wherein the starting time point of the preset time interval at intervals is the ending time of the current scanning, and the ending time point is the starting time of the next scanning.
In this embodiment, a current scan start time T is set, a task center starts scanning and querying all tasks in multiple scenes from the current scan start time T, and performs a second scan at preset time intervals after the scanning is completed, that is, each preset time interval performs one scan and querying all tasks in multiple scenes, so as to obtain a minimum task ID and a maximum task ID in all tasks in any task execution time earlier than the current scan start time T, and the minimum task ID and the maximum task ID are respectively recorded as Min and Max, wherein a start time point of the preset time interval of the interval is an end time of the current scan, and a termination time point is a start time of the next scan. For example, the task center scans and inquires all tasks in a plurality of scenes, and if the execution time of 1 ten thousands of tasks with the execution time of the task being earlier than the current scanning start time T is respectively T (T < T), all the tasks at the T moment are acquired, and the minimum task ID and the maximum task ID are acquired and respectively recorded as Min and Max.
S140, setting a closed section with the minimum task ID and the maximum task ID, and segmenting the closed section according to a preset step length to obtain a task list.
In this embodiment, the task center sets a closed section with a minimum task ID as a starting point and a maximum task ID as a node, and segments the closed section according to a step length set by a user to obtain a task list.
And S150, the task center sends the task list to a task callback service, the task callback service traverses the task list, extracts tasks with execution time of the tasks being earlier than the current scanning starting time, and randomly distributes the extracted tasks to each time point of a scanning interval to generate a delay queue, wherein the scanning interval is a preset time interval of each scanning interval.
In this embodiment, the task center sends the task list to the task callback service, the task callback service traverses the task list, extracts a task whose task execution time is earlier than the current scan start time T, and the extracted task requests the delay generation service, so that the delay generation service randomly distributes the extracted task to each time point of a preset time interval in the task scan, and generates a delay queue to complete rectification of the task. The scheduling traffic can be flexibly released to the working node of the executive party through the delay queue so as to avoid abnormal load of the working node.
Preferably, after the task callback service traverses the task list and extracts the task with the task execution time earlier than the current scanning start time, the type of the extracted task can be judged; if the extracted task is a time sensitive task, preferentially distributing the extracted task to the earliest time point of a preset time interval of a scanning interval so as to preferentially execute the task; if the extracted task is a throughput-priority task, the extracted task is only required to be randomly allocated to each time point of a scanning interval, wherein the scanning interval specifically refers to a preset time interval of each scanning interval.
S160, triggering the awakening of the tasks in the delay queue according to different time points, and selecting a corresponding executive working node from the executive cluster by the task callback triggering service according to the scene name of the awakened task and the application information of the access party to execute the tasks.
In this embodiment, the tasks in the delay queue trigger wakeup according to different time points, and the task callback triggering service selects a corresponding executive working node from the executive cluster to execute the task according to the scene name of the awakened task and the application information of the access party.
S170, the executive returns the executive result to the task center.
Preferably, when the execution party returns the execution result to the task center, if the execution result does not return to the task center after exceeding the designated time, the task center performs scheduling execution again on the current task in the next scheduling period, so as to ensure that the current task is executed, thereby improving the reliability of task scheduling execution.
Preferably, after the executive returns the executive result to the task center, if the executive result is normal, the current task is finished, for example, the order closes the task. Alternatively, the current task continues to execute in the next scheduling period, e.g., a product update task, a product sync task, etc.
Preferably, after the executive returns the executive result to the task center, if the executive result is abnormal, the task center performs scheduling execution again on the current task in the next scheduling period.
In the rectifying and dispatching method for the massive tasks, which is provided by the embodiment of the invention, the execution time of each task in the massive task list to be executed is set in advance, and the tasks are introduced into the delay rectifying queue in the task callback service so as to ensure that the execution of each task is controllable, thereby flexibly releasing the dispatching flow to the working node of an executing party, avoiding short-time flow flood peak, improving the utilization rate of the working node and reducing the overload problem of the working node when facing the massive dispatching tasks. In addition, the method can also ensure the reliability of task scheduling execution, for example, if the network times out or responds to scheduling failure, the task center can restart scheduling in the next scheduling period, and each task is ensured to be executed.
The above-described mass task oriented rectification scheduling method may be implemented in the form of a computer program that may be run on a computer device as shown in fig. 3.
Referring to fig. 3, fig. 3 is a schematic block diagram of a computer device according to an embodiment of the present invention. The computer device may be a commutative scheduling method for performing mass-oriented tasks.
With reference to FIG. 3, the computer device 500 includes a processor 502, a memory, and a network interface 505, connected by a system bus 501, where the memory may include a storage medium 503 and an internal memory 504.
The storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032, when executed, may cause the processor 502 to perform a mass task oriented rectification scheduling method, wherein the storage medium 503 may be a volatile storage medium or a non-volatile storage medium.
The processor 502 is used to provide computing and control capabilities to support the operation of the overall computer device 500.
The internal memory 504 provides an environment for the execution of a computer program 5032 in the storage medium 503, which computer program 5032, when executed by the processor 502, enables the processor 502 to perform a mass task oriented commutative scheduling method.
The network interface 505 is used for network communication, such as providing for transmission of data information, etc. It will be appreciated by those skilled in the art that the architecture shown in fig. 3 is merely a block diagram of some of the architecture relevant to the present inventive arrangements and is not limiting of the computer device 500 to which the present inventive arrangements may be implemented, and that a particular computer device 500 may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
The processor 502 is configured to run a computer program 5032 stored in a memory, so as to implement the corresponding functions in the above-mentioned rectification scheduling method for mass tasks.
Those skilled in the art will appreciate that the embodiment of the computer device shown in fig. 3 is not limiting of the specific construction of the computer device, and in other embodiments, the computer device may include more or less components than those shown, or certain components may be combined, or a different arrangement of components. For example, in some embodiments, the computer device may include only a memory and a processor, and in such embodiments, the structure and function of the memory and the processor are consistent with the embodiment shown in fig. 3, and will not be described again.
It should be appreciated that in an embodiment of the invention, the processor 502 may be a central processing unit (Central Processing Unit, CPU), the processor 502 may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSPs), application specific integrated circuits (Application Specific Integrated Circuit, ASICs), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. Wherein the general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
In another embodiment of the invention, a computer-readable storage medium is provided. The computer readable storage medium may be a volatile or nonvolatile computer readable storage medium. The computer readable storage medium stores a computer program, wherein the computer program when executed by a processor implements the steps included in the above-described mass task oriented rectification scheduling method.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, specific working procedures of the apparatus, device and unit described above may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein. Those of ordinary skill in the art will appreciate that the elements and algorithm steps described in connection with the embodiments disclosed herein may be embodied in electronic hardware, in computer software, or in a combination of the two, and that the elements and steps of the examples have been generally described in terms of function in the foregoing description to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus, device and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, and for example, the division of the units is merely a logical function division, there may be another division manner in actual implementation, or units having the same function may be integrated into one unit, for example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. In addition, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices, or elements, or may be an electrical, mechanical, or other form of connection.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment of the present invention.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention is essentially or part of what contributes to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a computer-readable storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned computer-readable storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, an optical disk, or other various media capable of storing program codes.
While the invention has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.

Claims (10)

1. The method is characterized in that the method is applied to a rectifying and scheduling device for mass tasks, wherein the device comprises an access party cluster, an execution party cluster, a task registration service, a task center, a task management service and a task callback service, and the method comprises the following steps:
creating an access party application through an API or a page, and registering different set scenes by the task management service under the access party application;
the method comprises the steps that an access party working node in an access party cluster builds a task of the same type under each registered scene, and the task of the same type built under each scene is registered to a task center through a task registration service, wherein information in the task comprises scene names, service parameters and task execution time;
Setting current scanning starting time, wherein the task center starts from the current scanning starting time, scans and inquires all tasks in a plurality of scenes once every interval preset time interval to acquire all tasks with task execution time earlier than any task execution time in the current scanning starting time, and determines a minimum task ID and a maximum task ID in all tasks, wherein the starting time point of the interval preset time interval is the ending time of the current scanning, and the ending time point is the starting time of the next scanning;
setting a closed interval by using the minimum task ID and the maximum task ID, and segmenting the closed interval according to a preset step length to obtain a task list;
the task center sends the task list to the task callback service, the task callback service traverses the task list, extracts tasks with task execution time earlier than the current scanning starting time, randomly distributes the extracted tasks to each time point of a scanning interval, and generates a delay queue, wherein the scanning interval is a preset time interval of each scanning interval;
the task in the delay queue triggers wake-up according to different time points, and a task callback triggering service selects a corresponding executive working node from the executive cluster to execute the task according to the scene name of the awakened task and the application information of the access party;
And the executive party returns an executive result to the task center.
2. The method of claim 1, wherein prior to the task management service registering different scenarios for a setting, the method comprises:
and detecting and judging the set scenes according to the detection strategy, and if the set scenes do not meet the detection requirement, disabling or deleting the scenes which do not meet the detection requirement.
3. The method according to claim 1, wherein before the registration of the same type of task constructed in each scenario to the task center through the task registration service, the method comprises:
the task center checks whether the current task is registered before according to the scene name and the service parameter in the task information;
and if the current task is not registered before, registering the current task to the task center, otherwise, if the current task is registered before, modifying task information data of the registered task according to the information of the current task.
4. The method of claim 1, wherein after the task callback service traverses the task list to extract tasks having task execution times earlier than the current scan start time, the method further comprises:
Judging the type of the extracted task;
if the extracted task is a time-sensitive task, preferentially distributing the extracted task to the earliest time point of a preset time interval of a scanning interval;
and if the extracted task is the throughput priority task, randomly distributing the extracted task to each time point of a preset time interval of a scanning interval.
5. The method of claim 1, wherein after the executive returns the execution result to the task center, the method comprises:
and the execution result is normal, the current task is ended, or the current task continues to be executed in the next scheduling period.
6. The method of claim 1, wherein after the executive returns the execution result to the task center, the method further comprises:
and the task center reschedules the current task to execute in the next scheduling period.
7. The method of claim 1, wherein the executive returns execution results to the task center, comprising:
if the execution result is not returned to the task center within the appointed time, the task center reschedules the current task in the next scheduling period to ensure that the current task is executed.
8. The device is characterized by comprising an access party cluster, an execution party cluster, a task registration service, a task center, a task management service and a task callback service;
creating an access party application through an API or a page, and registering different set scenes by the task management service under the access party application;
the method comprises the steps that an access party working node in an access party cluster builds a task of the same type under each registered scene, and the task of the same type built under each scene is registered to a task center through a task registration service, wherein information in the task comprises scene names, service parameters and task execution time;
setting current scanning starting time, wherein the task center starts from the current scanning starting time, scans and inquires all tasks in a plurality of scenes once every interval preset time interval to acquire all tasks with task execution time earlier than any task execution time in the current scanning starting time, and determines a minimum task ID and a maximum task ID in all tasks, wherein the starting time point of the interval preset time interval is the ending time of the current scanning, and the ending time point is the starting time of the next scanning;
Setting a closed interval by using the minimum task ID and the maximum task ID, and segmenting the closed interval according to a preset step length to obtain a task list;
the task center sends the task list to the task callback service, the task callback service traverses the task list, extracts tasks with task execution time earlier than the current scanning starting time, randomly distributes the extracted tasks to each time point of a scanning interval, and generates a delay queue, wherein the scanning interval is a preset time interval of each scanning interval;
the task in the delay queue triggers wake-up according to different time points, and a task callback triggering service selects a corresponding executive working node from the executive cluster to execute the task according to the scene name of the awakened task and the application information of the access party;
and the executive party returns an executive result to the task center.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 7 when executing the computer program.
10. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program which, when executed by a processor, implements the method according to any of claims 1 to 7.
CN202311369840.0A 2023-10-20 2023-10-20 Rectifying scheduling method, device, equipment and medium for mass tasks Pending CN117615014A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311369840.0A CN117615014A (en) 2023-10-20 2023-10-20 Rectifying scheduling method, device, equipment and medium for mass tasks

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311369840.0A CN117615014A (en) 2023-10-20 2023-10-20 Rectifying scheduling method, device, equipment and medium for mass tasks

Publications (1)

Publication Number Publication Date
CN117615014A true CN117615014A (en) 2024-02-27

Family

ID=89955034

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311369840.0A Pending CN117615014A (en) 2023-10-20 2023-10-20 Rectifying scheduling method, device, equipment and medium for mass tasks

Country Status (1)

Country Link
CN (1) CN117615014A (en)

Similar Documents

Publication Publication Date Title
US8856793B2 (en) System, method and program for scheduling computer program jobs
CA2812525C (en) Virtual server control system and program
CN107241380B (en) Method and apparatus for time-based adjusted load balancing
CN113656157B (en) Distributed task scheduling method and device, storage medium and electronic equipment
US20060184944A1 (en) Scheduling batch jobs
Okoshi et al. Real-world product deployment of adaptive push notification scheduling on smartphones
US9621507B2 (en) Control method and apparatus for data display
CN107861799B (en) Task processing method and device based on multi-thread environment
US9935817B2 (en) Managed execution and expiration of agent commands
EP3945420A1 (en) Method and apparatus for data processing, server and storage medium
CN106550021B (en) Push method and device for push message
CN117615014A (en) Rectifying scheduling method, device, equipment and medium for mass tasks
CN108632085A (en) Gray scale user management method, device, platform and storage medium
CN109002381B (en) Process communication monitoring method, electronic device and computer readable storage medium
CN111124652A (en) Resource pre-scheduling method, equipment and storage medium
US20080282247A1 (en) Method and Server for Synchronizing a Plurality of Clients Accessing a Database
CN113988559B (en) Disturbance-free method and device for questionnaire delivery
CN111143066A (en) Event processing method, device, equipment and storage medium
CN115964198A (en) Distributed flexible transaction processing method and device based on long transaction
CN114780177A (en) Transaction execution method and device, storage medium and electronic device
CN113553217A (en) Data recovery method and device, storage medium and computer equipment
US10474498B2 (en) Server scheduling tool
CN110740151A (en) micro-service adjusting method, device, server and computer storage medium
EP2975516B1 (en) Intelligent application back stack management
US20050198643A1 (en) Journaling proxy in activation solution

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination