CN111538572A - Task processing method, device, scheduling server and medium - Google Patents
Task processing method, device, scheduling server and medium Download PDFInfo
- Publication number
- CN111538572A CN111538572A CN202010227325.9A CN202010227325A CN111538572A CN 111538572 A CN111538572 A CN 111538572A CN 202010227325 A CN202010227325 A CN 202010227325A CN 111538572 A CN111538572 A CN 111538572A
- Authority
- CN
- China
- Prior art keywords
- processed
- tasks
- server
- execution server
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
Abstract
The embodiment of the invention discloses a task processing method, a task processing device, a scheduling server and a medium, wherein the method comprises the following steps: the method comprises the steps that N to-be-processed tasks sent by a terminal are received, state information of each execution server in a server cluster used for task processing and task quantity corresponding to each to-be-processed task are determined, a first execution server is screened out from the server cluster according to the state information of each execution server, and first residual load of the first execution server is determined; screening M first tasks to be processed from the N tasks to be processed according to the first residual load, and performing aggregation processing on the M first tasks to be processed to obtain a first task set to be processed; by implementing the method, the execution server can process the tasks to be processed as saturated as possible, the performance of the execution server is fully exerted, and the processing efficiency of the tasks to be processed is improved.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a task processing method, an apparatus, a scheduling server, and a medium.
Background
With the development of network technology and the increasing number of internet users, background servers of applications often receive batch tasks sent by users, processing efficiency is low when a single server is used for processing the batch tasks, and currently, server clusters are mostly used for processing the batch tasks in order to improve processing efficiency of the batch tasks, wherein the server clusters comprise a plurality of execution servers, and each execution server can process the tasks.
In the prior art, when a server cluster is used for processing a task, a scheduling server for task allocation often sends the task to an adaptive execution server immediately after receiving the task, so that the execution server processes the task.
Disclosure of Invention
The embodiment of the invention provides a task processing method, a task processing device, a scheduling server and a medium, which can merge tasks based on the running state of an execution server and then send the merged tasks to the execution server, so that the efficiency of task processing is improved.
In a first aspect, an embodiment of the present invention provides a task processing method, where the method includes:
receiving N to-be-processed tasks sent by a terminal, and determining state information of each execution server in a server cluster for task processing and task quantity corresponding to each to-be-processed task, wherein the state information comprises processing efficiency and residual load quantity, and N is a positive integer;
screening out a first execution server from the server cluster according to the state information of each execution server, and determining a first residual load of the first execution server;
screening M first tasks to be processed from the N tasks to be processed according to the first residual load, wherein the difference value between the sum of the task quantities of the M first tasks to be processed and the first residual load is smaller than a preset threshold value, and M is a positive integer smaller than or equal to N;
performing aggregation processing on the M first tasks to be processed to obtain a first task set to be processed;
and sending the first set of tasks to be processed to the first execution server, so that the first execution server processes the M first tasks to be processed in the first set of tasks to be processed.
In a second aspect, an embodiment of the present invention provides a task processing apparatus, where the apparatus includes:
the receiving module is used for receiving the N tasks to be processed sent by the terminal;
the system comprises a determining module, a processing module and a processing module, wherein the determining module is used for determining state information of each execution server in a server cluster for task processing and task amount corresponding to each task to be processed, the state information comprises processing efficiency and residual load capacity, and N is a positive integer;
the screening module is used for screening out a first execution server from the server cluster according to the state information of each execution server;
the determining module is further configured to determine a first remaining load of the first execution server;
the screening module is further configured to screen M first to-be-processed tasks from the N to-be-processed tasks according to the first remaining load amount, a difference value between a sum of task amounts of the M first to-be-processed tasks and the first remaining load amount is smaller than a preset threshold value, and M is a positive integer smaller than or equal to N;
the processing module is used for carrying out aggregation processing on the M first tasks to be processed to obtain a first task set to be processed;
a sending module, configured to send the first set of tasks to be processed to the first execution server, so that the first execution server processes M first tasks to be processed in the first set of tasks to be processed.
In a third aspect, an embodiment of the present invention provides a scheduling server, including a processor, an input interface, an output interface, and a memory, where the processor, the input interface, the output interface, and the memory are connected to each other, where the memory is used to store a computer program, and the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method according to the first aspect.
In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program includes program instructions, which, when executed by a processor, cause the processor to execute the method of the first aspect.
In the embodiment of the invention, when the scheduling server receives the tasks to be processed, the tasks to be processed which are required to be processed by the execution server are selected by taking the processing efficiency and the residual load of the execution server as reference factors, so that the execution server can process the tasks to be processed as saturated as possible, the performance of the execution server is fully exerted, and the processing efficiency of the server on the tasks to be processed is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic structural diagram of a task processing system according to an embodiment of the present invention;
FIG. 2 is a flow chart illustrating another task processing method according to an embodiment of the present invention;
FIG. 3 is a flow chart illustrating another task processing method according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a task processing device according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a scheduling server according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, a task processing system according to an embodiment of the present invention is provided, where the task processing system includes: the system comprises a scheduling server, at least one terminal connected with the scheduling server and a server cluster connected with the scheduling server, wherein the server cluster comprises at least one execution server. Among others, terminals may include, but are not limited to: the mobile phone, the tablet computer, the computer and the like are used for interacting with the user, for example, receiving a task to be processed uploaded by the user; and for interacting with the scheduling server, such as sending the pending task to the scheduling server to request for task processing services, and so on. The scheduling server is used for receiving the tasks to be processed sent by the terminals and distributing the tasks to be processed to each execution server, and specifically can receive N tasks to be processed sent by the terminals and determine state information of each execution server in a server cluster for processing the tasks and task quantity corresponding to each task to be processed, the scheduling server screens out a first execution server from the server cluster according to the state information of each execution server and determines first residual load quantity of the first execution server, the state information comprises processing efficiency and residual load quantity, the scheduling server screens out M first tasks to be processed from the N tasks to be processed according to the first residual load quantity and aggregates the M first tasks to be processed to obtain a first set of tasks to be processed, and the scheduling server sends the first set of tasks to be processed to the first execution server, so that the first execution server processes the M first tasks to be processed in the first set of tasks to be processed. By the mode, when the to-be-processed tasks are received, the scheduling server selects the to-be-processed tasks needing to be processed by the execution server by taking the processing efficiency and the residual load of the execution server as reference factors, so that the execution server can process the to-be-processed tasks as saturated as possible, the performance of the execution server is fully exerted, and the processing efficiency of the server on the to-be-processed tasks is improved.
Based on the description of the embodiment of the task processing system, an embodiment of the present invention provides a task processing method, which may be applied to the task processing system shown in fig. 1, and may be specifically executed by the scheduling server shown in fig. 1. Referring to fig. 2, the task processing procedure may include the following steps S201 to S207:
s201, a scheduling server receives N to-be-processed tasks sent by a terminal, and determines state information of each execution server in a server cluster for task processing and a task amount corresponding to each to-be-processed task.
In the embodiment of the invention, N to-be-processed tasks can be input by a user through the terminal, the N to-be-processed tasks are sent to the scheduling server after the terminal detects the to-be-processed tasks input by the user, and the scheduling server receives the N to-be-processed tasks sent by the terminal. The N to-be-processed tasks may be sent by the same terminal, or may be sent by different terminals received by the scheduling server at a certain time period.
After the scheduling server receives the N to-be-processed tasks, the state information of each execution server in the server cluster for task processing and the task amount corresponding to each to-be-processed task are determined. The task amount may be a memory size, such as 100 megabytes or 200 megabytes, that the server needs to occupy when processing the task to be processed, and the task amount of the task to be processed may be sent together when the terminal sends the task to be processed, or the server detects the task to be processed and determines the task amount of the task to be processed. The state information includes processing efficiency and remaining load capacity, the processing efficiency is used for representing processing performance of the execution server, and specifically may be, for example, processing 100 million tasks per second, processing 200 million tasks per second, and the like, different execution servers may represent different processing efficiencies due to different hardware devices or different numbers of current processing tasks, and the remaining load capacity may be, specifically, a remaining memory amount of the execution server, such as 500 million, 1000 million, and the like. Specifically, the dispatch server may manage and maintain a remaining load table, which is shown in table one below:
table one: residual load meter
| Execution server | Residual capacity (million) |
| Execution server 1 | 500 |
| Execution server 2 | 300 |
| … | … |
| Execution server N | 200 |
As shown in the first table, the remaining load table records the remaining load of each execution server in the server cluster, and in one implementation, each execution server may report the respective remaining load at regular time (every 2 seconds, every 5 seconds, etc.), the scheduling server determines whether the remaining load of each execution server changes, and if so, updates the first table in real time according to the reported data; in another implementation manner, when each execution server in the server cluster finds that the remaining load of the execution server changes through self-checking, the changed remaining load is reported to the scheduling server, and the scheduling server updates the first table in real time according to the reported data. The scheduling server may obtain the remaining load amount of each execution server from table one.
S202, the scheduling server screens out the first execution server from the server cluster according to the state information of each execution server, and determines a first residual load of the first execution server.
In the embodiment of the invention, after the scheduling server determines the task amount corresponding to each task to be processed and the state information of each execution server, the first execution server is screened out from the server cluster according to the state information of each execution server, and the first residual load of the first execution server is determined.
In one implementation, the specific manner in which the scheduling server screens out the first execution server from the server cluster according to the state information of each execution server may be that the scheduling server determines the processing efficiency in the state information of each execution server in the server cluster, and determines the execution server with the highest processing efficiency in the server cluster as the first execution server. For example, the server cluster includes an execution server 1, an execution server 2, and an execution server 3, where the processing efficiency of the execution server 1 is to process 300 million tasks per second, the processing efficiency of the execution server 2 is to process 200 million tasks per second, and the processing efficiency of the execution server 3 is to process 100 million tasks per second, then the scheduling server determines the execution server 1 as the first execution server.
In an implementation manner, a specific manner in which the scheduling server screens out the first execution server from the server cluster according to the state information of each execution server may also be that the scheduling server determines a remaining load amount in the state information of each execution server in the server cluster, and determines the execution server with the largest remaining load amount in the server cluster as the first execution server. For example, the server cluster includes an execution server 1, an execution server 2, and an execution server 3, where the remaining load capacity of the execution server 1 is 500 megabits, the remaining load capacity of the execution server 2 is 300 megabits, and the remaining load capacity of the execution server 3 is 200 megabits, and the scheduling server determines the execution server 1 as the first execution server.
In an implementation manner, a specific manner in which the scheduling server screens out the first execution server from the server cluster according to the state information of each execution server may also be that the scheduling server determines a first weight corresponding to the processing efficiency of each execution server in the server cluster according to a corresponding relationship between the processing efficiency and the weight; determining a second weight corresponding to the residual load of each execution server in the server cluster according to the corresponding relation between the residual load and the weight; and determining a target weight corresponding to each execution server according to the first weight and the second weight of each execution server, and determining the execution server with the highest target weight as the first execution server, wherein the target weight corresponding to each execution server is the sum of the first weight and the second weight corresponding to each execution server. The correspondence between the processing efficiency and the weight and the correspondence between the remaining load amount and the weight may be preset by a developer. For example, the server cluster includes an execution server 1 and an execution server 2, the processing efficiency of the execution server 1 is 300 million tasks processed per second, the corresponding first weight is 3, the remaining load capacity of the execution server 1 is 500 million, the corresponding second weight is 5, the target weight corresponding to the execution server 1 is 8, the processing efficiency of the execution server 2 is 200 million tasks processed per second, the corresponding first weight is 2, the remaining load capacity of the execution server 1 is 300 million, the corresponding second weight is 3, the target weight corresponding to the execution server 2 is 5, and the scheduling server determines the server 1 as the first execution server.
Further, after the scheduling server determines the first execution server, a first remaining load amount corresponding to the first execution server is determined.
S203, the scheduling server screens M first tasks to be processed from the N tasks to be processed according to the first residual load.
In the embodiment of the invention, after the scheduling server determines the first residual load corresponding to the first execution server, M first tasks to be processed are screened out from the N tasks to be processed according to the first residual load. If the first remaining load is greater than or equal to the sum of the task amounts corresponding to the N tasks to be processed, the scheduling server directly determines the N tasks to be processed as the M first tasks to be processed, and if the first remaining load is less than the sum of the task amounts corresponding to the N tasks to be processed, the scheduling server screens the M first tasks to be processed from the N tasks to be processed according to the first remaining load, wherein a difference between the sum of the task amounts of the M first tasks to be processed and the first remaining load is less than a preset threshold, M is a positive integer less than or equal to N, and a specific calculation manner of the difference between the sum and the first remaining load is that the sum is subtracted from the first remaining load, and the difference needs to be an integer.
In an implementation manner, a specific manner in which the scheduling server screens M first to-be-processed tasks from the N to-be-processed tasks according to the first remaining load may be that the scheduling server randomly combines the N to-be-processed tasks to obtain a plurality of to-be-processed task combinations, determines a sum of task amounts of the to-be-processed tasks included in each of the to-be-processed task combinations, and calculates a difference between the first remaining load and each sum, and if a combination in which the difference is smaller than a preset threshold exists when the difference is a positive number, the scheduling server determines a to-be-processed task combination corresponding to the sum in which the difference between the first to-be-processed task combination is the smallest, and determines each to-be-processed task in the to-be-processed task combination as the M first to-be-processed tasks.
In an implementation manner, a specific manner in which the scheduling server screens M first to-be-processed tasks from the N to-be-processed tasks according to the first remaining load may be that the scheduling server randomly screens M to-be-processed tasks from the N to-be-processed tasks, and if a difference between a sum of task amounts of the M to-be-processed tasks and the first remaining load is smaller than a preset threshold, determines the M to-be-processed tasks as the M first to-be-processed tasks. If the difference value between the sum of the task amounts of the M tasks to be processed and the first remaining load amount is greater than or equal to the preset threshold value, the scheduling server randomly filters the M tasks to be processed again, and calculates the difference value between the sum of the task amounts of the M tasks to be processed and the first remaining load amount again.
In an implementation manner, a specific manner in which the scheduling server screens the M first to-be-processed tasks from the N to-be-processed tasks according to the first remaining load may be that, after the scheduling server obtains the first remaining load, the scheduling server selects the M to-be-processed tasks from the N to-be-processed tasks based on the first remaining load, so that a difference between a sum of task amounts of the M to-be-processed tasks and the first remaining load is minimum, and when the difference between the sum of task amounts of the M to-be-processed tasks and the first remaining load is smaller than a preset threshold, the scheduling server determines the M to-be-processed tasks as the M first to-be-processed tasks. For example, if the first remaining load amount is 500 million, N is 3, M is 2, the task amount of the task 1 to be processed is 200, the task amount of the task 2 to be processed is 400, the task amount of the task 3 to be processed is 200, and the preset threshold value is 200, the scheduling server determines that the sum of the task 1 to be processed and the task 3 to be processed is 400 and the difference between the sum and the first task amount is 100, and determines that the task 1 to be processed and the task 3 to be processed are the first task to be processed.
S204, the scheduling server carries out aggregation processing on the M first tasks to be processed to obtain a first task set to be processed.
In the embodiment of the invention, after the scheduling server screens out the M first tasks to be processed, the M first tasks to be processed are aggregated to obtain a first task set to be processed, that is, the M tasks to be processed are put into the same set.
S205, the scheduling server sends the first set of tasks to be processed to the first execution server, so that the first execution server processes M first tasks to be processed in the first set of tasks to be processed.
In the embodiment of the present invention, after the scheduling server determines the first set of tasks to be processed, the scheduling server may send the first set of tasks to be processed to the first execution server, so that the first execution server processes M first tasks to be processed in the first set of tasks to be processed.
In an implementation manner, the first execution server includes multiple interfaces, processing priorities corresponding to tasks received by each interface are different, and the specific manner in which the scheduling server sends the first set of tasks to be processed to the first execution server may be that the scheduling server obtains a timestamp of each first task to be processed in the first set of tasks to be processed, determines an interface corresponding to each first task to be processed according to the timestamp of each first task to be processed, and establishes a connection between each task to be processed and the interface, and the scheduling server sends each task to be processed in the first set of tasks to be processed to a corresponding interface in the first server based on the connection, so that the first server receives each task to be processed. The timestamp of each first to-be-processed task may specifically include a time when the scheduling server receives each first to-be-processed task, and when the scheduling server receives N to-be-processed tasks, the scheduling server determines the timestamps of the N to-be-processed tasks.
In an implementation manner, the first execution server includes multiple interfaces, the types of tasks received by each interface are different, and the specific manner in which the scheduling server sends the first set of tasks to be processed to the first execution server may be that the scheduling server determines the task type of each first task to be processed in the first set of tasks to be processed, determines the interface corresponding to each first task to be processed according to the task type of each first task to be processed, and establishes a connection between each task to be processed and the interface, and the scheduling server sends each task to be processed in the first set of tasks to be processed to the corresponding interface in the first server based on the connection, so that the first server receives each task to be processed. The task type of the first to-be-processed task may be retrieval type, calculation type, resource acquisition type, and the like.
S206, the scheduling server receives a first processing result which is returned by the first execution server and aims at the first to-be-processed task set.
In the embodiment of the present invention, after all the first tasks to be processed in the first set of tasks to be processed are processed, the first execution server may combine the processing results of all the first tasks to be processed to obtain the first processing result, and return the first processing result to the scheduling server, and the scheduling server sends the first processing result to the corresponding terminal. Or, the first execution server may return the first processing result to the scheduling server immediately after processing any task in the first set of tasks to be processed to obtain the first processing result, and the scheduling server receives the first processing result. Further, the dispatch server may also receive processing results returned by other execution servers in the server cluster.
And S207, the scheduling server sends the first processing result to the terminal.
In the embodiment of the present invention, after receiving the first processing result returned by the first execution server, the scheduling server may send the first processing result to the terminal. Specifically, the scheduling server acquires communication information between the terminal and the scheduling server, wherein the communication information comprises a communication distance and a communication bandwidth; the scheduling server determines the sending time aiming at the first processing result according to the communication information; and sending a first processing result to the terminal when the calculated sending time is obtained, so that the time for receiving the first processing result by the terminal is the same as the preset time. The scheduling server may specifically calculate a transmission duration required for sending the processing result to the terminal based on the communication distance and the communication bandwidth, and send the first processing result to the terminal in consideration of the transmission duration factor, so that the time for the terminal to receive the processing result is the same as expected. In an implementation scenario, the scheduling server needs to return the processing results to different terminals at the same time, and at this time, the scheduling server may obtain the distance and the communication bandwidth between each terminal and the scheduling server, determine the transmission time for the processing results based on the distance and the communication bandwidth, preferentially transmit the processing results to the terminals with longer distance, and delay transmission of the processing results to the terminals with shorter distance. So as to realize the simultaneous reception of the processing results by the respective terminals. The above scenario may specifically be a red packet robbing scenario, and specifically, the scheduling server may detect a distance between each terminal and itself and a communication bandwidth, and determine a sending time for a red packet based on the above factors, so that each terminal has the same probability to robbe the red packet, and it is unlikely that some terminals cannot always rob the red packet due to an excessively long distance or a slow network speed.
In the embodiment of the invention, a scheduling server receives N to-be-processed tasks sent by a terminal, determines the state information of each execution server in a server cluster for task processing and the task amount corresponding to each to-be-processed task, screens out a first execution server from the server cluster according to the state information of each execution server, and determines the first residual load of the first execution server; the scheduling server screens M first tasks to be processed from the N tasks to be processed according to the first residual load, performs aggregation processing on the M first tasks to be processed to obtain a first task set to be processed, and sends the first task set to the first execution server, so that the first execution server processes the M first tasks to be processed in the first task set to be processed. By implementing the method, when the to-be-processed task is received, the to-be-processed task required to be processed by the execution server is selected by taking the processing efficiency and the residual load capacity of the execution server as reference factors, so that the execution server can process the to-be-processed task as saturated as possible, the performance of the execution server is fully exerted, the processing efficiency of the server on the to-be-processed task is improved, in addition, the sending time is determined based on the transmission duration of the data in the sending process, the terminal can receive the data at the expected time, and the stability of data receiving is improved.
Based on the description of the embodiments of the task processing system and method, the embodiment of the present invention provides another task processing method, which may be applied to the task processing system shown in fig. 1, and may be specifically executed by the scheduling server shown in fig. 1. Referring to fig. 3, the task processing procedure may include the following steps S301 to S306:
s301, the scheduling server receives the N to-be-processed tasks sent by the terminal, and determines the state information of each execution server in the server cluster for task processing and the task amount corresponding to each to-be-processed task.
In the embodiment of the invention, N to-be-processed tasks can be input by a user through the terminal, the N to-be-processed tasks are sent to the scheduling server after the terminal detects the to-be-processed tasks input by the user, and the scheduling server receives the N to-be-processed tasks sent by the terminal. The N to-be-processed tasks may be sent by the same terminal, or may be sent by different terminals received by the scheduling server at a certain time period. After the scheduling server receives the N to-be-processed tasks, the state information of each execution server in the server cluster for task processing and the task amount corresponding to each to-be-processed task are determined.
S302, the scheduling server screens out the first execution server from the server cluster according to the state information of each execution server, and determines a first residual load of the first execution server.
In the embodiment of the invention, after the scheduling server determines the task amount corresponding to each task to be processed and the state information of each execution server, the first execution server is screened out from the server cluster according to the state information of each execution server, and the first residual load of the first execution server is determined.
S303, the scheduling server screens M first tasks to be processed from the N tasks to be processed according to the first residual load, and performs aggregation processing on the M first tasks to be processed to obtain a first task set to be processed.
In the embodiment of the invention, after the scheduling server determines the first residual load corresponding to the first execution server, M first tasks to be processed are screened out from the N tasks to be processed according to the first residual load. If the first remaining load is greater than or equal to the sum of the task quantities corresponding to the N tasks to be processed, the scheduling server directly determines the N tasks to be processed as the M first tasks to be processed, and if the first remaining load is less than the sum of the task quantities corresponding to the N tasks to be processed, the scheduling server screens the M first tasks to be processed from the N tasks to be processed according to the first remaining load, wherein a difference between the sum of the task quantities of the M first tasks to be processed and the first remaining load is less than a preset threshold.
Further, when M is less than N, the scheduling server performs step S304.
S304, the scheduling server screens out a second execution server from the server cluster according to the state information of each execution server, and determines a second residual load of the second execution server.
In one implementation, the specific manner in which the scheduling server screens out the second execution server from the server cluster according to the state information of each execution server may be that the scheduling server determines the processing efficiency in the state information of each execution server in the server cluster, and determines the execution server with the second highest processing efficiency in the server cluster as the first execution server.
In an implementation manner, a specific manner in which the scheduling server screens out the second execution server from the server cluster according to the state information of each execution server may also be that the scheduling server determines a remaining load amount in the state information of each execution server in the server cluster, and determines an execution server with the next highest remaining load amount in the server cluster as the second execution server.
In an implementation manner, a specific manner in which the scheduling server screens out the second execution server from the server cluster according to the state information of each execution server may also be that the scheduling server determines a first weight corresponding to the processing efficiency of each execution server in the server cluster according to a corresponding relationship between the processing efficiency and the weight; determining a second weight corresponding to the residual load of each execution server in the server cluster according to the corresponding relation between the residual load and the weight; and determining a target weight corresponding to each execution server according to the first weight and the second weight of each execution server, and determining the execution server with the highest target weight as the second execution server, wherein each execution server target weight is the sum of the first weight and the second weight of each execution server. Further, the scheduling server determines a second remaining load amount of the second execution server.
S305, the scheduling server screens K second to-be-processed tasks from the N-M to-be-processed tasks except the M first to-be-processed tasks according to the second residual load.
In the embodiment of the invention, K is a positive integer smaller than N-M, after a terminal determines a second remaining load of a second execution server, whether the sum of the task amounts corresponding to the remaining N-M to-be-processed tasks is smaller than the second remaining load is detected, if yes, a scheduling server directly determines the N-M to-be-processed tasks as K second to-be-processed tasks, and performs aggregation processing on the K second to-be-processed tasks to obtain a second to-be-processed task set; if the sum of the task amounts corresponding to the remaining N-M to-be-processed tasks is greater than the second remaining load amount, the scheduling server screens out K second to-be-processed tasks in the same manner as that in step S203, and performs aggregation processing on the K second to-be-processed tasks to obtain a second to-be-processed task set.
Further, the scheduling server may continue to perform set division on the remaining N-M-K to-be-processed tasks according to the remaining load of each execution server in the server cluster, and determine the execution server corresponding to each set, so that each to-be-processed task in the N to-be-processed tasks corresponds to a corresponding execution server for processing.
S306, the scheduling server sends the first set of tasks to be processed to the first execution server, and sends the second set of tasks to be processed to the second execution server.
In this embodiment of the present invention, after the scheduling server receives the first set of tasks to be processed, the scheduling server may send the first set of tasks to be processed to the first execution server, so that the first execution server processes M first tasks to be processed in the first set of tasks to be processed, and send the second set of tasks to be processed to the second execution server, so that the second execution server processes K second tasks to be processed in the second set of tasks to be processed, further, the scheduling server may receive processing results returned by the first execution server and the second execution server, and return the received processing results to the terminal.
In the embodiment of the invention, when the scheduling server receives the tasks to be processed, the tasks to be processed which are required to be processed by the execution server are selected by taking the processing efficiency and the residual load of the execution server as reference factors, so that the server can process the tasks to be processed as saturated as possible, the performance of the execution server is fully exerted, and the processing efficiency of the server on the tasks to be processed is improved.
Based on the description of the task processing method embodiment, the embodiment of the invention also discloses a task processing device. The task processing device may be a computer program (including program code) running in the scheduling server, or may be a physical device included in the scheduling server. The task processing device may perform the methods shown in fig. 2 and 3. Referring to fig. 4, the task processing device 40 includes: a receiving module 401, a determining module 402, a screening module 403, a processing module 404, and a sending module 405.
A receiving module 401, configured to receive N to-be-processed tasks sent by a terminal;
a determining module 402, configured to determine state information of each execution server in a server cluster for task processing and a task amount corresponding to each task to be processed, where the state information includes processing efficiency and a remaining load amount, and N is a positive integer;
a screening module 403, configured to screen out a first execution server from the server cluster according to the status information of each execution server;
the determining module 402 is further configured to determine a first remaining load amount of the first execution server;
the screening module 403 is further configured to screen M first to-be-processed tasks from the N to-be-processed tasks according to the first remaining load amount, where a difference between a sum of task amounts of the M first to-be-processed tasks and the first remaining load amount is smaller than a preset threshold, and M is a positive integer smaller than or equal to N;
a processing module 404, configured to perform aggregation processing on the M first tasks to be processed to obtain a first task set to be processed;
a sending module 405, configured to send the first set of tasks to be processed to the first execution server, so that the first execution server processes M first tasks to be processed in the first set of tasks to be processed.
In an implementation manner, the determining module 402 is specifically configured to:
determining the processing efficiency in the state information of each execution server in the server cluster, and determining the execution server with the highest processing efficiency in the server cluster as a first execution server; alternatively, the first and second electrodes may be,
and determining the residual load amount in the state information of each execution server in the server cluster, and determining the execution server with the maximum residual load amount in the server cluster as a first execution server.
In an implementation manner, the determining module 402 is specifically configured to:
determining a first weight corresponding to the processing efficiency of each execution server in the server cluster according to the corresponding relation between the processing efficiency and the weight;
determining a second weight corresponding to the residual load of each execution server in the server cluster according to the corresponding relation between the residual load and the weight;
and determining a target weight corresponding to each execution server according to the first weight and the second weight corresponding to each execution server, and determining the execution server with the highest target weight as the first execution server, wherein the target weight corresponding to each execution server is the sum of the first weight and the second weight corresponding to each execution server.
In one implementation, the screening module 403 is specifically configured to:
screening a second execution server from the server cluster according to the state information of each execution server, and determining a second residual load of the second execution server;
screening K second tasks to be processed from the N-M tasks to be processed except the M first tasks to be processed according to the second residual load, wherein K is a positive integer smaller than or equal to N-M;
performing aggregation processing on the K second tasks to be processed to obtain a second task set to be processed;
and sending the second set of tasks to be processed to the second execution server, so that the second execution server processes the K second tasks to be processed in the second set of tasks to be processed.
In an implementation manner, the sending module 405 is specifically configured to:
acquiring a timestamp of each first task to be processed in the first task set to be processed;
determining an interface corresponding to each first task to be processed according to the timestamp of each first task to be processed, and establishing connection between each task to be processed and the interface, wherein the interface is an interface used for receiving a task request in the first server, and the processing priorities corresponding to the tasks received by different interfaces in the first server are different;
and sending each task to be processed in the first set of tasks to be processed to a corresponding interface in the first server based on the connection, so that the first server receives each task to be processed.
In an implementation manner, the receiving module 401 is further configured to receive a first processing result for the first set of tasks to be processed, which is returned by the first execution server;
the sending module 405 is further configured to send the first processing result to the terminal.
In an implementation manner, the sending module 405 is specifically configured to:
acquiring communication information between the terminal and the terminal, wherein the communication information comprises communication distance and communication bandwidth;
determining a transmission time for the first processing result according to the communication information;
and sending a first processing result to the terminal at the sending time, so that the time for receiving the first processing result by the terminal is the same as the preset time.
In the embodiment of the present invention, a receiving module 401 receives N to-be-processed tasks sent by a terminal, a determining module 402 determines state information of each execution server in a server cluster for task processing and a task amount corresponding to each to-be-processed task, a screening module 403 screens out a first execution server from the server cluster according to the state information of each execution server, and the determining module 402 determines a first remaining load amount of the first execution server; the screening module 403 screens M first to-be-processed tasks from the N to-be-processed tasks according to the first remaining load, the processing module 404 performs aggregation processing on the M first to-be-processed tasks to obtain a first to-be-processed task set, and the sending module 405 sends the first to-be-processed task set to the first execution server. By implementing the method, when the to-be-processed task is received, the to-be-processed task required to be processed by the execution server is selected by taking the processing efficiency and the residual load capacity of the execution server as reference factors, so that the execution server can process the to-be-processed task as saturated as possible, the performance of the execution server is fully exerted, and the processing efficiency of the server on the to-be-processed task is improved.
Fig. 5 is a schematic structural diagram of a dispatch server according to an embodiment of the present invention. As shown in fig. 5, the scheduling server includes: at least one processor 501, an input device 503, an output device 504, a memory 505, at least one communication bus 502. Wherein a communication bus 502 is used to enable connective communication between these components. The memory 505 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 505 may alternatively be at least one memory device located remotely from the processor 501. Wherein the processor 501 may be combined with the apparatus described in fig. 4, the memory 505 stores a set of program codes, and the processor 501, the input device 503, and the output device 504 call the program codes stored in the memory 505 to perform the following operations:
the input device 503 is configured to receive N to-be-processed tasks sent by a terminal, and determine state information of each execution server in a server cluster for task processing and a task amount corresponding to each to-be-processed task, where the state information includes processing efficiency and a remaining load amount, and N is a positive integer;
the processor 501 is configured to screen out a first execution server from the server cluster according to the state information of each execution server, and determine a first remaining load of the first execution server;
the processor 501 is configured to screen M first to-be-processed tasks from the N to-be-processed tasks according to the first remaining load amount, where a difference between a sum of task amounts of the M first to-be-processed tasks and the first remaining load amount is smaller than a preset threshold, and M is a positive integer smaller than or equal to N;
the processor 501 is configured to perform aggregation processing on the M first tasks to be processed to obtain a first task set to be processed;
the output device 504 is configured to send the first set of tasks to be processed to the first execution server, so that the first execution server processes M first tasks to be processed in the first set of tasks to be processed.
In one implementation, the processor 501 is specifically configured to:
determining the processing efficiency in the state information of each execution server in the server cluster, and determining the execution server with the highest processing efficiency in the server cluster as a first execution server; alternatively, the first and second electrodes may be,
and determining the residual load amount in the state information of each execution server in the server cluster, and determining the execution server with the maximum residual load amount in the server cluster as a first execution server.
In one implementation, the processor 501 is specifically configured to:
determining a first weight corresponding to the processing efficiency of each execution server in the server cluster according to the corresponding relation between the processing efficiency and the weight;
determining a second weight corresponding to the residual load of each execution server in the server cluster according to the corresponding relation between the residual load and the weight;
and determining a target weight corresponding to each execution server according to the first weight and the second weight corresponding to each execution server, and determining the execution server with the highest target weight as the first execution server, wherein the target weight corresponding to each execution server is the sum of the first weight and the second weight corresponding to each execution server.
In one implementation, the processor 501 is specifically configured to:
screening a second execution server from the server cluster according to the state information of each execution server, and determining a second residual load of the second execution server;
screening K second tasks to be processed from the N-M tasks to be processed except the M first tasks to be processed according to the second residual load, wherein K is a positive integer smaller than or equal to N-M;
performing aggregation processing on the K second tasks to be processed to obtain a second task set to be processed;
and sending the second set of tasks to be processed to the second execution server, so that the second execution server processes the K second tasks to be processed in the second set of tasks to be processed.
In one implementation, the processor 501 is specifically configured to:
acquiring a timestamp of each first task to be processed in the first task set to be processed;
determining an interface corresponding to each first task to be processed according to the timestamp of each first task to be processed, and establishing connection between each task to be processed and the interface, wherein the interface is an interface used for receiving a task request in the first server, and the processing priorities corresponding to the tasks received by different interfaces in the first server are different;
an output device 504, configured to send, based on the connection, each to-be-processed task in the first to-be-processed task set to a corresponding interface in the first server, so that the first server receives each to-be-processed task.
In one implementation, the input device 503 is configured to receive a first processing result for the first set of tasks to be processed, which is returned by the first execution server;
an output device 504, configured to send the first processing result to the terminal.
In one implementation, the processor 501 is specifically configured to:
acquiring communication information between the terminal and the terminal, wherein the communication information comprises communication distance and communication bandwidth;
determining a transmission time for the first processing result according to the communication information;
the output device 504 is configured to send the first processing result to the terminal at the sending time, so that the time when the terminal receives the first processing result is the same as a preset time.
In the embodiment of the present invention, an input device 503 receives N to-be-processed tasks sent by a terminal, a processor 501 determines state information of each execution server in a server cluster for task processing and a task amount corresponding to each to-be-processed task, the processor 501 screens out a first execution server from the server cluster according to the state information of each execution server, and the processor 501 determines a first remaining load amount of the first execution server; and according to the first remaining load, M first tasks to be processed are screened from the N tasks to be processed, the processor 501 performs aggregation processing on the M first tasks to be processed to obtain a first set of tasks to be processed, and the output device 504 sends the first set of tasks to be processed to the first execution server. By implementing the method, when the to-be-processed task is received, the to-be-processed task required to be processed by the execution server is selected by taking the processing efficiency and the residual load capacity of the execution server as reference factors, so that the execution server can process the to-be-processed task as saturated as possible, the performance of the execution server is fully exerted, and the processing efficiency of the server on the to-be-processed task is improved.
The module in the embodiment of the present invention may be implemented by a general-purpose integrated circuit, such as a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC).
It should be understood that, in the embodiment of the present invention, the Processor 501 may be a Central Processing Unit (CPU), and the Processor may also be other general processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The bus 502 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like, and the bus 502 may be divided into a site bus, a data bus, a control bus, or the like, where fig. 5 is merely a thick line for ease of illustration, but does not indicate only one bus or one type of bus.
It will be understood by those skilled in the art that all or part of the processes of the methods of the above embodiments may be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The computer-readable storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above disclosure is only for the purpose of illustrating the preferred embodiments of the present invention, and it is therefore to be understood that the invention is not limited by the scope of the appended claims.
Claims (10)
1. A method for processing a task, the method comprising:
receiving N to-be-processed tasks sent by a terminal, and determining state information of each execution server in a server cluster for task processing and task quantity corresponding to each to-be-processed task, wherein the state information comprises processing efficiency and residual load quantity, and N is a positive integer;
screening out a first execution server from the server cluster according to the state information of each execution server, and determining a first residual load of the first execution server;
screening M first tasks to be processed from the N tasks to be processed according to the first residual load, wherein the difference value between the sum of the task quantities of the M first tasks to be processed and the first residual load is smaller than a preset threshold value, and M is a positive integer smaller than or equal to N;
performing aggregation processing on the M first tasks to be processed to obtain a first task set to be processed;
and sending the first set of tasks to be processed to the first execution server, so that the first execution server processes the M first tasks to be processed in the first set of tasks to be processed.
2. The method of claim 1, wherein the screening out the first execution server from the server cluster according to the status information of each execution server comprises:
determining the processing efficiency in the state information of each execution server in the server cluster, and determining the execution server with the highest processing efficiency in the server cluster as a first execution server; alternatively, the first and second electrodes may be,
and determining the residual load amount in the state information of each execution server in the server cluster, and determining the execution server with the maximum residual load amount in the server cluster as a first execution server.
3. The method of claim 1, wherein the screening out the first execution server from the server cluster according to the status information of each execution server comprises:
determining a first weight corresponding to the processing efficiency of each execution server in the server cluster according to the corresponding relation between the processing efficiency and the weight;
determining a second weight corresponding to the residual load of each execution server in the server cluster according to the corresponding relation between the residual load and the weight;
and determining a target weight corresponding to each execution server according to the first weight and the second weight corresponding to each execution server, and determining the execution server with the highest target weight as the first execution server, wherein the target weight corresponding to each execution server is the sum of the first weight and the second weight corresponding to each execution server.
4. The method according to claim 1, wherein after the aggregating the M first tasks to be processed to obtain a first set of tasks to be processed, the method further comprises:
screening a second execution server from the server cluster according to the state information of each execution server, and determining a second residual load of the second execution server;
screening K second tasks to be processed from the N-M tasks to be processed except the M first tasks to be processed according to the second residual load, wherein K is a positive integer smaller than or equal to N-M;
performing aggregation processing on the K second tasks to be processed to obtain a second task set to be processed;
and sending the second set of tasks to be processed to the second execution server, so that the second execution server processes the K second tasks to be processed in the second set of tasks to be processed.
5. The method of claim 1, wherein sending the first set of tasks to be processed to the first execution server comprises:
acquiring a timestamp of each first task to be processed in the first task set to be processed;
determining an interface corresponding to each first task to be processed according to the timestamp of each first task to be processed, and establishing connection between each task to be processed and the interface, wherein the interface is an interface used for receiving a task request in the first server, and the processing priorities corresponding to the tasks received by different interfaces in the first server are different;
and sending each task to be processed in the first set of tasks to be processed to a corresponding interface in the first server based on the connection, so that the first server receives each task to be processed.
6. The method of claim 1, wherein after sending the first set of tasks to be processed to the first execution server, the method further comprises:
receiving a first processing result for a first set of tasks to be processed returned by the first execution server;
and sending the first processing result to the terminal.
7. The method of claim 6, wherein sending the first processing result to the terminal comprises:
acquiring communication information between the terminal and the terminal, wherein the communication information comprises communication distance and communication bandwidth;
determining a transmission time for the first processing result according to the communication information;
and sending a first processing result to the terminal at the sending time, so that the time for receiving the first processing result by the terminal is the same as the preset time.
8. A task processing apparatus, characterized in that the apparatus comprises:
the receiving module is used for receiving the N tasks to be processed sent by the terminal;
the system comprises a determining module, a processing module and a processing module, wherein the determining module is used for determining state information of each execution server in a server cluster for task processing and task amount corresponding to each task to be processed, the state information comprises processing efficiency and residual load capacity, and N is a positive integer;
the screening module is used for screening out a first execution server from the server cluster according to the state information of each execution server;
the determining module is further configured to determine a first remaining load of the first execution server;
the screening module is further configured to screen M first to-be-processed tasks from the N to-be-processed tasks according to the first remaining load amount, a difference value between a sum of task amounts of the M first to-be-processed tasks and the first remaining load amount is smaller than a preset threshold value, and M is a positive integer smaller than or equal to N;
the processing module is used for carrying out aggregation processing on the M first tasks to be processed to obtain a first task set to be processed;
a sending module, configured to send the first set of tasks to be processed to the first execution server, so that the first execution server processes M first tasks to be processed in the first set of tasks to be processed.
9. A dispatch server comprising a processor, an input interface, an output interface, and a memory, the processor, the input interface, the output interface, and the memory being interconnected, wherein the memory is configured to store a computer program comprising program instructions, the processor being configured to invoke the program instructions to perform the method of any of claims 1-7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program comprising program instructions that, when executed by a processor, cause the processor to carry out the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010227325.9A CN111538572A (en) | 2020-03-26 | 2020-03-26 | Task processing method, device, scheduling server and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010227325.9A CN111538572A (en) | 2020-03-26 | 2020-03-26 | Task processing method, device, scheduling server and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111538572A true CN111538572A (en) | 2020-08-14 |
Family
ID=71978436
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010227325.9A Pending CN111538572A (en) | 2020-03-26 | 2020-03-26 | Task processing method, device, scheduling server and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111538572A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108170525A (en) * | 2016-12-07 | 2018-06-15 | 晨星半导体股份有限公司 | The device and method of the task load configuration of dynamic adjustment multi-core processor |
| CN113220458A (en) * | 2021-05-26 | 2021-08-06 | 西安热工研究院有限公司 | Real-time scheduling method and device for realizing load balance of database connection pool |
| CN113515358A (en) * | 2021-04-30 | 2021-10-19 | 北京奇艺世纪科技有限公司 | Task scheduling method and device, electronic equipment and storage medium |
-
2020
- 2020-03-26 CN CN202010227325.9A patent/CN111538572A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108170525A (en) * | 2016-12-07 | 2018-06-15 | 晨星半导体股份有限公司 | The device and method of the task load configuration of dynamic adjustment multi-core processor |
| CN113515358A (en) * | 2021-04-30 | 2021-10-19 | 北京奇艺世纪科技有限公司 | Task scheduling method and device, electronic equipment and storage medium |
| CN113515358B (en) * | 2021-04-30 | 2024-04-12 | 北京奇艺世纪科技有限公司 | Task scheduling method and device, electronic equipment and storage medium |
| CN113220458A (en) * | 2021-05-26 | 2021-08-06 | 西安热工研究院有限公司 | Real-time scheduling method and device for realizing load balance of database connection pool |
| CN113220458B (en) * | 2021-05-26 | 2022-11-01 | 西安热工研究院有限公司 | Real-time scheduling method and device for realizing load balance of database connection pool |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109104336B (en) | Service request processing method and device, computer equipment and storage medium | |
| CN110858843B (en) | Service request processing method and device and computer readable storage medium | |
| CN111538572A (en) | Task processing method, device, scheduling server and medium | |
| CN107819797B (en) | Access request processing method and device | |
| CN109144700B (en) | Method and device for determining timeout duration, server and data processing method | |
| CN105516347A (en) | Method and device for load balance allocation of streaming media server | |
| CN111045810B (en) | Task scheduling processing method and device | |
| CN110933136A (en) | Service node selection method, device, equipment and readable storage medium | |
| US10044833B2 (en) | Data processing method and apparatus used for terminal application | |
| CN112596985B (en) | IT asset detection method, device, equipment and medium | |
| CN112363834A (en) | Task processing method, device, terminal and storage medium | |
| CN117032987A (en) | Distributed task scheduling method, system, equipment and computer readable medium | |
| CN111371536B (en) | Control instruction sending method and device | |
| CN108830724B (en) | Resource data packet processing method and terminal equipment | |
| CN109308219B (en) | Task processing method and device and distributed computer system | |
| CN116226071A (en) | Data statistics method, device, equipment and storage medium | |
| CN113835905B (en) | Message queue load balancing method and device, electronic equipment and medium | |
| CN116382892A (en) | Load balancing method and device based on multi-cloud fusion and cloud service | |
| CN113747506A (en) | Resource scheduling method, device and network system | |
| CN110602187A (en) | Terminal sharing method, device and equipment | |
| CN113439252A (en) | Terminal control method, terminal control device, storage medium and electronic equipment | |
| CN113453043B (en) | Video scheduling optimization method and device, computer equipment and storage medium | |
| CN113162990B (en) | Message sending method, device, equipment and storage medium | |
| CN116701019A (en) | Message queue scheduling method, device, equipment and storage medium | |
| CN115604242A (en) | Node scheduling method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |