CN113986511A - Task management method and related device - Google Patents

Task management method and related device Download PDF

Info

Publication number
CN113986511A
CN113986511A CN202111293353.1A CN202111293353A CN113986511A CN 113986511 A CN113986511 A CN 113986511A CN 202111293353 A CN202111293353 A CN 202111293353A CN 113986511 A CN113986511 A CN 113986511A
Authority
CN
China
Prior art keywords
task
queues
target
tasks
queue
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111293353.1A
Other languages
Chinese (zh)
Inventor
刘泽许
李明强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Tupu Network Technology Co ltd
Original Assignee
Guangzhou Tupu Network Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Tupu Network Technology Co ltd filed Critical Guangzhou Tupu Network Technology Co ltd
Priority to CN202111293353.1A priority Critical patent/CN113986511A/en
Publication of CN113986511A publication Critical patent/CN113986511A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/48Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806Task transfer initiation or dispatching
    • G06F9/4843Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083Techniques for rebalancing the load in a distributed system
    • G06F9/5088Techniques for rebalancing the load in a distributed system involving task migration

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer And Data Communications (AREA)

Abstract

In the task management method and the related device, the server obtains the idle condition of each task queue in each task allocation period, and determines the weight of each task queue according to the idle condition of each task queue; then, generating a task distribution plan according to the weight of each task queue; and finally, distributing the tasks to be distributed to the target queues according to the task distribution plan. Because the method can dynamically adjust the task allocation plan according to the idle state of each task queue, the tasks to be processed cached in each task queue can be more balanced.

Description

Task management method and related device
Technical Field
The present application relates to the field of computers, and in particular, to a task management method and a related apparatus.
Background
Under the scene of large flow and big data, the stability of the service needs to be ensured, meanwhile, the scheduling distribution is carried out on the processing tasks, the reasonable utilization of the computing resources is realized, and therefore the task processing efficiency is improved.
However, research finds that some existing scheduling methods do not sense the processing condition of tasks when the tasks are allocated, so that the problem of unreasonable scheduling results exists.
Disclosure of Invention
In order to overcome at least one of the deficiencies in the prior art, the present application provides a task management method and a related device, including:
in a first aspect, the present application provides a task management method applied to a server, where the server is provided with a plurality of task queues, and the method includes:
aiming at the current period, acquiring respective idle states of the plurality of task queues;
determining respective weights of the plurality of task queues according to respective idle conditions of the plurality of task queues;
determining a task distribution plan among the plurality of task queues in a weighted polling mode according to the respective weights of the plurality of task queues;
and distributing the tasks to be distributed to a target queue in the plurality of task queues according to the task distribution plan to serve as the tasks to be processed in the target queue.
In a second aspect, the present application provides a task management device, applied to a server, where the server is provided with a plurality of task queues, and the task management device includes:
the state module is used for acquiring respective idle states of the plurality of task queues according to the current period;
the weighting module is used for determining the respective weights of the plurality of task queues according to the respective idle conditions of the plurality of task queues;
the planning module is used for determining a task distribution plan among the plurality of task queues in a weighted polling mode according to the respective weights of the plurality of task queues;
and the distribution module is used for distributing the tasks to be distributed to the target queues in the plurality of task queues according to the task distribution plan to serve as the tasks to be processed in the target queues.
In a third aspect, the present application provides a server, where the server includes a processor and a memory, where the memory stores a computer program, and the computer program, when executed by the processor, implements the task management method.
In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program, which when executed by a processor, implements the task management method.
Compared with the prior art, the method has the following beneficial effects:
in the task management method and the related device, the server obtains the idle condition of each task queue in each task allocation period, and determines the weight of each task queue according to the idle condition of each task queue; then, generating a task distribution plan according to the weight of each task queue; and finally, distributing the tasks to be distributed to the target queues according to the task distribution plan. Because the method can dynamically adjust the task allocation plan according to the idle state of each task queue, the tasks to be processed cached in each task queue can be more balanced.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
Fig. 1 is a hardware structure of a server according to an embodiment of the present disclosure;
FIG. 2 is a flowchart of a task management method according to an embodiment of the present disclosure;
FIG. 3 is a representation of a minimum plan provided by an embodiment of the present application;
FIG. 4 is a representation of a global schedule provided by an embodiment of the present application;
fig. 5 is a schematic structural diagram of a task management device according to an embodiment of the present application.
Icon: 120-a memory; 130-a processor; 140-a communication unit; 201-state module; 202-a weight module; 203-planning module; 204-allocation module.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
In order to fully utilize the computing resources of the server, the balance needs to be realized as much as possible when tasks are allocated, and therefore, a large number of scheduling and allocating methods are provided. Research finds that tasks are distributed according to task consumption capacity of task consumers in related methods, and a scheduling distribution mode is not properly adjusted according to actual processing progress in a task processing process, so that the problem of overstocked tasks exists. The task consumer in this embodiment may be a service instance running in a process or thread manner in the server, or a service child node managed by the server.
Taking a weighted polling algorithm as an example, in the algorithm, a user configures corresponding weight for each task consumer according to the task consumption capability of each task consumer, wherein the weight of each task consumer is positively correlated with the task consumption capability of the task consumer; and then, after receiving the tasks to be distributed, distributing according to the weight of each task consumer.
For example, assume that there are three task consumers, consumer a, consumer B, and consumer C; and, the fixed weights among the three task consumers are (5, 1, 2) in order, i.e., the sum of the weights is 8. When the server distributes the tasks to be distributed, the server selects the consumers with the largest weight to distribute according to the updated reference weight; and after the distribution, adjusting the reference weight of each task consumer to obtain a new reference weight, wherein the specific process is as follows:
1 st task to be assigned:
the fixed weights (5, 1, 2) are used as the current reference weights. At this time, if the task consumer with the largest reference weight is consumer a, the task to be distributed is distributed to consumer a, and the reference weight of consumer a is subtracted from 8 to obtain a temporary weight (-3, 1, 2) between the three. Since the sum of the weights of the three needs to be maintained at 8, new reference weights (2, 4) are obtained by adding each of the temporary weights to the corresponding fixed weight.
The 2 nd to-be-assigned considers:
since the task consumer with the largest reference weight is the consumer C, the task to be distributed is distributed to the consumer C, and the reference weight of the consumer C is subtracted from 8, so that temporary weights (2, 2 and-4) among the consumer C and the consumer C are obtained. Since the sum of the weights of the three needs to be maintained at 8, each of the temporary weights is added to the corresponding fixed weight to obtain a new reference weight (7, 3, -2).
By analogy, a task allocation schedule as shown in table 1 can be obtained:
TABLE 1
Figure BDA0003335485860000041
Figure BDA0003335485860000051
However, although the consumption ability of the task consumer can reflect the task processing speed of the task consumer to some extent, research finds that the task processing speed is also related to task data of the task itself, so that a certain difference exists in the processing time between tasks, which in turn causes fluctuation of the task backlog, and therefore, the processing condition of the task needs to be sensed and the weight of each task consumer needs to be adjusted.
For example, it is assumed that the task to be assigned is an image recognition task, and the time consumed for image recognition is related to the information of the size, resolution, and the like of the image to be recognized, so that even if two image recognition tasks of the same type require different times due to the difference in picture quality.
In view of this, the present embodiment provides a task management method applied to a server. In the method, a server is provided with a plurality of task queues, and each task queue is used for caching a task to be processed; then, in each task allocation period, the server acquires the idle state of each task queue, and dynamically adjusts a task allocation plan according to the idle state of each queue, wherein the task allocation plan enables more tasks to be allocated to the more idle task queues.
In this embodiment, the task to be allocated is obtained from a task processing request sent by a user terminal. The user terminal may be, but is not limited to, a mobile terminal, a tablet computer, a laptop computer, or a built-in device in a motor vehicle, etc., or any combination thereof. In some embodiments, the mobile terminal may include a smart home device, a wearable device, a smart mobile device, a virtual reality device, an augmented reality device, or the like, or any combination thereof. In some embodiments, the smart home devices may include smart lighting devices, control devices for smart electrical devices, smart monitoring devices, smart televisions, smart cameras, or walkie-talkies, or the like, or any combination thereof. In some embodiments, the wearable device may include a smart bracelet, a smart lace, smart glass, a smart helmet, a smart watch, a smart garment, a smart backpack, a smart accessory, and the like, or any combination thereof. In some embodiments, the smart mobile device may include a smartphone, a Personal Digital Assistant (PDA), a gaming device, a navigation device, or a Point of Sale (POS) device, or the like, or any combination thereof.
The server in the present embodiment may be, but is not limited to, a Web server, an FTP (File Transfer Protocol) server, a data processing server, and the like. In addition, the server may be a single server or a server group. The set of servers can be centralized or distributed (e.g., the servers can be a distributed system). In some embodiments, the server may be local or remote to the user terminal. In some embodiments, the server may be implemented on a cloud platform; by way of example only, the Cloud platform may include a private Cloud, a public Cloud, a hybrid Cloud, a Community Cloud, a distributed Cloud, a cross-Cloud (Inter-Cloud), a Multi-Cloud (Multi-Cloud), and the like, or any combination thereof. In some embodiments, the server may be implemented on an electronic device having one or more components.
The embodiment further provides a schematic structural diagram of the server, as shown in fig. 1, the server includes a memory 120, a processor 130, and a communication unit 140.
The memory 120, processor 130, and communication unit 140 are electrically connected to each other directly or indirectly to enable data transmission or interaction. For example, the components may be electrically connected to each other via one or more communication buses or signal lines.
The Memory 120 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like. The memory 120 is used for storing a program, and the processor 130 executes the program after receiving the execution instruction.
The communication unit 140 is used for transceiving data through a network. The Network may include a wired Network, a Wireless Network, a fiber optic Network, a telecommunications Network, an intranet, the internet, a Local Area Network (LAN), a Wide Area Network (WAN), a Wireless Local Area Network (WLAN), a Metropolitan Area Network (MAN), a Wide Area Network (WAN), a Public Switched Telephone Network (PSTN), a bluetooth Network, a ZigBee Network, or a Near Field Communication (NFC) Network, or the like, or any combination thereof. In some embodiments, the network may include one or more network access points. For example, the network may include wired or wireless network access points, such as base stations and/or network switching nodes, through which one or more components of the service request processing system may connect to the network to exchange data and/or information.
The processor 130 may be an integrated circuit chip having signal processing capabilities, and may include one or more processing cores (e.g., a single-core processor or a multi-core processor). Merely by way of example, the Processor may include a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), an Application Specific Instruction Set Processor (ASIP), a Graphics Processing Unit (GPU), a Physical Processing Unit (PPU), a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), a controller, a microcontroller Unit, a Reduced Instruction Set computer (Reduced Instruction Set computer), a microprocessor, or the like, or any combination thereof.
Based on the above description, the task management method provided in this embodiment is described in detail below with reference to fig. 2. It should be understood that the operations of the flow diagrams may be performed out of order, and steps without logical context may be performed in reverse order or simultaneously. One skilled in the art, under the guidance of this application, may add one or more other operations to, or remove one or more operations from, the flowchart. As shown in fig. 2, the method includes:
s101, aiming at the current period, obtaining respective idle states of a plurality of task queues.
Since the task backlog status of each task queue changes dynamically, in this implementation, the respective idle statuses of the plurality of task queues are periodically obtained, and the weight of each task queue is adjusted in each round period. For example, the server may obtain the respective free status of the plurality of task queues once after each task assignment is completed and then wait for 1 minute.
Of course, the division manner of the task cycle is only an example provided in this embodiment, and those skilled in the art can make appropriate adjustments as needed.
For example, the server may provide a registration subscription service for recording the status of each task queue in the current server, including the task idle status and the number of remaining tasks to be processed. If the task consumer in this embodiment is a service instance, the registration subscription service is further configured to record a function of each service instance and a correspondence between each service instance and the task queue.
Therefore, the server can obtain the idle condition of each task queue through the registration subscription service.
S102, determining respective weights of the plurality of task queues according to respective idle states of the plurality of task queues.
Wherein the weight of each task queue is positively correlated with the idle status of the task queue. In an alternative embodiment, if the plurality of task queues have the same queue length, the server may directly use the free capacity of each task queue as the free status.
Optionally, in other embodiments, if the queue lengths of the plurality of task queues are different, at this time, the server may obtain the remaining number of the to-be-processed tasks in each task queue, and then, calculate, for each task queue, an inverse ratio of the remaining number corresponding to the task queue, as the weight of the task queue.
For example, assuming that there are 3 task queues, the server divides the remaining number of each task queue by the greatest common divisor, and the obtained proportional relation can be expressed as (2:10: 5); then, the respective inverses are used as weights. Therefore, the ratio of the weights of the 5 task queues is (1/2:1/10:1/5), and since the weights in this embodiment need to be integers, the ratio of the weights after scaling is (5:1: 2).
Of course, the server may also sum (2:10:5), and then divide each ratio by the sum result 17, taking the integer in the ratio result as the weight, so that the ratio of the weights of the 3 task queues is (6:1: 3).
And S103, determining a task distribution plan among the plurality of task queues in a weighted polling mode according to the respective weights of the plurality of task queues.
And S104, distributing the tasks to be distributed to a target queue in the plurality of task queues according to the task distribution plan to serve as the tasks to be processed in the target task queues.
As described above for the weighted polling method, the algorithm can obtain a more uniform task allocation plan according to the weight of each task queue; and when the server receives a task to be distributed, distributing the task according to the task distribution plan.
Therefore, according to the embodiment, the server obtains the idle state of each task queue in each task allocation period, and determines the weight of each task queue according to the idle state of each task queue; then, generating a task distribution plan according to the weight of each task queue; and finally, distributing the tasks to be distributed to the target queues according to the task distribution plan. Because the method can dynamically adjust the task allocation plan according to the idle state of each task queue, the tasks to be processed cached in each task queue can be more balanced.
Research shows that before tasks to be distributed are distributed according to a task distribution plan, the task plan needs to be inquired to determine a current target queue, and the distribution efficiency is seriously influenced after the number of the tasks to be distributed reaches a certain scale.
In view of this, the server obtains the planned number of tasks to be allocated in the current period; generating a minimum task table matched with respective weights of a plurality of task queues in a weighted polling mode; and then, generating a global task table for the tasks to be allocated in the planned number according to the minimum task table, wherein the global task table records the allocation sequence when the tasks to be allocated in the planned number are allocated to a plurality of task queues.
And finally, taking the global task table as a task distribution plan among a plurality of task queues.
Illustratively, continuing to take the above 3 task queues as an example, which are queue a, queue B, and queue C, respectively, and the weight ratio between the task queues is (5:1:2), a minimum task table as shown in fig. 3 can be determined according to the weighted polling, where the recording order of the queue identifiers in the minimum task table represents the task allocation order.
If the planned number of tasks to be allocated in the current period is 80, the server may generate 10 minimum task tables, and finally, the 10 minimum task tables are spliced together, so that the global task table shown in fig. 4 may be obtained.
And when the server allocates the tasks to be allocated according to the global task table, determining a target queue from the tasks according to the allocation sequence in the global task table, and removing the identifier of the target queue from the global task table after the allocation is finished. Therefore, the server can quickly determine the target queue of the task to be distributed and distribute the target queue of the task to be distributed through the global task table.
Research also finds that tasks to be allocated often carry task data, and some tasks to be allocated carry more data, so that if the task data of the tasks to be allocated are directly allocated to the task queue, not only a large amount of time is consumed for transferring the task data, but also the storage capacity of the task queue is required to be high.
For example, with the increasing development of artificial intelligence, deep learning, optimization and iteration of model algorithms such as a neural network and the like, the capability of the model algorithms is more and more powerful, which also results in a large number of business scenarios starting to use artificial intelligence technology, and some business scenarios use artificial intelligence services provided by a cloud computing platform. The user terminal sends the task data to the server of the cloud computing platform, and the server of the cloud computing platform feeds back the computing result to the user terminal.
Moreover, sometimes a service needs to be matched with a plurality of service instances based on a neural network, and a server of the cloud computing platform processes a large amount of services at the same time, which means that tasks need to be scheduled and allocated, and service data of the services to be processed directly serve as objects of scheduling and allocation, which can seriously reduce the data transfer efficiency.
In view of this, the server receives a task processing request sent by the user terminal, where the task processing request includes task data of a task to be allocated; and then, caching the task data of the task to be distributed to obtain a cache position.
Further, the server generates task information of the tasks to be distributed, wherein the task information of the tasks to be distributed comprises a cache position; and taking the task information of the tasks to be distributed as a distribution object when distributing the tasks to be distributed.
That is, in this embodiment, service data of a service to be allocated is cached at a specific location, task information for recording the cache location is generated, and the task information is used as an allocation object, so that data stored in a task queue is task information, and the data volume of the task information is much smaller than that of the task data, thereby improving task allocation efficiency.
In addition, the task information further includes preprocessing information of the task to be processed, and therefore, in an optional implementation manner, before the task information of the task to be allocated is generated, the server performs preprocessing on the task data of the task to be allocated to obtain the preprocessing information of the task to be processed.
Then, the server generates task information including a cache address of the task to be allocated and the preprocessing information.
For example, when the task data is a picture, the preprocessing operation may be compression, scaling, format conversion, and the like on the picture. At this time, the obtained preprocessing information may be information including the format, resolution, and the like of the picture.
When the task data is text, the preprocessing operation may be segmentation clipping of the text. At this time, the obtained preprocessing information may include the number of the cut texts and the size of each text segment.
When the task data is a video, the preprocessing operation may be to adjust the bitrate and the resolution of the video, or to perform video capture and other operations on the video. At this time, the obtained preprocessing information may include information of the resolution, format, and number of video shots of the video.
When the task data is audio, the processing operation may be an operation of performing archive format conversion, data format conversion, or the like on the audio. At this time, the obtained preprocessing information may include information such as an audio format of the audio.
Based on the task information, aiming at each task queue, the server acquires the task information of a target task from the task queue; acquiring task data of the target task according to the task information of the target task; and then, feeding back the processing result of the task data of the target task to the user terminal initiating the target task.
In an optional implementation manner, a server provides service instances for a plurality of task queues respectively, and the server inputs task data of a target task into the service instance corresponding to the task queue to obtain a processing result; and feeding back the processing result to the user terminal initiating the target task.
Illustratively, the target task is assumed to be an image recognition task, and the service instance of the target task is an image recognition model. The server obtains a cache address of the image to be identified according to the task information of the target task, and then obtains the image to be identified through the cache address; then, the image to be recognized is input into the image recognition model, and the recognition result output by the image recognition model is obtained.
In addition, the task information of the target task also comprises a constraint condition of the target task, and the server inputs task data of the target task into a service instance corresponding to the task queue to obtain a result to be analyzed; and comparing the result to be analyzed with the constraint condition to obtain a processing result.
For example, the constraint may be a recognition threshold of the image recognition model. If the image recognition model is used for face detection, the user terminal can set a recognition threshold value for the face confidence level according to needs, and the server determines an object larger than the recognition threshold value as a face according to the face confidence level output by the face recognition model.
In this embodiment, the service instances of the plurality of task queues are server instances of the same type, and the task information also carries the task type. Therefore, before the server allocates the task to be allocated to the target queue of the plurality of task queues, it needs to determine whether the task type of the task to be allocated matches with the type of the service instance of the plurality of task queues, and if so, allocate the task to be allocated to the target queue of the plurality of task queues.
Illustratively, the server provides a plurality of functions, namely a face recognition function, an object detection function and an abnormality detection function. The face recognition function corresponds to a plurality of instances of the face recognition model; the target detection function corresponds to a plurality of target detection model instances; the abnormality detection function corresponds to a plurality of instances of abnormality detection as well. And, each instance corresponds to a task queue, that is, if the face recognition function corresponds to an instance of 3 face recognition models, the server provides three corresponding task queues for the 3 instances.
Therefore, the server can distribute the task information to the service instance matched with the task type according to the task type in the task information.
It is worth noting that for each type of service instance, the server can adjust the number of the service instances appropriately according to the backlog condition of the task queue. Taking the plurality of task queues in the embodiment as an example, when the number of tasks to be processed in all the task queues increases after the tasks are allocated for the preset number of cycles, a preset number of service instances are newly added, so that the pressure of the current task queue is relieved.
Research also finds that during the period of receiving and distributing the tasks to be processed, the tasks to be processed in the task queue are gradually consumed at the same time, if the plan number of the tasks to be distributed is too large, the fact that a lot of time is consumed for distributing the tasks is required is meant, and if the plan number of the tasks to be distributed is too small, the fact that the weight of each task queue needs to be adjusted frequently is meant.
Therefore, in order to determine the planned number of tasks to be allocated more reasonably, in this embodiment, the server obtains the remaining number of tasks to be processed in each of the plurality of task queues, and then determines the planned number of tasks to be allocated in the current period according to the remaining number of each of the plurality of task queues.
For example, assuming that the remaining number of each task queue is (6:30:15), and the greatest common divisor between the remaining numbers of each task queue is 3, the planned number of tasks to be allocated is (6+30+15)/3 ═ 17.
Of course, the present embodiment also provides a reference interval for the planned number, and when the ratio of the common divisor between the sum of the remaining number and the remaining number is not located in the reference interval, the reference value is used as the planned number of the tasks to be allocated.
Illustratively, assuming that the reference interval is [30,100], when the calculated number of plans is less than 30, 30 is used as the number of plans, and when the calculated number of plans is greater than 100, 100 is used as the number of plans, so that the number of plans is within a reasonable range.
Research has also found that the time taken to allocate a task in a previous cycle is not only related to the number of mission plans, but also to the number of received tasks to be allocated. That is, if the task to be assigned is not received for a long time, it will also take a long time to assign the task in the task plan number. However, during the period of waiting for the tasks to be allocated, the idle status between the task queues has changed greatly, so in this embodiment, when the scheduled number of tasks to be allocated is not obtained after the preset time duration is exceeded, the server may obtain the remaining number of the tasks to be processed in each of the current plurality of task queues again, and recalculate the scheduled number of the tasks to be allocated.
Research also finds that when there are more backlogs of tasks to be processed in a task queue, if tasks to be allocated are continuously allocated to the queue, a long time is required for processing the tasks. Therefore, in this embodiment, the candidate queues are first screened to determine a plurality of task queues.
In an optional implementation manner, the server obtains the remaining number of the tasks to be processed in each candidate queue, and then determines the candidate queue whose remaining number is smaller than the number threshold as the task queue.
In an optional other implementation manner, the server obtains a sum of the remaining number of the to-be-processed tasks in each candidate queue, then obtains a ratio between the remaining number of the to-be-processed tasks in each candidate queue and the sum of the remaining number, and determines the candidate queue with the ratio smaller than a ratio threshold value as the task queue.
Based on the same inventive concept as the task management method of the present embodiment, the present embodiment also provides a device related to the task management method, including:
the embodiment also provides a task management device which is applied to the server, wherein the server is provided with a plurality of task queues. It should be understood that the task management means comprises at least one functional module which can be stored in the form of software in a memory. As shown in fig. 5, the task management device may include, functionally divided:
the state module 201 is configured to obtain respective idle states of the plurality of task queues according to a current cycle.
In this embodiment, the status module 201 is used to implement step S101 in fig. 2, and for the detailed description of the status module 201, refer to the detailed description of step S101.
The weighting module 202 is configured to determine respective weights of the plurality of task queues according to respective idle statuses of the plurality of task queues.
In this embodiment, the weighting module 202 is configured to implement step S102 in fig. 2, and for a detailed description of the weighting module 202, refer to a detailed description of step S102.
And the planning module 203 is used for determining a task distribution plan among the plurality of task queues in a weighted polling mode according to the respective weights of the plurality of task queues.
In this embodiment, the planning module 203 is configured to implement step S103 in fig. 2, and for a detailed description of the planning module 203, refer to a detailed description of step S103.
The allocating module 204 is configured to allocate the task to be allocated to a target queue of the multiple task queues according to the task allocation plan, so as to serve as the task to be processed in the target queue.
In this embodiment, the allocating module 204 is configured to implement the step S104 in fig. 2, and for the detailed description of the allocating module 204, refer to the detailed description of the step S104.
The embodiment also provides a server, where the server includes a processor and a memory, where the memory stores a computer program, and when the computer program is executed by the processor, the task management method provided in the embodiment is implemented.
The present embodiment also provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the task management method provided in the present embodiment is implemented.
It should be noted that, in the description of the present application, the terms "first", "second", "third", etc. are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only for various embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the present application, and all such changes or substitutions are included in the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. A task management method applied to a server provided with a plurality of task queues, the method comprising:
aiming at the current period, acquiring respective idle states of the plurality of task queues;
determining respective weights of the plurality of task queues according to respective idle conditions of the plurality of task queues;
determining a task distribution plan among the plurality of task queues in a weighted polling mode according to the respective weights of the plurality of task queues;
and distributing the tasks to be distributed to a target queue in the plurality of task queues according to the task distribution plan to serve as the tasks to be processed in the target queue.
2. The task management method according to claim 1, wherein the determining a task allocation plan among the plurality of task queues by a weighted round robin method according to the respective weights of the plurality of task queues comprises:
acquiring the planned number of tasks to be distributed in the current period;
generating a minimum task table matched with the respective weights of the plurality of task queues in a weighted polling mode;
generating a global task table for the tasks to be allocated in the plan number according to the minimum task table, wherein the global task table records an allocation sequence when the tasks to be allocated in the plan number are allocated to the plurality of task queues;
and taking the global task table as a task distribution plan among the plurality of task queues.
3. The method of task management according to claim 1, wherein before allocating tasks to be allocated to the plurality of task queues according to the task allocation plan, the method further comprises:
receiving a task processing request sent by a user terminal, wherein the task processing request comprises task data of the task to be distributed;
caching the task data of the task to be distributed to obtain a cache position;
generating task information of the task to be distributed, wherein the task information of the task to be distributed comprises the cache position;
and taking the task information of the task to be distributed as a distribution object when the task to be distributed is distributed.
4. The task management method of claim 3, further comprising:
acquiring task information of a target task from the task queue aiming at each task queue;
acquiring task data of the target task according to the task information of the target task;
and feeding back a processing result of the task data of the target task to the user terminal initiating the target task.
5. The task management method according to claim 4, wherein the task information further includes preprocessing information of the task to be processed, and before the generating the task information of the task to be allocated, the method further includes:
and preprocessing the task data of the task to be distributed to obtain preprocessing information of the task to be processed.
6. The task management method according to claim 4, wherein the server provides service instances for the plurality of task queues, respectively, and the feeding back the processing result of the task data of the target task to the user terminal that initiated the target task comprises:
inputting the task data of the target task into a service instance corresponding to the task queue to obtain the processing result;
and feeding back the processing result to the user terminal initiating the target task.
7. The task management method according to claim 6, wherein the task information of the target task further includes a constraint condition of the target task, and the inputting task data of the target task into the service instance corresponding to the task queue and obtaining the processing result includes:
inputting the task data of the target task into a service instance corresponding to the task queue to obtain a result to be analyzed;
and comparing the result to be analyzed with the constraint condition to obtain the processing result.
8. A task management apparatus applied to a server provided with a plurality of task queues, the task management apparatus comprising:
the state module is used for acquiring respective idle states of the plurality of task queues according to the current period;
the weighting module is used for determining the respective weights of the plurality of task queues according to the respective idle conditions of the plurality of task queues;
the planning module is used for determining a task distribution plan among the plurality of task queues in a weighted polling mode according to the respective weights of the plurality of task queues;
and the distribution module is used for distributing the tasks to be distributed to the target queues in the plurality of task queues according to the task distribution plan to serve as the tasks to be processed in the target queues.
9. A server, characterized in that the server comprises a processor and a memory, the memory storing a computer program which, when executed by the processor, implements the task management method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which, when executed by a processor, implements the task management method according to any one of claims 1 to 7.
CN202111293353.1A 2021-11-03 2021-11-03 Task management method and related device Pending CN113986511A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111293353.1A CN113986511A (en) 2021-11-03 2021-11-03 Task management method and related device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111293353.1A CN113986511A (en) 2021-11-03 2021-11-03 Task management method and related device

Publications (1)

Publication Number Publication Date
CN113986511A true CN113986511A (en) 2022-01-28

Family

ID=79746082

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111293353.1A Pending CN113986511A (en) 2021-11-03 2021-11-03 Task management method and related device

Country Status (1)

Country Link
CN (1) CN113986511A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118170521A (en) * 2024-04-11 2024-06-11 北京壁仞科技开发有限公司 Task allocation method and task allocation device

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118170521A (en) * 2024-04-11 2024-06-11 北京壁仞科技开发有限公司 Task allocation method and task allocation device

Similar Documents

Publication Publication Date Title
CN107545338B (en) Service data processing method and service data processing system
WO2021147353A1 (en) Order dispatch
CN108667867B (en) Data storage method and device
US20150295970A1 (en) Method and device for augmenting and releasing capacity of computing resources in real-time stream computing system
CN103927229A (en) Scheduling Mapreduce Jobs In A Cluster Of Dynamically Available Servers
CN103873587B (en) A kind of method and device that scheduling is realized based on cloud platform
CN110166507B (en) Multi-resource scheduling method and device
CN110569252B (en) Data processing system and method
Ma et al. Towards revenue-driven multi-user online task offloading in edge computing
Wang et al. Joint server assignment and resource management for edge-based MAR system
CN105808341B (en) A kind of methods, devices and systems of scheduling of resource
CN111506434B (en) Task processing method and device and computer readable storage medium
CN106790332B (en) Resource scheduling method, system and main node
CN114416352A (en) Computing resource allocation method and device, electronic equipment and storage medium
CN113495779A (en) Task scheduling method and device and task execution system
CN111124644B (en) Method, device and system for determining task scheduling resources
Thai et al. Executing bag of distributed tasks on the cloud: Investigating the trade-offs between performance and cost
Sharma et al. A Dynamic optimization algorithm for task scheduling in cloud computing with resource utilization
CN104202305A (en) Transcoding processing method and device, server
CN113986511A (en) Task management method and related device
CN110716809B (en) Method and device for scheduling cloud resources
CN112561301A (en) Work order distribution method, device, equipment and computer readable medium
CN116737370A (en) Multi-resource scheduling method, system, storage medium and terminal
CN115562861A (en) Method and apparatus for data processing for data skew
CN115033355A (en) Task scheduling method, electronic device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination