CN111813554A - Task scheduling processing method and device, electronic equipment and storage medium - Google Patents
Task scheduling processing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111813554A CN111813554A CN202010693109.3A CN202010693109A CN111813554A CN 111813554 A CN111813554 A CN 111813554A CN 202010693109 A CN202010693109 A CN 202010693109A CN 111813554 A CN111813554 A CN 111813554A
- Authority
- CN
- China
- Prior art keywords
- task
- processed
- priority
- configuration file
- queue
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 22
- 238000012545 processing Methods 0.000 claims abstract description 49
- 238000000034 method Methods 0.000 claims abstract description 24
- 238000004590 computer program Methods 0.000 claims description 14
- 239000003999 initiator Substances 0.000 claims description 7
- 230000011218 segmentation Effects 0.000 claims description 4
- 230000008569 process Effects 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5018—Thread allocation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5021—Priority
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application discloses a task scheduling processing method, a task scheduling processing device, an electronic device and a computer readable storage medium, wherein the method comprises the following steps: receiving a task to be processed and acquiring a preset priority configuration file; determining the priority corresponding to the task to be processed according to the preset priority configuration file, and judging whether the priority is greater than a target priority; if the priority is higher than the target priority, segmenting the tasks to be processed to obtain a plurality of subtasks to be processed; and dispatching each to-be-processed subtask to a plurality of task queues respectively, so as to utilize the thread corresponding to each task queue to perform concurrent processing, and summarizing to obtain the processing result of the to-be-processed task. After receiving the task to be processed, the priority configuration file capable of being flexibly configured is obtained to determine the priority corresponding to the task to be processed, and the task with the high priority higher than the target priority is split and then is concurrently processed, so that the average service processing time delay can be effectively reduced, and the overall performance of the system is improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for task scheduling processing, an electronic device, and a computer-readable storage medium.
Background
The performance is an important index for testing the overall performance of the storage system, and common performance improvement means mainly comprise: hardware upgrading and software logic optimization, wherein the hardware upgrading is a means with the fastest effect and the lowest risk, such as replacing a CPU with stronger performance and expanding memory configuration with higher specification; the software logic tuning mainly focuses on the bottleneck analysis and tuning strategy of related software functional components.
The traditional task scheduling method can comprise a scheduling mode of allocating CPU execution to tasks according to the order of submitting the tasks, and a scheduling mode of time slice rotation and the like for preventing a certain task from monopolizing the CPU. Therefore, how to provide a task scheduling manner for improving the system performance is a problem to be solved by those skilled in the art.
Disclosure of Invention
The application aims to provide a task scheduling processing method and device, an electronic device and a computer readable storage medium, which can effectively reduce the average service processing time delay and improve the overall performance of a system.
In order to achieve the above object, the present application provides a task scheduling processing method, including:
receiving a task to be processed and acquiring a preset priority configuration file;
determining the priority corresponding to the task to be processed according to the preset priority configuration file, and judging whether the priority is greater than a target priority;
if the priority is greater than the target priority, segmenting the to-be-processed task to obtain a plurality of to-be-processed subtasks;
and distributing each to-be-processed subtask to a plurality of task queues respectively, so as to perform concurrent processing by using threads corresponding to each task queue, and summarizing to obtain a processing result of the to-be-processed task.
Optionally, the determining, according to the preset priority configuration file, the priority corresponding to the task to be processed includes:
acquiring a task type included by the task to be processed and/or a user identity corresponding to an initiator of the task to be processed;
and determining the priority corresponding to the task to be processed based on the task type and/or the user identity according to the preset priority configuration file.
Optionally, the segmenting the to-be-processed task to obtain a plurality of to-be-processed subtasks includes:
analyzing the task to be processed, and determining the front-back dependency relationship between the task type and the data included in the task to be processed;
and according to the front-back dependency relationship and the task type, dividing the task to be processed into a plurality of independent sub-tasks to be processed.
Optionally, the method further includes:
receiving an updating instruction aiming at the preset priority configuration file;
and modifying, adding or deleting the priority configuration information in the preset priority configuration file according to the updating instruction.
Optionally, a one-to-one binding relationship exists between the thread and the task queue, and a one-to-one binding relationship exists between the task queue and the CPU core.
Optionally, the dispatching each to-be-processed subtask to a plurality of task queues respectively, so as to perform concurrent processing by using threads corresponding to each task queue, including:
determining a total number of the sub-tasks to be processed;
acquiring the current task number in all task queues, and arranging the tasks in a sequence from low to high to obtain a queue list;
sequentially selecting a corresponding number of target queues in the queue list according to the total number and the arrangement sequence;
and respectively dispatching each to-be-processed subtask to the target queue so that the to-be-processed subtask is processed by the thread corresponding to the target queue.
In order to achieve the above object, the present application provides a task scheduling processing apparatus, including:
the task receiving module is used for receiving the task to be processed and acquiring a preset priority configuration file;
the priority judging module is used for determining the priority corresponding to the task to be processed according to the preset priority configuration file and judging whether the priority is greater than a target priority or not;
the task segmentation module is used for segmenting the to-be-processed task to obtain a plurality of to-be-processed subtasks if the priority is greater than the target priority;
and the concurrent processing module is used for respectively dispatching each to-be-processed subtask to a plurality of task queues so as to perform concurrent processing by utilizing the thread corresponding to each task queue, and summarizing to obtain the processing result of the to-be-processed task.
Optionally, the priority determining module includes:
the acquiring unit is used for acquiring the task type included by the task to be processed and/or the user identity corresponding to the initiator of the task to be processed;
and the determining unit is used for determining the priority corresponding to the task to be processed based on the task type and/or the user identity according to the preset priority configuration file.
To achieve the above object, the present application provides an electronic device including:
a memory for storing a computer program;
a processor for implementing the steps of any of the task scheduling processing methods disclosed in the foregoing when executing the computer program.
To achieve the above object, the present application provides a computer-readable storage medium, on which a computer program is stored, the computer program, when being executed by a processor, implementing the steps of any one of the task scheduling processing methods disclosed in the foregoing.
According to the above scheme, the task scheduling processing method provided by the application includes: receiving a task to be processed and acquiring a preset priority configuration file; determining the priority corresponding to the task to be processed according to the preset priority configuration file, and judging whether the priority is greater than a target priority; if the priority is greater than the target priority, segmenting the to-be-processed task to obtain a plurality of to-be-processed subtasks; and distributing each to-be-processed subtask to a plurality of task queues respectively, so as to perform concurrent processing by using threads corresponding to each task queue, and summarizing to obtain a processing result of the to-be-processed task. According to the method and the device, after the task to be processed is received, the flexibly configurable priority configuration file is obtained to determine the priority corresponding to the task to be processed, and the task with the high priority higher than the target priority is split and then is concurrently processed, so that the average service processing time delay can be effectively reduced, and the overall performance of the system is improved.
The application also discloses a task scheduling processing device, an electronic device and a computer readable storage medium, which can also achieve the technical effects.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of a task scheduling processing method disclosed in an embodiment of the present application;
FIG. 2 is a diagram illustrating a binding relationship among threads, task queues, and CPU cores, disclosed in an embodiment of the present application;
fig. 3 is a flowchart of a task scheduling method disclosed in an embodiment of the present application;
fig. 4 is a structural diagram of a task scheduling processing apparatus disclosed in an embodiment of the present application;
fig. 5 is a block diagram of an electronic device disclosed in an embodiment of the present application;
fig. 6 is a block diagram of another electronic device disclosed in the embodiments of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The embodiment of the application discloses a task scheduling processing method, which can effectively reduce the average processing time delay of services and improve the overall performance of a system.
Referring to fig. 1, a task scheduling processing method disclosed in the embodiment of the present application includes:
s101: receiving a task to be processed and acquiring a preset priority configuration file;
in the embodiment of the application, a task to be processed is received first, where the task to be processed may include a service request such as a data IO request. Specifically, the embodiment may obtain a preset priority configuration file, where the preset priority configuration file may be a configuration file created in advance for describing task priorities, and a mapping relationship between task identification information and corresponding priorities may be saved in the file in advance, and the mapping relationship may be used as a threshold for subsequently determining high and low priorities, that is, a target priority. The task identification information may include, but is not limited to, a task type, a task name, a task ID number, and the like. That is, the priority configuration may not only configure the correspondence between the task types and the priorities of the general scenarios, but also configure the corresponding priorities for the specific tasks according to the task names or ID numbers.
It can be understood that, in the embodiment of the present application, the time point for acquiring the priority profile may be: when the system is started and initialized, loading a pre-established priority configuration file; or a preset priority configuration file can be acquired after the task to be processed is received for the first time; and in the running process of the system, a preset priority configuration file can be imported through a network download or file transmission interface.
In a preferred implementation manner, the preset priority configuration file in the embodiment of the present application is a flexibly configurable file. For example, an update instruction for a preset priority profile may be received; and modifying, adding or deleting the priority configuration information in the preset priority configuration file according to the updating instruction. Specifically, in order to implement more flexible configuration of tasks, the storage system may perform priority configuration on different tasks by using a configuration file in an XML format, so that different tasks may be processed differently according to the priority configuration during the operation of the system. When the configuration of the newly added task type or the priority is updated, only the configuration file needs to be modified, the code logic does not need to be modified, and the flexibility of service scheduling is improved.
S102: determining the priority corresponding to the task to be processed according to the preset priority configuration file, and judging whether the priority is greater than a target priority;
in this step, the priority corresponding to the task to be processed can be searched according to the preset priority configuration file, and whether the priority is greater than the target priority is judged.
In a specific implementation, the determining, according to the preset priority configuration file, the priority corresponding to the task to be processed may specifically include: acquiring a task type included by a task to be processed and/or a user identity corresponding to an initiator of the task to be processed; and determining the priority corresponding to the task to be processed based on the task type and/or the user identity according to the preset priority configuration file. As can be seen from the above, when determining the priority corresponding to the task to be processed, the embodiment of the present application searches for the corresponding priority according to the task type corresponding to the current task to be processed and/or the user identity corresponding to the initiator of the task to be processed, that is, the priority configuration file may configure not only the corresponding relationship between the task type and the priority, but also the corresponding relationship between the user identity and the priority, for example, configure a higher priority for the administrator identity than for the common user.
It should be noted that, if only the corresponding relationship between the task type and the priority is configured in the preset priority configuration file, or only the corresponding relationship between the user identity and the priority is configured, the corresponding relationship of the task to be processed can be directly determined according to a single type of information. If the preset priority configuration file configures the two corresponding relationships at the same time, in a specific implementation manner, the higher priority of the two may be selected to be determined as the final priority query result. In another specific embodiment, the final priority can be calculated by a weighted average algorithm according to the two priority results.
As a specific implementation manner, when determining the priority corresponding to the task to be processed, the embodiment of the present application may first search for the corresponding priority in the configuration file according to the task type corresponding to the current task to be processed, and if there is no corresponding priority, may further search according to the task name or the task ID number, so as to meet the priority determination of the common task and the special task.
S103: if the priority is greater than the target priority, segmenting the to-be-processed task to obtain a plurality of to-be-processed subtasks;
if the priority of the current task to be processed is greater than the target priority, the task to be processed can be divided into a plurality of subtasks to be processed. Specifically, in the process of segmenting the task to be processed to obtain a plurality of subtasks to be processed, the task to be processed may be first analyzed to determine a front-back dependency relationship between a task type and data included in the task to be processed; and dividing the task to be processed into a plurality of independent sub tasks to be processed according to the front-back dependency relationship and the task type. That is, the step splits the task to be processed into a plurality of independent steps without data dependency and without coupling, so that subsequent concurrent processing can be realized.
S104: and distributing each to-be-processed subtask to a plurality of task queues respectively, so as to perform concurrent processing by using threads corresponding to each task queue, and summarizing to obtain a processing result of the to-be-processed task.
It can be understood that, after the task segmentation is completed, each to-be-processed subtask can be respectively dispatched to a plurality of task queues, a thread corresponding to each task queue can read the subtask from the queue for concurrent processing, and finally, a processing result corresponding to the whole to-be-processed task is obtained by summarizing the processing results of each thread.
As a preferred implementation, referring to fig. 2, in the embodiment of the present application, there is a one-to-one binding relationship between a thread and a task queue, and there is a one-to-one binding relationship between a task queue and a CPU core. That is, the system operation architecture of the present embodiment adopts a multithreading architecture, which reduces the performance consumption of the inter-thread synchronization policy while improving the concurrent processing service capability of the system. A one-to-one binding relationship exists between the threads and the task queues, namely, each thread only processes tasks in the binding queues, so that the decoupling of the task queues among the threads is realized. Meanwhile, one-to-one binding relationship exists between the task queue and the CPU kernel, namely, the true concurrent pipeline processing can be realized by combining the CPU affinity binding technology, so that the overhead caused by continuously switching the CPU kernel in the task processing process can be avoided. Preferably, the task queue in the present application may adopt a lock-free queue, so that processes of locking and unlocking the queue may be avoided, and resource consumption may be reduced.
It should be noted that, for the to-be-processed tasks with priorities smaller than the target priority, the embodiments of the present application may perform serial processing in a conventional manner, that is, do not need to split the to-be-processed tasks, so as to avoid unnecessary resource overhead.
According to the above scheme, the task scheduling processing method provided by the application includes: receiving a task to be processed and acquiring a preset priority configuration file; determining the priority corresponding to the task to be processed according to the preset priority configuration file, and judging whether the priority is greater than a target priority; if the priority is greater than the target priority, segmenting the to-be-processed task to obtain a plurality of to-be-processed subtasks; and distributing each to-be-processed subtask to a plurality of task queues respectively, so as to perform concurrent processing by using threads corresponding to each task queue, and summarizing to obtain a processing result of the to-be-processed task. According to the method and the device, after the task to be processed is received, the flexibly configurable priority configuration file is obtained to determine the priority corresponding to the task to be processed, and the task with the high priority higher than the target priority is split and then is concurrently processed, so that the average service processing time delay can be effectively reduced, and the overall performance of the system is improved.
The embodiment of the application discloses another task scheduling processing method, and compared with the previous embodiment, the embodiment further explains and optimizes the technical scheme. Referring to fig. 3, specifically:
s201: receiving a task to be processed and acquiring a preset priority configuration file;
s202: determining the priority corresponding to the task to be processed according to the preset priority configuration file, and judging whether the priority is greater than a target priority;
s203: if the priority is greater than the target priority, segmenting the to-be-processed task to obtain a plurality of to-be-processed subtasks;
s204: determining a total number of the sub-tasks to be processed;
s205: acquiring the current task number in all task queues, and arranging the tasks in a sequence from low to high to obtain a queue list;
s206: sequentially selecting a corresponding number of target queues in the queue list according to the total number and the arrangement sequence;
s207: and respectively dispatching each to-be-processed subtask to the target queue so that the to-be-processed subtask is processed by the thread corresponding to the target queue.
In the embodiment of the application, after the task to be processed is split into a plurality of subtasks, the subtasks can be distributed in a load balancing manner. Specifically, the total number of the to-be-processed subtasks may be determined first, the number of to-be-processed tasks in all task queues is obtained, and the queue lists are obtained by arranging in order from low to high. And then, according to the total number of the subtasks to be processed, selecting a corresponding number of target queues according to the list sequence so that the threads corresponding to the target queues process the subtasks.
It should be noted that, when the target queue is determined, the queue in the list that is earlier in order may be directly selected as the target queue according to the corresponding proportion of the total number of the to-be-processed subtasks, for example, if the to-be-processed subtasks are 18, and the proportion is 2, the first 9 queues in the list are selected as the target queues, and 2 subtasks are distributed to each target queue. Preferably, in the embodiment of the present application, a different number of subtasks may be distributed to each target queue by combining the current task amount of each queue, that is, a queue with a smaller load pressure distributes a larger number of subtasks, and a queue with a larger load pressure distributes a smaller number of subtasks or does not distribute tasks.
In the following, a task scheduling processing device provided in an embodiment of the present application is introduced, and a task scheduling processing device described below and a task scheduling processing method described above may be referred to each other.
Referring to fig. 4, a task scheduling processing apparatus provided in an embodiment of the present application specifically includes:
the task receiving module 301 is configured to receive a task to be processed and obtain a preset priority configuration file;
a priority determining module 302, configured to determine, according to the preset priority configuration file, a priority corresponding to the task to be processed, and determine whether the priority is greater than a target priority;
the task segmentation module 303 is configured to segment the to-be-processed task to obtain a plurality of to-be-processed subtasks if the priority is greater than the target priority;
the concurrent processing module 304 is configured to respectively dispatch each to-be-processed subtask to multiple task queues, so as to perform concurrent processing by using threads corresponding to each task queue, and obtain a processing result of the to-be-processed task by summarizing.
For the specific implementation process of the modules 301 to 304, reference may be made to the corresponding content disclosed in the foregoing embodiments, and details are not repeated here.
On the basis of the foregoing embodiment, as a preferred implementation manner, the priority determining module 302 may specifically include:
the acquiring unit is used for acquiring the task type included by the task to be processed and/or the user identity corresponding to the initiator of the task to be processed;
and the determining unit is used for determining the priority corresponding to the task to be processed based on the task type and/or the user identity according to the preset priority configuration file.
The present application further provides an electronic device, and as shown in fig. 5, an electronic device provided in an embodiment of the present application includes:
a memory 100 for storing a computer program;
the processor 200, when executing the computer program, may implement the steps provided by the above embodiments.
Specifically, the memory 100 includes a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer-readable instructions, and the internal memory provides an environment for the operating system and the computer-readable instructions in the non-volatile storage medium to run. The processor 200 may be a Central Processing Unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chip in some embodiments, and provides computing and controlling capability for the electronic device, and when executing the computer program stored in the memory 100, the steps of the task scheduling processing method disclosed in any of the foregoing embodiments may be implemented.
On the basis of the above embodiment, as a preferred implementation, referring to fig. 6, the electronic device further includes:
and an input interface 300 connected to the processor 200, for acquiring computer programs, parameters and instructions imported from the outside, and storing the computer programs, parameters and instructions into the memory 100 under the control of the processor 200. The input interface 300 may be connected to an input device for receiving parameters or instructions manually input by a user. The input device may be a touch layer covered on a display screen, or a button, a track ball or a touch pad arranged on a terminal shell, or a keyboard, a touch pad or a mouse, etc.
And a display unit 400 connected to the processor 200 for displaying data processed by the processor 200 and for displaying a visualized user interface. The display unit 400 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch panel, or the like.
And a network port 500 connected to the processor 200 for performing communication connection with each external terminal device. The communication technology adopted by the communication connection can be a wired communication technology or a wireless communication technology, such as a mobile high definition link (MHL) technology, a Universal Serial Bus (USB), a High Definition Multimedia Interface (HDMI), a wireless fidelity (WiFi), a bluetooth communication technology, a low power consumption bluetooth communication technology, an ieee802.11 s-based communication technology, and the like.
While FIG. 6 shows only an electronic device having the assembly 100 and 500, those skilled in the art will appreciate that the configuration shown in FIG. 6 is not intended to be limiting of electronic devices and may include fewer or more components than those shown, or some components may be combined, or a different arrangement of components.
The present application also provides a computer-readable storage medium, which may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk. The storage medium has a computer program stored thereon, and the computer program realizes the steps of the task scheduling processing method disclosed in any one of the foregoing embodiments when executed by the processor.
After receiving the task to be processed, the priority configuration file capable of being flexibly configured is obtained to determine the priority corresponding to the task to be processed, and the task with the high priority higher than the target priority is split and then is concurrently processed, so that the average service processing time delay can be effectively reduced, and the overall performance of the system is improved. Meanwhile, the queues and the threads have a one-to-one binding mapping relation, and decoupling of the task queues among the threads is achieved. When the scheduling strategy is updated and the task priority needs to be modified, the priority configuration file can be directly modified without modifying the code logic, so that the flexibility of service scheduling is further improved.
The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description. It should be noted that, for those skilled in the art, it is possible to make several improvements and modifications to the present application without departing from the principle of the present application, and such improvements and modifications also fall within the scope of the claims of the present application.
It is further noted that, in the present specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Claims (10)
1. A task scheduling processing method is characterized by comprising the following steps:
receiving a task to be processed and acquiring a preset priority configuration file;
determining the priority corresponding to the task to be processed according to the preset priority configuration file, and judging whether the priority is greater than a target priority;
if the priority is greater than the target priority, segmenting the to-be-processed task to obtain a plurality of to-be-processed subtasks;
and distributing each to-be-processed subtask to a plurality of task queues respectively, so as to perform concurrent processing by using threads corresponding to each task queue, and summarizing to obtain a processing result of the to-be-processed task.
2. The method according to claim 1, wherein the determining the priority corresponding to the task to be processed according to the preset priority profile includes:
acquiring a task type included by the task to be processed and/or a user identity corresponding to an initiator of the task to be processed;
and determining the priority corresponding to the task to be processed based on the task type and/or the user identity according to the preset priority configuration file.
3. The task scheduling processing method according to claim 1, wherein the splitting the to-be-processed task to obtain a plurality of to-be-processed subtasks includes:
analyzing the task to be processed, and determining the front-back dependency relationship between the task type and the data included in the task to be processed;
and according to the front-back dependency relationship and the task type, dividing the task to be processed into a plurality of independent sub-tasks to be processed.
4. The task scheduling processing method according to claim 1, further comprising:
receiving an updating instruction aiming at the preset priority configuration file;
and modifying, adding or deleting the priority configuration information in the preset priority configuration file according to the updating instruction.
5. The method according to any one of claims 1 to 4, wherein a one-to-one binding relationship exists between the thread and the task queue, and a one-to-one binding relationship exists between the task queue and a CPU core.
6. The task scheduling processing method according to claim 5, wherein the dispatching each of the to-be-processed subtasks to a plurality of task queues respectively, so as to perform concurrent processing by using threads corresponding to each task queue, includes:
determining a total number of the sub-tasks to be processed;
acquiring the current task number in all task queues, and arranging the tasks in a sequence from low to high to obtain a queue list;
sequentially selecting a corresponding number of target queues in the queue list according to the total number and the arrangement sequence;
and respectively dispatching each to-be-processed subtask to the target queue so that the to-be-processed subtask is processed by the thread corresponding to the target queue.
7. A task scheduling processing apparatus, comprising:
the task receiving module is used for receiving the task to be processed and acquiring a preset priority configuration file;
the priority judging module is used for determining the priority corresponding to the task to be processed according to the preset priority configuration file and judging whether the priority is greater than a target priority or not;
the task segmentation module is used for segmenting the to-be-processed task to obtain a plurality of to-be-processed subtasks if the priority is greater than the target priority;
and the concurrent processing module is used for respectively dispatching each to-be-processed subtask to a plurality of task queues so as to perform concurrent processing by utilizing the thread corresponding to each task queue, and summarizing to obtain the processing result of the to-be-processed task.
8. The task scheduling processing apparatus according to claim 7, wherein the priority determining module includes:
the acquiring unit is used for acquiring the task type included by the task to be processed and/or the user identity corresponding to the initiator of the task to be processed;
and the determining unit is used for determining the priority corresponding to the task to be processed based on the task type and/or the user identity according to the preset priority configuration file.
9. An electronic device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the task scheduling processing method according to any one of claims 1 to 6 when executing the computer program.
10. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the task scheduling processing method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010693109.3A CN111813554A (en) | 2020-07-17 | 2020-07-17 | Task scheduling processing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010693109.3A CN111813554A (en) | 2020-07-17 | 2020-07-17 | Task scheduling processing method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111813554A true CN111813554A (en) | 2020-10-23 |
Family
ID=72866074
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010693109.3A Withdrawn CN111813554A (en) | 2020-07-17 | 2020-07-17 | Task scheduling processing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111813554A (en) |
Cited By (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112307046A (en) * | 2020-11-26 | 2021-02-02 | 北京金堤征信服务有限公司 | Data acquisition method and device, computer readable storage medium and electronic equipment |
| CN112363812A (en) * | 2020-11-17 | 2021-02-12 | 浪潮云信息技术股份公司 | Database connection queue management method based on task classification and storage medium |
| CN112445614A (en) * | 2020-11-03 | 2021-03-05 | 华帝股份有限公司 | Thread data storage management method, computer equipment and storage medium |
| CN112463339A (en) * | 2020-12-11 | 2021-03-09 | 北京浪潮数据技术有限公司 | Multitask scheduling method, system, equipment and storage medium |
| CN112612615A (en) * | 2020-12-28 | 2021-04-06 | 中孚安全技术有限公司 | Data processing method and system based on multithreading memory allocation and context scheduling |
| CN112685156A (en) * | 2020-12-28 | 2021-04-20 | 北京五八信息技术有限公司 | Task execution method and device, electronic equipment and computer readable medium |
| CN112751856A (en) * | 2020-12-30 | 2021-05-04 | 湖南快乐阳光互动娱乐传媒有限公司 | Media processing system capable of arranging flow |
| CN112783630A (en) * | 2021-02-01 | 2021-05-11 | 佛山市顺德区美的洗涤电器制造有限公司 | Task scheduling method, intelligent gas stove and computer readable storage medium |
| CN113114757A (en) * | 2021-04-09 | 2021-07-13 | 中国工商银行股份有限公司 | File transmission method, device and equipment |
| CN113190358A (en) * | 2021-05-25 | 2021-07-30 | 曙光信息产业(北京)有限公司 | Job distribution method and device, electronic equipment and readable storage medium |
| CN113296915A (en) * | 2021-06-18 | 2021-08-24 | 瀚云科技有限公司 | Task generation method and system based on industrial internet platform |
| CN113327053A (en) * | 2021-06-21 | 2021-08-31 | 中国农业银行股份有限公司 | Task processing method and device |
| CN113419833A (en) * | 2021-06-24 | 2021-09-21 | 中国信息通信研究院 | Method and device for quantum cloud computing platform task scheduling and quantum cloud computing platform task scheduling server |
| CN113703941A (en) * | 2021-08-30 | 2021-11-26 | 竞技世界（北京）网络技术有限公司 | Task scheduling method and system and electronic equipment |
| CN113806093A (en) * | 2021-09-22 | 2021-12-17 | 土巴兔集团股份有限公司 | Distribution calculation method of content tag calculation task and related device |
| CN114168314A (en) * | 2021-10-27 | 2022-03-11 | 厦门国际银行股份有限公司 | Multithreading concurrent data index batch processing method and device and storage medium |
| CN114363411A (en) * | 2021-12-28 | 2022-04-15 | 奇安信科技集团股份有限公司 | Data transmission method, device, system and computer storage medium |
| CN114880101A (en) * | 2022-07-01 | 2022-08-09 | 成都登临科技有限公司 | AI treater, electronic part and electronic equipment |
| CN114979411A (en) * | 2021-05-06 | 2022-08-30 | 中移互联网有限公司 | Distributed image processing method, device, equipment and system |
| CN115658325A (en) * | 2022-11-18 | 2023-01-31 | 北京市大数据中心 | Data processing method, data processing device, multi-core processor, electronic device, and medium |
| CN115941803A (en) * | 2022-12-12 | 2023-04-07 | 中国联合网络通信集团有限公司 | Task processing method and device, electronic equipment and storage medium |
| CN116107724A (en) * | 2023-04-04 | 2023-05-12 | 山东浪潮科学研究院有限公司 | AI (advanced technology attachment) acceleration core scheduling management method, device, equipment and storage medium |
| CN116109110A (en) * | 2023-04-11 | 2023-05-12 | 华能信息技术有限公司 | Task scheduling method for service center |
| CN116339956A (en) * | 2023-05-29 | 2023-06-27 | 天翼云科技有限公司 | Distribution method and device of configuration tasks, electronic equipment and storage medium |
| CN116670647A (en) * | 2020-12-30 | 2023-08-29 | 华为技术有限公司 | Task scheduling method and device |
| WO2023201947A1 (en) * | 2022-04-19 | 2023-10-26 | Zhejiang Dahua Technology Co., Ltd. | Methods, systems, and storage media for task dispatch |
| WO2023231937A1 (en) * | 2022-05-30 | 2023-12-07 | 华为技术有限公司 | Scheduling apparatus and method, and related device |
| CN117193966A (en) * | 2023-08-10 | 2023-12-08 | 博智安全科技股份有限公司 | Distributed asset task scheduling method |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130227582A1 (en) * | 2007-03-07 | 2013-08-29 | International Business Machines Corporation | Prediction Based Priority Scheduling |
| CN106802826A (en) * | 2016-12-23 | 2017-06-06 | 中国银联股份有限公司 | A kind of method for processing business and device based on thread pool |
| CN107315629A (en) * | 2017-06-14 | 2017-11-03 | 北京小米移动软件有限公司 | Task processing method, device and storage medium |
| CN110196775A (en) * | 2019-05-30 | 2019-09-03 | 苏州浪潮智能科技有限公司 | A kind of calculating task processing method, device, equipment and readable storage medium storing program for executing |
| US20190294469A1 (en) * | 2019-06-13 | 2019-09-26 | Intel Corporation | Techniques to dynamically partition tasks |
| CN110489223A (en) * | 2019-08-26 | 2019-11-22 | 北京邮电大学 | Method for scheduling task, device and electronic equipment in a kind of isomeric group |
| CN111367630A (en) * | 2019-07-12 | 2020-07-03 | 北京关键科技股份有限公司 | Multi-user multi-priority distributed cooperative processing method based on cloud computing |
-
2020
- 2020-07-17 CN CN202010693109.3A patent/CN111813554A/en not_active Withdrawn
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130227582A1 (en) * | 2007-03-07 | 2013-08-29 | International Business Machines Corporation | Prediction Based Priority Scheduling |
| CN106802826A (en) * | 2016-12-23 | 2017-06-06 | 中国银联股份有限公司 | A kind of method for processing business and device based on thread pool |
| CN107315629A (en) * | 2017-06-14 | 2017-11-03 | 北京小米移动软件有限公司 | Task processing method, device and storage medium |
| CN110196775A (en) * | 2019-05-30 | 2019-09-03 | 苏州浪潮智能科技有限公司 | A kind of calculating task processing method, device, equipment and readable storage medium storing program for executing |
| US20190294469A1 (en) * | 2019-06-13 | 2019-09-26 | Intel Corporation | Techniques to dynamically partition tasks |
| CN111367630A (en) * | 2019-07-12 | 2020-07-03 | 北京关键科技股份有限公司 | Multi-user multi-priority distributed cooperative processing method based on cloud computing |
| CN110489223A (en) * | 2019-08-26 | 2019-11-22 | 北京邮电大学 | Method for scheduling task, device and electronic equipment in a kind of isomeric group |
Cited By (37)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112445614A (en) * | 2020-11-03 | 2021-03-05 | 华帝股份有限公司 | Thread data storage management method, computer equipment and storage medium |
| CN112363812A (en) * | 2020-11-17 | 2021-02-12 | 浪潮云信息技术股份公司 | Database connection queue management method based on task classification and storage medium |
| CN112307046A (en) * | 2020-11-26 | 2021-02-02 | 北京金堤征信服务有限公司 | Data acquisition method and device, computer readable storage medium and electronic equipment |
| CN112463339A (en) * | 2020-12-11 | 2021-03-09 | 北京浪潮数据技术有限公司 | Multitask scheduling method, system, equipment and storage medium |
| CN112612615A (en) * | 2020-12-28 | 2021-04-06 | 中孚安全技术有限公司 | Data processing method and system based on multithreading memory allocation and context scheduling |
| CN112685156A (en) * | 2020-12-28 | 2021-04-20 | 北京五八信息技术有限公司 | Task execution method and device, electronic equipment and computer readable medium |
| CN112612615B (en) * | 2020-12-28 | 2022-12-06 | 中孚安全技术有限公司 | Data processing method and system based on multithreading memory allocation and context scheduling |
| CN112751856A (en) * | 2020-12-30 | 2021-05-04 | 湖南快乐阳光互动娱乐传媒有限公司 | Media processing system capable of arranging flow |
| CN116670647A (en) * | 2020-12-30 | 2023-08-29 | 华为技术有限公司 | Task scheduling method and device |
| CN112783630A (en) * | 2021-02-01 | 2021-05-11 | 佛山市顺德区美的洗涤电器制造有限公司 | Task scheduling method, intelligent gas stove and computer readable storage medium |
| CN113114757B (en) * | 2021-04-09 | 2022-09-30 | 中国工商银行股份有限公司 | File transmission method, device and equipment |
| CN113114757A (en) * | 2021-04-09 | 2021-07-13 | 中国工商银行股份有限公司 | File transmission method, device and equipment |
| CN114979411B (en) * | 2021-05-06 | 2023-07-04 | 中移互联网有限公司 | Distributed image processing method, device, equipment and system |
| CN114979411A (en) * | 2021-05-06 | 2022-08-30 | 中移互联网有限公司 | Distributed image processing method, device, equipment and system |
| CN113190358A (en) * | 2021-05-25 | 2021-07-30 | 曙光信息产业(北京)有限公司 | Job distribution method and device, electronic equipment and readable storage medium |
| CN113296915A (en) * | 2021-06-18 | 2021-08-24 | 瀚云科技有限公司 | Task generation method and system based on industrial internet platform |
| CN113296915B (en) * | 2021-06-18 | 2023-07-18 | 瀚云科技有限公司 | Task generation method and system based on industrial Internet platform |
| CN113327053A (en) * | 2021-06-21 | 2021-08-31 | 中国农业银行股份有限公司 | Task processing method and device |
| CN113419833B (en) * | 2021-06-24 | 2023-12-29 | 中国信息通信研究院 | Method and device for task scheduling of quantum cloud computing platform and task scheduling server of quantum cloud computing platform |
| CN113419833A (en) * | 2021-06-24 | 2021-09-21 | 中国信息通信研究院 | Method and device for quantum cloud computing platform task scheduling and quantum cloud computing platform task scheduling server |
| CN113703941A (en) * | 2021-08-30 | 2021-11-26 | 竞技世界（北京）网络技术有限公司 | Task scheduling method and system and electronic equipment |
| CN113806093A (en) * | 2021-09-22 | 2021-12-17 | 土巴兔集团股份有限公司 | Distribution calculation method of content tag calculation task and related device |
| CN114168314A (en) * | 2021-10-27 | 2022-03-11 | 厦门国际银行股份有限公司 | Multithreading concurrent data index batch processing method and device and storage medium |
| CN114363411A (en) * | 2021-12-28 | 2022-04-15 | 奇安信科技集团股份有限公司 | Data transmission method, device, system and computer storage medium |
| CN114363411B (en) * | 2021-12-28 | 2023-11-17 | 奇安信科技集团股份有限公司 | Data transmission method, device, system and computer storage medium |
| WO2023201947A1 (en) * | 2022-04-19 | 2023-10-26 | Zhejiang Dahua Technology Co., Ltd. | Methods, systems, and storage media for task dispatch |
| WO2023231937A1 (en) * | 2022-05-30 | 2023-12-07 | 华为技术有限公司 | Scheduling apparatus and method, and related device |
| CN114880101B (en) * | 2022-07-01 | 2022-09-30 | 成都登临科技有限公司 | AI treater, electronic part and electronic equipment |
| CN114880101A (en) * | 2022-07-01 | 2022-08-09 | 成都登临科技有限公司 | AI treater, electronic part and electronic equipment |
| CN115658325A (en) * | 2022-11-18 | 2023-01-31 | 北京市大数据中心 | Data processing method, data processing device, multi-core processor, electronic device, and medium |
| CN115658325B (en) * | 2022-11-18 | 2024-01-23 | 北京市大数据中心 | Data processing method, device, multi-core processor, electronic equipment and medium |
| CN115941803A (en) * | 2022-12-12 | 2023-04-07 | 中国联合网络通信集团有限公司 | Task processing method and device, electronic equipment and storage medium |
| CN116107724A (en) * | 2023-04-04 | 2023-05-12 | 山东浪潮科学研究院有限公司 | AI (advanced technology attachment) acceleration core scheduling management method, device, equipment and storage medium |
| CN116109110A (en) * | 2023-04-11 | 2023-05-12 | 华能信息技术有限公司 | Task scheduling method for service center |
| CN116339956A (en) * | 2023-05-29 | 2023-06-27 | 天翼云科技有限公司 | Distribution method and device of configuration tasks, electronic equipment and storage medium |
| CN116339956B (en) * | 2023-05-29 | 2023-10-10 | 天翼云科技有限公司 | Distribution method and device of configuration tasks, electronic equipment and storage medium |
| CN117193966A (en) * | 2023-08-10 | 2023-12-08 | 博智安全科技股份有限公司 | Distributed asset task scheduling method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111813554A (en) | Task scheduling processing method and device, electronic equipment and storage medium | |
| CN108182111B (en) | Task scheduling system, method and device | |
| US9424019B2 (en) | Updating hardware libraries for use by applications on a computer system with an FPGA coprocessor | |
| US8127119B2 (en) | Control-flow prediction using multiple independent predictors | |
| US20130346985A1 (en) | Managing use of a field programmable gate array by multiple processes in an operating system | |
| CN113535367A (en) | Task scheduling method and related device | |
| US9086911B2 (en) | Multiprocessing transaction recovery manager | |
| US20220253341A1 (en) | Memory-aware placement for virtual gpu enabled systems | |
| US7243354B1 (en) | System and method for efficiently processing information in a multithread environment | |
| US8589942B2 (en) | Non-real time thread scheduling | |
| CN112527449A (en) | Cluster node label creating method and system, electronic device and storage medium | |
| US10757190B2 (en) | Method, device and computer program product for scheduling multi-cloud system | |
| CN109669767B (en) | Task packaging and scheduling method and system for multi-type context dependence | |
| CN108139929B (en) | Task scheduling apparatus and method for scheduling a plurality of tasks | |
| US8667008B2 (en) | Search request control apparatus and search request control method | |
| CN116324718A (en) | Processor with multiple fetch and decode pipelines | |
| CN113485838A (en) | Server distribution method and device, electronic equipment and computer readable storage medium | |
| CN100507842C (en) | Method, device for controlling relation between control piece on interface and control piece display system | |
| CN107634978B (en) | Resource scheduling method and device | |
| US10303523B2 (en) | Method and apparatus to migrate stacks for thread execution | |
| US20220300322A1 (en) | Cascading of Graph Streaming Processors | |
| CN107391262B (en) | Job scheduling method and device | |
| US20140047454A1 (en) | Load balancing in an sap system | |
| CN103870313A (en) | Virtual machine task scheduling method and system | |
| CN112579280A (en) | Scheduling method and device of cloud resources and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20201023 |