CN112685181B - Push task scheduling method and system for balancing CPU resources - Google Patents
Push task scheduling method and system for balancing CPU resources Download PDFInfo
- Publication number
- CN112685181B CN112685181B CN202011592637.6A CN202011592637A CN112685181B CN 112685181 B CN112685181 B CN 112685181B CN 202011592637 A CN202011592637 A CN 202011592637A CN 112685181 B CN112685181 B CN 112685181B
- Authority
- CN
- China
- Prior art keywords
- task
- push
- lock
- time
- pushing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 21
- 230000007246 mechanism Effects 0.000 claims abstract description 8
- 230000003111 delayed effect Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The invention provides a push task scheduling method and a system for balancing CPU resources, wherein the method comprises the following steps: s1, performing time prediction on a pushing task by using a minimum heap mechanism to obtain expected time; s2, when the pushing task reaches the expected time, the pushing task is put on an unpredictable task linked list; s3, putting the push task on the unpredictable task linked list into a lock-free queue of a corresponding time consumption level in a working thread according to task time consumption; s4, each working thread traverses the push task in the lock-free queue from the highest priority to the lowest priority and executes the push task; s5, placing the executed push task in a lock-free queue for completing the task, recovering the executed push task from the lock-free queue for completing the task to the minimum heap group by the main push frame, and modifying the expected time. According to the invention, under the condition that the time-consuming task and the time-consuming task are not mutually influenced, the task is thinned, and the CPU resource is fully utilized, so that the pushing performance is improved.
Description
Technical Field
The invention relates to the technical field of network performance, in particular to a push task scheduling method and a system for balancing CPU resources.
Background
The existing method for scheduling the pushing task of the CPU resource comprises the following steps: and classifying the pushing tasks, and distributing different CPU cores for pushing. Because the push task has time-consuming tasks and time-consuming tasks, the tasks are classified in order to prevent the tasks from being influenced, and different CPU cores are allocated to each type of task. When this scheme has a problem, some CPU cores are in idle state for a long time, and some CPU cores are in 99% state for a long time. The push task is greatly delayed, and the CPU resource is not fully utilized.
Disclosure of Invention
The invention aims to provide a method and a system for scheduling a pushing task for balancing CPU resources, which are used for refining tasks, fully utilizing the CPU resources and improving the pushing performance under the condition that time-consuming tasks and time-consuming-free tasks are ensured not to be affected mutually.
The invention provides a push task scheduling method for balancing CPU resources, which comprises the following steps:
s1, performing execution time prediction on a push task by using a minimum heap mechanism to obtain the expected time of the push task;
S2, traversing whether all pushing tasks reach expected time, and when the pushing tasks reach the expected time, putting the pushing tasks on an unpredictable task linked list;
S3, time consumption levels are set for the lock-free queues of the working threads of each CPU core, and push tasks on the unpredictable task linked list are put into the lock-free queues of the corresponding time consumption levels in the working threads according to task time consumption;
S4, each working thread traverses and executes the push task in the lock-free queue from the highest priority to the lowest priority according to the dynamically allocated priority;
S5, placing the executed push task in a lock-free queue for completing the task, recovering the executed push task from the lock-free queue for completing the task to the minimum heap group by the main push frame, and modifying the expected time.
Further, in step S1, the method for predicting the execution time of the push task by using the minimum heap mechanism to obtain the expected time consumption is as follows:
S11, splitting a push task into push tasks with minimum granularity, and entering a lock-free queue of a new task through configuration updating;
S12, the main pushing frame polls a lock-free queue of the new task and judges whether a new pushing task arrives in the lock-free queue of the new task, if so, the new pushing task is added to the minimum heap group;
S13, sorting the minimum stack groups according to different links, wherein the minimum stack top is the expected time of the next pushing task.
Further, in step S12, the main pushing framework updates the version number when adding the new pushing task to the minimum heap group; and when the version numbers obtained when the executed push tasks are recovered to the minimum heap group in the step S6 are inconsistent, eliminating the task in the lock-free queue for completing the task, and not continuing to push the task.
Further, in step S3, the time-consuming levels of the lock-free queues of the working threads of the same CPU core are similar.
Further, each working thread in step S4 is further allocated with a corresponding execution hierarchy according to the time-consuming level, that is, when traversing to a certain time-consuming level, the subsequent lock-free queue will not be traversed.
Further, when the CPU core configured by the user is insufficient and the push task data volume is too large, a delay occurs in the push task, a time period with a large delay is skipped.
The invention also provides a pushing task scheduling system for balancing the CPU resources, and the pushing task scheduling system adopts the pushing task scheduling method to perform pushing task scheduling.
In summary, due to the adoption of the technical scheme, the beneficial effects of the invention are as follows:
1. according to the method and the device, the time consumption influence of the pushing task can be greatly reduced by setting the time consumption grade.
2. According to the invention, the CPU resource can be fully utilized by the mode that the working thread occupies the push task in the lock-free queue.
3. The invention utilizes the minimum heap mechanism to predict the execution time of the push task, and can greatly reduce the traversal checking cost of the push task.
Therefore, the pushing task scheduling method can refine the task and fully utilize CPU resources under the condition that the time-consuming task and the time-consuming-free task are not mutually influenced, so that the pushing performance is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following description will briefly describe the drawings in the embodiments, it being understood that the following drawings only illustrate some embodiments of the present invention and should not be considered as limiting the scope, and that other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a data flow diagram of a push architecture of a method for scheduling push tasks for balancing CPU resources according to the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Examples
The embodiment provides a push task scheduling method for balancing CPU resources, which is shown in fig. 1, and includes the following steps:
s1, performing execution time prediction on a push task by using a minimum heap mechanism to obtain the expected time of the push task; specifically:
S11, splitting a push task into push tasks with minimum granularity, and entering a lock-free queue of a new task through configuration updating;
S12, the main pushing frame polls a lock-free queue of the new task and judges whether a new pushing task arrives in the lock-free queue of the new task, if so, the new pushing task is added to the minimum heap group;
S13, sorting the minimum stack groups according to different links, wherein the minimum stack top is the expected time of the next pushing task.
By the minimum heap mechanism in the step S1, the traversing of the push task can be greatly reduced to only checking whether the minimum heap top task reaches the expected time, so that the traversing checking overhead of the push task is greatly reduced.
S2, traversing whether all pushing tasks reach expected time, and when the pushing tasks reach the expected time, putting the pushing tasks on an unpredictable task linked list;
Since the push task is divided into a second task, a minute task, an hour task and a day task, the aggregation of time tasks depends forward, and thus the backward task is delayed slightly, when the push task reaches the expected time, the push task is put on an unpredictable task linked list, and then the push task on the unpredictable task linked list is checked in a short time by the following steps. In addition, when the CPU core configured by the user is insufficient and the data volume of the push task is too large, the push task is delayed, and the time period with the large delay is skipped, so that the push task in the latest period can be normally executed.
S3, time consumption levels are set for the lock-free queues of the working threads of each CPU core, and push tasks on the unpredictable task linked list are put into the lock-free queues of the corresponding time consumption levels in the working threads according to task time consumption; in some embodiments, the time consumption levels of the lock-free queues of the working threads of the same CPU core are similar, for example, the lock-free queues of the working threads with the same or adjacent time consumption levels belong to the same CPU core, so that when the CPU cores are allocated, the push tasks with similar time consumption levels can be allocated to the same CPU core according to the CPU cores, the thread numbers and the time consumption levels, and the threads of the CPU cores are guaranteed to be balanced as much as possible, so that the time consumption influence of the push tasks is greatly reduced.
S4, each working thread traverses and executes the push task in the lock-free queue from the highest priority to the lowest priority according to the dynamically allocated priority; therefore, CPU resources are fully utilized in a mode that a working thread occupies a pushing task in the lock-free queue. Furthermore, in order to prevent the time-consuming task from occupying the working thread for a long time, so as to influence the execution progress of the time-consuming task, each working thread in the step S4 is further allocated with a corresponding execution hierarchy according to the time-consuming hierarchy, that is, the subsequent lock-free queue is not traversed when the time-consuming hierarchy is traversed to a certain time-consuming hierarchy, so that the time-consuming task is not stranded when the time-consuming task is pushed.
S5, placing the executed push task in a lock-free queue for completing the task, recovering the executed push task from the lock-free queue for completing the task to the minimum heap group by the main push frame, and modifying the expected time. In some embodiments, the main push framework in step S12 updates the version number when adding a new push task to the minimum heap group; and when the version numbers obtained when the executed push tasks are recovered to the minimum heap group in the step S6 are inconsistent, eliminating the task in the lock-free queue for completing the task, and not continuing to push the task. Thus, the push task scheduling is completed through the steps S1 to S5.
In some embodiments, a push task scheduling system for balancing CPU resources is further implemented, where the push task scheduling system performs push task scheduling by using the push task scheduling method described above. The specific process is not described in detail.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (4)
1. The push task scheduling method for balancing the CPU resources is characterized by comprising the following steps of:
s1, performing execution time prediction on a push task by using a minimum heap mechanism to reduce traversal checking overhead of the push task and obtain expected time of the push task;
S2, traversing whether all pushing tasks reach expected time, and when the pushing tasks reach the expected time, putting the pushing tasks on an unpredictable task linked list;
s3, time consumption levels are set for the lock-free queues of the working threads of each CPU core, and push tasks on the unpredictable task linked list are put into the lock-free queues of the corresponding time consumption levels in the working threads according to task time consumption; step S3, the time consumption levels of the lock-free queues of the working threads of the same CPU core are similar;
S4, each working thread traverses and executes the push task in the lock-free queue from the highest priority to the lowest priority according to the dynamically allocated priority; in the step S4, each working thread is distributed with a corresponding execution level according to the time consumption level, namely, when traversing to a certain time consumption level, the subsequent lock-free queue is not traversed;
S5, placing the executed push task in a lock-free queue for completing the task, recovering the executed push task from the lock-free queue for completing the task to a minimum heap group by the main push frame, and modifying the expected time;
in step S1, the method for predicting the execution time of the push task by using the minimum heap mechanism to obtain the expected time consumption is as follows:
S11, splitting a push task into push tasks with minimum granularity, and entering a lock-free queue of a new task through configuration updating;
S12, the main pushing frame polls a lock-free queue of the new task and judges whether a new pushing task arrives in the lock-free queue of the new task, if so, the new pushing task is added to the minimum heap group;
S13, sorting the minimum stack groups according to different links, wherein the minimum stack top is the expected time of the next pushing task.
2. The method for scheduling push tasks for balancing CPU resources according to claim 1, wherein in step S12, the main push frame updates a version number when a new push task is added to the minimum heap group; and when the version numbers obtained when the executed push tasks are recovered to the minimum heap group in the step S6 are inconsistent, eliminating the task in the lock-free queue for completing the task, and not continuing to push the task.
3. The method for scheduling the push task for balancing the CPU resources according to claim 1, wherein when the CPU core configured by the user is insufficient and the amount of data of the push task is too large, a delay occurs in the push task, a time period with a large delay is skipped.
4. A push task scheduling system for balancing CPU resources, wherein the push task scheduling system performs push task scheduling by using the push task scheduling method according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011592637.6A CN112685181B (en) | 2020-12-29 | 2020-12-29 | Push task scheduling method and system for balancing CPU resources |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011592637.6A CN112685181B (en) | 2020-12-29 | 2020-12-29 | Push task scheduling method and system for balancing CPU resources |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112685181A CN112685181A (en) | 2021-04-20 |
| CN112685181B true CN112685181B (en) | 2024-06-04 |
Family
ID=75455050
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011592637.6A Active CN112685181B (en) | 2020-12-29 | 2020-12-29 | Push task scheduling method and system for balancing CPU resources |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112685181B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104391754A (en) * | 2014-10-13 | 2015-03-04 | 北京星网锐捷网络技术有限公司 | Method and device for processing task exception |
| CN104915253A (en) * | 2014-03-12 | 2015-09-16 | 中国移动通信集团河北有限公司 | Work scheduling method and work processor |
| CN107301085A (en) * | 2017-05-31 | 2017-10-27 | 深圳市神云科技有限公司 | A kind of cloud platform method for allocating tasks based on queue |
| CN109716297A (en) * | 2016-09-16 | 2019-05-03 | 华为技术有限公司 | Optimization is directed to the operating system timer of high activity ratio |
| CN109756565A (en) * | 2018-12-26 | 2019-05-14 | 成都科来软件有限公司 | A kind of Multitask Data method for pushing based on statistical form |
| CN110096340A (en) * | 2018-01-29 | 2019-08-06 | 北京世纪好未来教育科技有限公司 | Timed task processing method and processing device |
| CN110311965A (en) * | 2019-06-21 | 2019-10-08 | 长沙学院 | Method for scheduling task and system under a kind of cloud computing environment |
| CN111506438A (en) * | 2020-04-03 | 2020-08-07 | 华夏龙晖（北京）汽车电子科技股份有限公司 | Shared resource access method and device |
| CN111813515A (en) * | 2020-06-29 | 2020-10-23 | 中国平安人寿保险股份有限公司 | Multi-process-based task scheduling method and device, computer equipment and medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2002069584A1 (en) * | 2001-02-26 | 2002-09-06 | Maple Optical Systems, Inc. | Data packet transmission scheduling based on anticipated finish times |
| US8131983B2 (en) * | 2008-04-28 | 2012-03-06 | International Business Machines Corporation | Method, apparatus and article of manufacture for timeout waits on locks |
-
2020
- 2020-12-29 CN CN202011592637.6A patent/CN112685181B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104915253A (en) * | 2014-03-12 | 2015-09-16 | 中国移动通信集团河北有限公司 | Work scheduling method and work processor |
| CN104391754A (en) * | 2014-10-13 | 2015-03-04 | 北京星网锐捷网络技术有限公司 | Method and device for processing task exception |
| CN109716297A (en) * | 2016-09-16 | 2019-05-03 | 华为技术有限公司 | Optimization is directed to the operating system timer of high activity ratio |
| CN107301085A (en) * | 2017-05-31 | 2017-10-27 | 深圳市神云科技有限公司 | A kind of cloud platform method for allocating tasks based on queue |
| CN110096340A (en) * | 2018-01-29 | 2019-08-06 | 北京世纪好未来教育科技有限公司 | Timed task processing method and processing device |
| CN109756565A (en) * | 2018-12-26 | 2019-05-14 | 成都科来软件有限公司 | A kind of Multitask Data method for pushing based on statistical form |
| CN110311965A (en) * | 2019-06-21 | 2019-10-08 | 长沙学院 | Method for scheduling task and system under a kind of cloud computing environment |
| CN111506438A (en) * | 2020-04-03 | 2020-08-07 | 华夏龙晖（北京）汽车电子科技股份有限公司 | Shared resource access method and device |
| CN111813515A (en) * | 2020-06-29 | 2020-10-23 | 中国平安人寿保险股份有限公司 | Multi-process-based task scheduling method and device, computer equipment and medium |
Non-Patent Citations (1)
| Title |
|---|
| "多线程应用中的定时器管理算法";姚崇华 等;《计算机工程》;20100120(第02期);81-83 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112685181A (en) | 2021-04-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112162865B (en) | Scheduling method and device of server and server | |
| EP3770774B1 (en) | Control method for household appliance, and household appliance | |
| US10659316B1 (en) | State machine controlled dynamic distributed computing | |
| US8719834B2 (en) | Information processing system, method, program and integrated circuit for maintaining balance of processing loads with respect to real-time tasks | |
| CN110489217A (en) | A kind of method for scheduling task and system | |
| CN104199739B (en) | A kind of speculating type Hadoop dispatching methods based on load balancing | |
| Petrov et al. | Adaptive performance model for dynamic scaling Apache Spark Streaming | |
| CN113032102A (en) | Resource rescheduling method, device, equipment and medium | |
| CN114625500A (en) | Method and application for scheduling micro-service application based on topology perception in cloud environment | |
| Qian et al. | S-storm: A slot-aware scheduling strategy for even scheduler in storm | |
| US9424078B2 (en) | Managing high performance computing resources using job preemption | |
| CN112925634A (en) | Heterogeneous resource scheduling method and system | |
| CN111857990B (en) | Method and system for enhancing YARN long-type service scheduling | |
| CN112685181B (en) | Push task scheduling method and system for balancing CPU resources | |
| US20220035665A1 (en) | Sharing of compute resources between the virtualized radio access network (vran) and other workloads | |
| CN112860401A (en) | Task scheduling method and device, electronic equipment and storage medium | |
| CN106020971B (en) | CPU scheduling method and device in cloud host system | |
| Ibrahim et al. | Improving mapreduce performance with progress and feedback based speculative execution | |
| CN112130927A (en) | Reliability-enhanced mobile edge computing task unloading method | |
| CN115361349A (en) | Resource using method and device | |
| JP6285850B2 (en) | Process migration method and cluster system | |
| CN109558214B (en) | Host machine resource management method and device in heterogeneous environment and storage medium | |
| Yu et al. | Towards dynamic resource provisioning for traffic mining service cloud | |
| CN112579257A (en) | Scheduling method and device of virtual central processing unit core and related equipment | |
| Nascimento et al. | Dynamic self-scheduling for parallel applications with task dependencies |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 610041 12th, 13th and 14th floors, unit 1, building 4, No. 966, north section of Tianfu Avenue, Chengdu hi tech Zone, China (Sichuan) pilot Free Trade Zone, Chengdu, Sichuan Applicant after: Kelai Network Technology Co.,Ltd. Address before: 41401-41406, 14th floor, unit 1, building 4, No. 966, north section of Tianfu Avenue, Chengdu hi tech Zone, Chengdu Free Trade Zone, Sichuan 610041 Applicant before: Chengdu Kelai Network Technology Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |