WO2016197716A1 - Procédé et dispositif d'ordonnancement de tâches - Google Patents

Procédé et dispositif d'ordonnancement de tâches Download PDF

Info

Publication number
WO2016197716A1
WO2016197716A1 PCT/CN2016/080057 CN2016080057W WO2016197716A1 WO 2016197716 A1 WO2016197716 A1 WO 2016197716A1 CN 2016080057 W CN2016080057 W CN 2016080057W WO 2016197716 A1 WO2016197716 A1 WO 2016197716A1
Authority
WO
WIPO (PCT)
Prior art keywords
job
queue
calculated
factor
priority
Prior art date
Application number
PCT/CN2016/080057
Other languages
English (en)
Chinese (zh)
Inventor
郑鹏飞
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2016197716A1 publication Critical patent/WO2016197716A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/48—Indexing scheme relating to G06F9/48
    • G06F2209/484—Precedence

Definitions

  • This document relates to, but is not limited to, the field of computer technology, and in particular, to a job scheduling method and apparatus.
  • MapReduce is a widely used distributed system processing framework for processing large-scale data.
  • the scheduler is a core component of MapReduce and has a critical impact on the performance of the entire system.
  • common schedulers are: fair scheduler, FIFO scheduler.
  • small jobs dominate the load. For example, in Facebook and Bing applications, jobs with less than 10 jobs accounted for 82% and 61% of the total number of jobs, respectively, while in Google, the average execution time for MapReduce jobs was 395 seconds, at Yahoo! In a production cluster, more than 80% of the job completion time is less than ten minutes.
  • the large increase in small jobs is because small jobs such as random queries are starting to run more and more in MapReduce clusters, where latency is a critical factor influencing the experience of small jobs.
  • the fair scheduler is currently the most widely used scheduler.
  • the core idea of the fair scheduler is to make all users and jobs as close as possible to the 'substantially equal' cluster resources. Since all users and jobs are emphasized to occupy resources fairly, The fair scheduler has a longer response time to interactive small jobs.
  • Many distributed systems use the FIFO scheduling scheduler by default, but the FIFO scheduler also has some problems, such as the operation is prone to "starvation", can not be applied to multi-user scenarios, etc., therefore, the scope of application is narrow.
  • a small job priority scheduling strategy is proposed.
  • the small job priority scheduling strategy is compared with the fair scheduling strategy.
  • the test results show that the average response time of the job using the small job priority scheduling strategy is shorter than the average response time of the job using the fair scheduling strategy, especially to a large extent. Shorten the response time of small jobs and have little impact on the response time of large jobs.
  • small job priority strategies may cause “hunger” in large jobs, such as having large jobs.
  • a large number of small jobs are continuously submitted to the MapReduce system. Due to the priority scheduling of short jobs, large jobs may not be scheduled for processing, resulting in "starvation.”
  • This paper provides a job scheduling method and device, which can shorten the response time of small jobs, prevent "starvation” in large jobs, and achieve the purpose of improving job scheduling efficiency.
  • An embodiment of the present invention provides a job scheduling method, where the method includes: when a free resource occurs, calculating the job queue according to a size factor of each job in the job queue and a waiting factor of each job in the job queue. a priority of each job, and sorting the jobs in the job queue according to the priority; scheduling the jobs in the job queue according to the order of the sorted job queues.
  • the priority of each job in the job queue is calculated according to the size factor of each job in the job queue and the waiting factor of each job in the job queue, and Sorting the jobs in the job queue according to the priority, including: when an idle resource occurs, calculating a size factor of each job in the job queue according to a task processing condition of each job in the job queue; Calculating a waiting factor of the job in the job queue in a time scheduled for each job in the job queue; calculating the job according to the calculated size factor of each job and the waiting factor of each job The priority of each job in the queue.
  • calculating a size factor of each job in the job queue according to a task processing condition of each job in the job queue including: when an idle resource occurs, determining the job queue separately Whether each job has been scheduled to be processed; if the job in the job queue has never been scheduled, the size factor of the job is calculated according to the input data amount of the job; if the job in the job queue Has been scheduled for processing, based on the job The size of the job is calculated by the time it takes to perform the remaining tasks.
  • calculating a size factor of the job according to a time required for the job to execute the remaining tasks including: calculating the job queue The current task progress of the middle job and the time used by the job in the job queue to reach the current progress; according to the calculated current task progress of the job and the time taken by the calculated job to reach the current progress, calculate the Calculating an execution rate of the job; calculating a remaining task quantity of the job according to the calculated current task progress of the job; calculating according to the calculated execution rate of the job and the remaining task quantity of the calculated job The time required for the job to execute the remaining tasks; and the size factor of the job is calculated according to the time required for the calculated job to execute the remaining tasks.
  • the priority of the job in the job queue is calculated according to the size factor of the job in the job queue and the waiting factor of the job in the job queue, and the job queue is The jobs are sorted according to the priority, including: when an idle resource occurs, each job in the job queue is calculated according to a size factor of each job in the job queue and a wait factor of each job in the job queue. a priority, and sorting the jobs in the job queue according to the priority; wherein the wait factor is positively related to the priority, and the size factor is negatively correlated with the priority.
  • An embodiment of the present invention provides a job scheduling apparatus, where the apparatus includes: a calculating unit, a sorting unit, and a processing unit; wherein the calculating unit is configured to, according to a size factor of each job in the job queue, when an idle resource occurs Calculating a priority of each job in the job queue with a wait factor of each job in the job queue; the sorting unit is configured to set the job in the job queue according to the priority calculated by the calculation unit The processing unit is configured to perform scheduling processing on the jobs in the job queue according to the order of the sorted job queues.
  • the calculating unit is configured to calculate a size factor of each job in the job queue according to a task processing condition of each job in the job queue when an idle resource occurs; according to each of the job queues Calculating the waiting factor of the job in the job queue before the job is scheduled; calculating each job in the job queue according to the calculated size factor of each job and the waiting factor of each job Priority.
  • the calculating unit is configured to determine, when an idle resource occurs, whether each job in the job queue has been scheduled for processing; if the job in the job queue has not been scheduled, Calculating a size factor of the job according to the input data of the job; if the job in the job queue has been scheduled, the size of the job is calculated according to a time required for the job to execute the remaining tasks factor.
  • the calculating unit is configured to calculate a current task progress of the job in the job queue and a time used to reach the current progress; according to the calculated current task progress of the job and the calculated Calculating the execution rate of the job when the job reaches the current progress; calculating the remaining task number of the job according to the calculated current task progress of the job; and calculating the execution rate of the job according to the calculated Calculating the remaining tasks of the job, calculating the time required for the job to execute the remaining tasks; and calculating the size factor of the job according to the time required for the calculated job to execute the remaining tasks.
  • the sorting unit is configured to calculate each of the job queues according to a size factor of each job in the job queue and a wait factor of each job in the job queue when an idle resource occurs a priority of the job, and sorting the jobs in the job queue according to the priority; wherein the wait factor is positively correlated with the priority; the size factor is negatively correlated with the priority.
  • the embodiment of the invention further provides a computer readable storage medium storing computer executable instructions, which are implemented when executed by a processor.
  • the embodiment of the invention provides a job scheduling method and device, which first calculates the priority of the job by the size factor and the waiting factor of the job, and then performs scheduling processing according to the priority of the job, so that the response time of the small job can be shortened. At the same time, it prevents "starvation" in large operations and achieves the technical effect of improving job scheduling efficiency.
  • FIG. 1 is a schematic flowchart of a job scheduling method according to an embodiment of the present invention
  • FIG. 2 is a schematic flowchart of a calculation of a job size factor according to an embodiment of the present invention
  • FIG. 3 is a schematic flowchart of a job scheduling method according to an embodiment of the present disclosure
  • FIG. 4 is a schematic flowchart of another job scheduling method according to an embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of a job scheduling apparatus according to an embodiment of the present invention.
  • a job scheduling method according to an embodiment of the present invention is shown, which may include:
  • the priority of the job is dynamically calculated. Whenever there is an idle resource, the priority of each job in the job queue needs to be recalculated. In the specific implementation process, the job queues can be sorted according to the order of priority from high to low.
  • step S101 may include:
  • the size factor of the job is calculated according to the input data amount of the job; wherein the input data amount of the job may specifically be the number of tasks of the job.
  • the size factor of the job is calculated according to the time required for the job to execute the remaining tasks.
  • the specific execution steps include:
  • S10111 Calculate the current task progress of the job in the job queue and the time used in the job queue to reach the current progress;
  • S10112 Calculate an execution rate of the job according to the calculated current task progress of the job and the calculated time used by the job to reach the current progress;
  • S10113 Calculate the remaining tasks of the job according to the calculated current task progress of the job
  • S10114 Calculate, according to the calculated execution rate of the job and the calculated remaining task amount of the job, a time required for the job to execute the remaining tasks;
  • S10115 Calculate the size factor of the job according to the calculated time required for the job to execute the remaining tasks.
  • S1012 Calculate a wait factor of the job in the job queue according to a time scheduled for each job in the job queue;
  • the waiting time refers to the time when the job was last allocated resources to the current time.
  • S1013 Calculate the priority of each job in the job queue according to the calculated size factor of each job and the waiting factor of each job.
  • S102 Perform scheduling processing on the jobs in the job queue according to the order of the job queues after sorting.
  • the data locality refers to the data that needs to be scheduled and processed on the node of the idle resource.
  • the data of the job needs to be transmitted to the node of the idle resource, which is time-consuming and resource-consuming. Therefore, if the job to be processed in the job queue does not satisfy the data locality, the next job is scheduled according to the order of the job queue; if the job queue under the current user name does not satisfy the data locality, then according to The order of the user queues is scheduled for the next user; if there is no job that satisfies the data locality in the job queue under all the user names in the user queue, the idle resources are directly allocated to the users who are to be executed the map task. The highest priority user, in turn, allocates idle resources to the first job in the job queue under the highest priority user name to be executed.
  • the current task progress of the job is calculated according to the number of tasks that have been completed in the job and the progress of the running task; wherein, the current task progress of the job can be calculated by using equation (1):
  • the job j is taken as an example for description
  • the parameter jobPS j represents the current task progress of the job j
  • the parameter c represents the number of tasks that have been completed in the job j
  • the parameters Indicates the progress of the task being run in job j
  • parameter R j represents the set of tasks running in job j
  • parameter r represents the number of tasks running in job j.
  • the job j is taken as an example for description
  • the parameter jobExT j represents the linear time used by the job j to reach the current progress
  • the parameter The sum of the time spent representing the currently completed task in job j
  • the parameter Represents the sum of the time taken by the running task in job j to complete the current progress
  • parameter C j is the set of tasks that job j has completed
  • parameter R j represents the set of tasks that are running in job j
  • parameter c is already in job j
  • the parameter time t is the time taken to complete the task t in the job j
  • the parameter r represents the number of tasks being run in the job j
  • the parameter curT is the current time
  • the parameter start t is the task t in the job j starts running. time.
  • Equation (2) The specific calculation method is: first subtract the time of the task from the current time of the task to get the time taken by the task to complete the current progress, and then accumulate the time taken by all the running tasks in the job j to complete the current progress, You can get the total time spent on the current progress of all running tasks in job j.
  • the execution rate of the job is calculated; wherein the execution rate of the job can be calculated by using the formula (3) :
  • jobExR j jobPS j /jobExT (3)
  • the job j is taken as an example.
  • the parameter jobExR j represents the execution rate of the job j
  • the parameter jobPS j represents the current task progress of the job j
  • the parameter jobExT j represents the time used by the job j to reach the current progress.
  • the remaining number of tasks of the job is calculated; wherein, the formula (4) can be used to calculate the remaining tasks of the job:
  • jobRS j N-jobPS j (4)
  • the job j is taken as an example.
  • the parameter jobRS j indicates the number of remaining tasks of the job j
  • the parameter jobPS j indicates the current task progress of the job j
  • the parameter N is the total number of tasks of the job j.
  • the time required for the job to execute the remaining tasks is calculated; wherein the job execution can be calculated by using the equation (5) Time required to complete the remaining tasks:
  • jobRT j jobRS j /jobExR j (5)
  • the job j is taken as an example for explanation.
  • the parameter jobRT j represents the remaining time required for the job j to complete all tasks
  • the parameter jobRS j represents the remaining task number of the job j
  • the parameter jobExR j represents the execution rate of the job j. .
  • the size factor of the job is calculated; wherein the size factor of the job can be calculated by using equation (6) or (7):
  • the job j is taken as an example.
  • the parameter jobSizeFactor j represents the size factor of the job j
  • the parameter jobRT j represents the time required for the job j to execute the remaining tasks
  • the parameter n represents the job queue in the current user name.
  • the parameter jobRTasksNum j represents the number of remaining tasks of job j
  • the parameter n represents the number of jobs in the job queue under the current user name.
  • the size factor of the job is calculated by equation (7) according to the number of tasks of the job;
  • the size factor of the job is calculated based on the time required to execute the remaining tasks.
  • the specific steps of calculating the size factor of the job are as shown in FIG. 2, including:
  • S202 Calculate, by formula (2), a time used by the job to reach a current progress
  • S203 calculating a current execution schedule of the job according to the current task schedule calculated by the formula (1) and a time calculated by the formula (2) to reach the current progress, and calculating an execution rate of the job by the formula (3);
  • S205 an execution rate of the job calculated according to the formula (3) and a remaining task amount of the job calculated by the formula (4), and calculating, by the formula (5), a time required for the job to execute the remaining tasks;
  • the present invention adds a wait factor to the calculation of job priorities to balance idle resources.
  • the allocation in order to shorten the response time of small operations, to avoid “starvation” in other operations.
  • the waiting time of the job is from the time when the job last obtained the idle resource to the current time. The larger the wait factor for a job, the longer the job waits for other jobs.
  • the waiting factor of the job can be calculated according to the waiting time of the job; wherein, the waiting time of the job can be calculated by using equation (8):
  • the job j is taken as an example.
  • the parameter jobWT j represents the waiting time of the job j
  • the parameter n represents the number of jobs in the job queue under the current user name.
  • the priority of the job can be calculated; where (9) can be used to calculate the priority of the job:
  • Priority j jobWaitFactor j +(1/jobSizeFactor j ) (9)
  • the job j is taken as an example.
  • the parameter jobWaitFactor j is the waiting factor of the job j obtained by the equation (8)
  • the parameter jobSizeFactor is the size factor of the job j obtained by the equation (6) or the equation (7).
  • the parameter priority j is the priority of the calculated job j.
  • the priority of the job is inversely proportional to the size factor of the job, and is proportional to the wait factor of the job.
  • the remaining time required for the job to complete all tasks and the waiting time of the job are converted into the priority of the job, taking into account the remaining time required for the job to complete all tasks and the waiting time of the job.
  • the priority of the small job is increased by the size factor of the job, so as to achieve the technical effect of shortening the response time of the small job; the priority of the large job is improved by the waiting factor of the job, so as to avoid the technical effect of avoiding "starvation” in the large operation.
  • This scheduling method can greatly shorten the response time of small jobs, prevent "starvation” in large jobs, improve the user experience, and achieve the technical effect of improving job scheduling efficiency.
  • the scheduling method of the present invention can not only shorten the homework response time of a small job, but also cause a big job "starvation" and improve the user experience.
  • S302 Select a first user in the user queue according to a sequence of user queues for scheduling processing
  • the specific process of calculating the priority includes:
  • the total number of tasks for the three jobs a, b, and c in the job queue is 10.
  • the waiting time for the a job, the b job, and the c job is 0 s.
  • S304 Sort the job queues under the current user name from high to low according to the priority, sort The result is: c, b, a;
  • S305 Determine, according to the order of the sorted job queues, that the jobs in the job queue under the current user name satisfy the data locality
  • S306 Perform scheduling processing on the first job c in the sorted job queue, that is, allocate idle resources to the c job in the job queue.
  • the technical solution of the present invention can preferentially schedule the small job, and considers the execution rate of the job, which not only shortens the response time of the small job to a large extent, but also improves the response time.
  • the efficiency of job scheduling has greatly improved the user experience of small jobs.
  • the scheduling process of the job is described in another embodiment.
  • the specific steps are as shown in FIG. 4, including:
  • S402 The first user in the user queue is selected for scheduling processing according to the sequence of the user queue, because the first user in the user queue has been scheduled to be processed.
  • S403 Calculate a priority of three jobs in the job queue under the current user name before the scheduling process
  • the specific process of calculating the priority includes:
  • S4034 Calculate the remaining tasks of three jobs a, b, and c in the job queue by equation (4) The quantity, wherein the total number of tasks of the first job, the second job, and the third job in the job queue are 100, 10, and 5, respectively, and the calculation results of a, b, and c are respectively: 50, 2, and 4;
  • S4035 Calculate the time required for the three tasks a, b, and c in the job queue to execute the remaining tasks by equation (5).
  • the calculation results of a, b, and c are: 90s, 5s, and 8s, respectively;
  • S404 Sort the job queues under the current user name from high to low according to the priority, and the sort result is: a, c, b.
  • S405 Determine, according to the order of the sorted job queues, that the jobs in the job queue under the current user name satisfy the data locality;
  • S406 Perform scheduling processing on the first job a in the job queue according to the order of the sorted job queues, that is, allocate idle resources to the a job in the job queue.
  • the technical solution of the present invention can use the waiting factor to influence the priority of the operation, so as to realize the processing of such operations as soon as possible, and to avoid the phenomenon of “starvation” to a large extent.
  • the apparatus 50 includes: a calculating unit 501, a sorting unit 502, and a processing unit 503;
  • the calculating unit 501 is configured to: when a free resource occurs, calculate a priority of each job in the job queue according to a size factor of each job in the job queue and a waiting factor of each job in the job queue;
  • the sorting unit 502 is configured to sort the jobs in the job queue according to the priorities calculated by the calculating unit 501;
  • the processing unit 503 is configured to perform scheduling processing on the jobs in the job queue according to the order of the sorted job queues.
  • the calculating unit 501 is configured to calculate, according to the task processing condition of each job in the job queue, a size factor of each job in the job queue when an idle resource occurs; according to the job queue Calculating a waiting factor of the job in the job queue at a time scheduled once before each job; calculating each of the job queues according to the calculated size factor of each job and the waiting factor of each job The priority of the job.
  • the calculating unit 501 is configured to determine, when an idle resource occurs, whether each job in the job queue has been scheduled to be processed; if the job in the job queue has not been scheduled, Calculating a size factor of the job according to the input data of the job; if the job in the job queue has been scheduled to be processed, calculating the job according to the time required for the job to execute the remaining tasks Size factor.
  • the calculating unit 501 is configured to calculate a current task progress of the job in the job queue and a time used to reach the current progress; and calculate the current task progress and the calculated according to the calculated job. Calculating the execution rate of the job when the job reaches the current progress; calculating the remaining task number of the job according to the calculated current task progress of the job; and calculating the execution rate of the job according to the calculated job And calculating, by the calculated number of remaining tasks of the job, a time required for the job to execute the remaining tasks; and calculating a size factor of the job according to the time required for the calculated job to execute the remaining tasks .
  • the sorting unit 502 is configured to calculate, when an idle resource occurs, each of the job queues according to a size factor of each job in the job queue and a wait factor of each job in the job queue.
  • the priority of the jobs, and the jobs in the job queue are sorted according to the priority; wherein the wait factor is positively related to the priority; the size factor is negatively correlated with the priority.
  • the instructions are related to hardware (eg, a processor) that can be stored in a computer readable storage medium, such as a read only memory, a magnetic disk, or an optical disk.
  • a computer readable storage medium such as a read only memory, a magnetic disk, or an optical disk.
  • all or part of the steps of the above embodiments may also be implemented using one or more integrated circuits.
  • each module/unit in the above embodiment may be implemented in the form of hardware, for example, by implementing an integrated circuit to implement its corresponding function, or may be implemented in the form of a software function module, for example, executing a program stored in the memory by a processor. Instructions to achieve their corresponding functions. This application is not limited to any specific combination of hardware and software.
  • a job scheduling method and apparatus provided by an embodiment of the present invention first calculates a priority of a job by a size factor and a wait factor of the job, and then performs scheduling processing according to the priority of the job, so that the response time of the small job can be shortened. At the same time, it prevents "starvation" in large operations and achieves the technical effect of improving job scheduling efficiency.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

L'invention concerne un procédé d'ordonnancement de tâches comprenant: lorsque des ressources libres apparaissent, le calcul d'une priorité de chaque tâche dans une file d'attente de tâches en fonction d'un facteur de taille de chaque tâche dans la file d'attente de tâches et d'un facteur d'attente de chaque tâche dans la file d'attente de tâches, et le tri des tâches dans la file d'attente de tâches selon les priorités; l'exécution d'un traitement de programmation sur les tâches dans la file d'attente de tâches en fonction de l'ordre de la file d'attente de tâches après le tri.
PCT/CN2016/080057 2016-01-18 2016-04-22 Procédé et dispositif d'ordonnancement de tâches WO2016197716A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610031403.1A CN106980532A (zh) 2016-01-18 2016-01-18 一种作业调度方法和装置
CN201610031403.1 2016-01-18

Publications (1)

Publication Number Publication Date
WO2016197716A1 true WO2016197716A1 (fr) 2016-12-15

Family

ID=57503252

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/080057 WO2016197716A1 (fr) 2016-01-18 2016-04-22 Procédé et dispositif d'ordonnancement de tâches

Country Status (2)

Country Link
CN (1) CN106980532A (fr)
WO (1) WO2016197716A1 (fr)

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108345501A (zh) * 2017-01-24 2018-07-31 全球能源互联网研究院 一种分布式资源调度方法和系统
CN111045810A (zh) * 2019-12-17 2020-04-21 浙江大华技术股份有限公司 一种任务调度处理方法及装置
CN111221645A (zh) * 2019-11-14 2020-06-02 中国民航信息网络股份有限公司 一种航空作业处理方法和系统
US10705761B2 (en) 2018-09-14 2020-07-07 Yandex Europe Ag Method of and system for scheduling transmission of I/O operations
CN111813553A (zh) * 2020-07-17 2020-10-23 华侨大学 一种基于可选因子周期任务动态优先级低能耗方法
US10908982B2 (en) 2018-10-09 2021-02-02 Yandex Europe Ag Method and system for processing data
US10996986B2 (en) 2018-12-13 2021-05-04 Yandex Europe Ag Method and system for scheduling i/o operations for execution
US11003600B2 (en) 2018-12-21 2021-05-11 Yandex Europe Ag Method and system for scheduling I/O operations for processing
US11010090B2 (en) 2018-12-29 2021-05-18 Yandex Europe Ag Method and distributed computer system for processing data
US11048547B2 (en) 2018-10-09 2021-06-29 Yandex Europe Ag Method and system for routing and executing transactions
US11055160B2 (en) 2018-09-14 2021-07-06 Yandex Europe Ag Method of determining potential anomaly of memory device
US11061720B2 (en) 2018-09-14 2021-07-13 Yandex Europe Ag Processing system and method of detecting congestion in processing system
CN113379308A (zh) * 2021-06-30 2021-09-10 北京斯年智驾科技有限公司 一种任务派发方法、装置、电子设备和存储介质
CN113448705A (zh) * 2021-06-25 2021-09-28 皖西学院 一种不平衡作业调度算法
US11184745B2 (en) 2019-02-06 2021-11-23 Yandex Europe Ag Actor system and method for transmitting a message from a first actor to a second actor
CN113886052A (zh) * 2021-10-26 2022-01-04 上海商汤科技开发有限公司 任务调度方法、装置、设备、存储介质
US11288254B2 (en) 2018-10-15 2022-03-29 Yandex Europe Ag Method of and system for processing request in distributed database

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108052443A (zh) * 2017-10-30 2018-05-18 北京奇虎科技有限公司 一种测试任务调度方法、装置、服务器及存储介质
CN109189581B (zh) * 2018-09-20 2021-08-31 郑州云海信息技术有限公司 一种作业调度方法和装置
CN110287015A (zh) * 2019-06-27 2019-09-27 四川中电启明星信息技术有限公司 Pc自动化作业被动调度方法
CN111949408A (zh) * 2020-08-16 2020-11-17 广东奥飞数据科技股份有限公司 一种边缘计算资源的动态分配方法
CN111930514B (zh) * 2020-09-14 2021-09-10 四川中电启明星信息技术有限公司 一种资源优化分配方法及系统
CN112181645A (zh) * 2020-09-21 2021-01-05 中国建设银行股份有限公司 一种资源调度的方法、装置、设备及存储介质
CN113568725A (zh) * 2021-07-14 2021-10-29 中国科学院计算技术研究所 深度学习作业优先级调度方法及深度学习作业系统
CN113946430B (zh) * 2021-12-20 2022-05-06 北京并行科技股份有限公司 一种作业调度方法、计算设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101770402A (zh) * 2008-12-29 2010-07-07 中国移动通信集团公司 MapReduce系统中的Map任务调度方法、设备及系统
CN103336714A (zh) * 2013-06-20 2013-10-02 北京奇虎科技有限公司 一种作业调度方法和装置
CN104102794A (zh) * 2014-08-06 2014-10-15 浪潮电子信息产业股份有限公司 一种hadoop调度法的优化方法
CN104281492A (zh) * 2013-07-08 2015-01-14 无锡南理工科技发展有限公司 一种异构环境下的Hadoop任务公平调度方法
WO2015152871A1 (fr) * 2014-03-31 2015-10-08 Hewlett-Packard Development Company, L.P. Établissement de priorités dans un trafic de réseau d'un système de traitement distribué

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101770402A (zh) * 2008-12-29 2010-07-07 中国移动通信集团公司 MapReduce系统中的Map任务调度方法、设备及系统
CN103336714A (zh) * 2013-06-20 2013-10-02 北京奇虎科技有限公司 一种作业调度方法和装置
CN104281492A (zh) * 2013-07-08 2015-01-14 无锡南理工科技发展有限公司 一种异构环境下的Hadoop任务公平调度方法
WO2015152871A1 (fr) * 2014-03-31 2015-10-08 Hewlett-Packard Development Company, L.P. Établissement de priorités dans un trafic de réseau d'un système de traitement distribué
CN104102794A (zh) * 2014-08-06 2014-10-15 浪潮电子信息产业股份有限公司 一种hadoop调度法的优化方法

Cited By (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108345501B (zh) * 2017-01-24 2021-10-29 全球能源互联网研究院有限公司 一种分布式资源调度方法和系统
CN108345501A (zh) * 2017-01-24 2018-07-31 全球能源互联网研究院 一种分布式资源调度方法和系统
US11055160B2 (en) 2018-09-14 2021-07-06 Yandex Europe Ag Method of determining potential anomaly of memory device
US11449376B2 (en) 2018-09-14 2022-09-20 Yandex Europe Ag Method of determining potential anomaly of memory device
US10705761B2 (en) 2018-09-14 2020-07-07 Yandex Europe Ag Method of and system for scheduling transmission of I/O operations
US11061720B2 (en) 2018-09-14 2021-07-13 Yandex Europe Ag Processing system and method of detecting congestion in processing system
US10908982B2 (en) 2018-10-09 2021-02-02 Yandex Europe Ag Method and system for processing data
US11048547B2 (en) 2018-10-09 2021-06-29 Yandex Europe Ag Method and system for routing and executing transactions
US11288254B2 (en) 2018-10-15 2022-03-29 Yandex Europe Ag Method of and system for processing request in distributed database
US10996986B2 (en) 2018-12-13 2021-05-04 Yandex Europe Ag Method and system for scheduling i/o operations for execution
US11003600B2 (en) 2018-12-21 2021-05-11 Yandex Europe Ag Method and system for scheduling I/O operations for processing
US11010090B2 (en) 2018-12-29 2021-05-18 Yandex Europe Ag Method and distributed computer system for processing data
US11184745B2 (en) 2019-02-06 2021-11-23 Yandex Europe Ag Actor system and method for transmitting a message from a first actor to a second actor
CN111221645A (zh) * 2019-11-14 2020-06-02 中国民航信息网络股份有限公司 一种航空作业处理方法和系统
CN111221645B (zh) * 2019-11-14 2023-06-16 中国民航信息网络股份有限公司 一种航空作业处理方法和系统
CN111045810B (zh) * 2019-12-17 2023-08-29 浙江大华技术股份有限公司 一种任务调度处理方法及装置
CN111045810A (zh) * 2019-12-17 2020-04-21 浙江大华技术股份有限公司 一种任务调度处理方法及装置
CN111813553B (zh) * 2020-07-17 2023-05-26 华侨大学 一种基于可选因子周期任务动态优先级低能耗方法
CN111813553A (zh) * 2020-07-17 2020-10-23 华侨大学 一种基于可选因子周期任务动态优先级低能耗方法
CN113448705A (zh) * 2021-06-25 2021-09-28 皖西学院 一种不平衡作业调度算法
CN113379308A (zh) * 2021-06-30 2021-09-10 北京斯年智驾科技有限公司 一种任务派发方法、装置、电子设备和存储介质
CN113886052A (zh) * 2021-10-26 2022-01-04 上海商汤科技开发有限公司 任务调度方法、装置、设备、存储介质

Also Published As

Publication number Publication date
CN106980532A (zh) 2017-07-25

Similar Documents

Publication Publication Date Title
WO2016197716A1 (fr) Procédé et dispositif d'ordonnancement de tâches
US20220027202A1 (en) Stream-based accelerator processing of computational graphs
CN113535367B (zh) 任务调度方法及相关装置
US11010199B2 (en) Efficient critical thread scheduling for non-privileged thread requests
RU2647657C2 (ru) Назначение и планирование потоков для нескольких очередей с учетом приоритета
US9342355B2 (en) Joint optimization of multiple phases in large data processing
WO2017016421A1 (fr) Procédé d'exécution de tâches dans une grappe et dispositif utilisant ce procédé
US20130339966A1 (en) Sequential cooperation between map and reduce phases to improve data locality
US20130139172A1 (en) Controlling the use of computing resources in a database as a service
KR101350755B1 (ko) 클라우드 컴퓨팅에서 다중 워크플로우를 위한 비용기반 스케줄링 방법 및 그 시스템
JP2012511204A (ja) リソースを最適化するためのタスク再編成方法
US9947068B2 (en) System and method for GPU scheduling
KR20110080735A (ko) 컴퓨팅 시스템 및 방법
US11995016B2 (en) Input/output command rebalancing in a virtualized computer system
CN106775948B (zh) 一种基于优先级的云任务调度方法及装置
KR101770191B1 (ko) 자원 할당 방법 및 그 장치
CN112925616A (zh) 任务分配方法、装置、存储介质及电子设备
CN116881003A (zh) 资源分配方法、装置、服务设备及存储介质
CN116610422A (zh) 一种任务调度方法、装置和系统
CN112685158A (zh) 一种任务调度方法、装置、电子设备及存储介质
CN110532099A (zh) 资源隔离方法和装置以及电子设备和介质
CN112433841B (zh) 一种资源池调度方法、系统、服务器和存储介质
Ullah et al. Notice of Violation of IEEE Publication Principles: Hadoop Preemptive Deadline Constraint Scheduler
Huo et al. Dynamic Throughput Allocation among Multiple Servers for Heterogeneous Storage System
CN113138909A (zh) 负载统计方法、装置、存储介质及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16806614

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16806614

Country of ref document: EP

Kind code of ref document: A1