AU2007261607A1 - Resource-based scheduler - Google Patents

Description

WO 2007/149224 PCT/US2007/013394 RESOURCE-BASED SCHEDULER FIELD OF THE INVENTION [0001] The present invention relates to scheduling computer jobs. In particular, embodiments of the present invention relate to scheduling computer jobs based on resource utilization criteria of the particular jobs and utilization of computer resources to be used by the jobs. BACKGROUND [0002] Today's computer operating systems (O/S) utilize multitasking schedulers to give the appearance of more than one computerjob (e.g., process) running at the same time. There are many different scheduling algorithms, but generally the concept is that a small time slice known as a quantum is given to one thread of a process and then another thread of the process or another process, etc. The length of the quantum is very small, typically in the range of 20 to 120 milliseconds. Due to the human perception of time, it appears that the jobs are running concurrently. [0003] The central processing unit (CPU) tends to be the fastest component of most computer systems, while other computer resources such as disk 1/0, network 1/0, and even memory tend to be much slower. For example, disk 1/0 may be about a million times slower than the CPU if measured in terms of data transfer from the disk and data transfer within the CPU. As a result, the CPU often waits for these slower resources. For example, a three-gigahertz CPU often sits idle while waiting for a disk drive to retrieve data at an average access time measured in milliseconds. [0004] Since several different jobs are often vying for the same resources, jobs often collide with each other, which results in the slowing of one or more of the jobs. From a user's perspective, job collisions manifest themselves as unresponsive applications, jerky cursor movement and slowly rendered graphics. [00051 Throttling is a technique for minimizing these negative impacts. Throttling prevents an application or job from using more than an allocated amount of resources. Types of throttling include disk 1/0 throttling, CPU throttling and network throttling. For example, CPU throttling can involve establishing a target CPU utilization limit for an application and forcing the application to stop working if the application exceeds the target limit. Throttling is sometimes applied to computer resources for maintenance -1- WO 2007/149224 PCT/US2007/013394 applications or less important computer jobs. While throttling has benefits, the computer job's resource use is not totally transparent to other jobs and applications. [0006] The above problems are even more perplexing because computer resources are generally wasted over a 24-hour period. For example, most desktops utilize less than five percent of the computer's available resources, and high traffic servers often utilize around 20 percent. Even computers that utilize 80-90 percent of resources still have 10-20 percent of resources available. [00071 To recover and utilize these otherwise lost resources, what is needed is a technique that allows one or more jobs to execute in a computer system without significantly impacting other jobs or applications. The technique should not consume a user's time in scheduling the job nor should it negatively impact the user's interaction with the computer system when the job is running. [0008] The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. BRIEF DESCRIPTION OF THE DRAWINGS [0009] The present invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which: [00101 FIG. 1A is a diagram of a resource-based scheduler having resource-based scheduling worklist, in accordance with an embodiment of the present invention. [0011] FIG. lB is a diagram of a resource-based scheduler, in accordance with another embodiment of the present invention. [0012] FIG. 2 is a flowchart of resource-based scheduling, in accordance with an embodiment of the present invention. [0013] FIG. 3 is a block diagram that illustrates a computer system upon which an embodiment of the invention may be implemented. DETAILED DESCRIPTION [00141 In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, that the present invention may be practiced WO 2007/149224 PCT/US2007/013394 without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present invention. OVERVIEW [00151 The majority of computers do not utilize all of their resource capacity 100 percent of the time. This is typically true even of computers that seemingly are in high use twenty-four hours a day, seven days a week, as well as computers that are only turned on for a portion of each day. Therefore, computer time and resources are wasted. For example, over a twenty-four hour period, a computer system that is used quite heavily, and which may have brief spikes in activity, may on average use only about five to twenty percent of its resources. [00161 A computer job is scheduled based on utilization of a resource and a utilization criterion that the computer job has pertaining to the resource, in accordance with an embodiment of the present invention. For example, a computer job might have a utilization criterion that a disk 1/0 resource has 60 percent available capacity in order for the computer job to be scheduled. If the available capacity of the disk I/O resource is less than 60 percent, then the computer job is not scheduled, in this embodiment. [0017] In one embodiment, a resource-based scheduler (RBS) puts each computer job onto at least one of several different resource-based scheduling worklist. These resource based scheduling worklists include, but are not limited to, a disk I/O scheduling worklist, CPU scheduling worklist, and network I/O scheduling worklist. Thus, a particular scheduling worklist comprises computer jobs waiting to use the particular resource. Each of the computer jobs has a utilization criterion pertaining to the particular resource, in this embodiment. The RBS selects one of the resources to be next to have a computerjob scheduled. The selection is based on a priority of the resource, in one embodiment. For example, a disk 1/0 resource might have a higher priority than a CPU resource. In one embodiment, resources that are designated as slower resources are given a higher priority than resources that are designated as faster resources. 100181 After selecting which resource is to be scheduled, the RBS selects for execution one of the computer jobs from the worklist corresponding to the selected resource. The RBS selects the computerjob based on criteria that match the available capacity of the resource with the utilization criterion of the computerjobs, in one embodiment. However, the RBS may also use other factors such as the priority of the -3- WO 2007/149224 PCT/US2007/013394 computer jobs, required execution order of computer jobs, the order in which computer jobs were placed on the worklist, etc. [00191 Note that, in one embodiment, the RBS allows efficient utilization of multiple computer resources by scheduling each computer job when the resources to be utilized by the job are (fully or partially) available even though other resources not needed by that job are being (fully or partially) utilized by other jobs and reduces impingement on the performance of the other jobs. RESOURCE BASED SCHEDULER [0020] FIG. 1A is a diagram of a resource-based scheduler (RBS) 100 having resource-based scheduling worklists 120, in accordance with an embodiment of the present invention. The RBS 100 pro-actively schedules computer jobs based on available capacity of different computer resources. As examples, the different resources could include, but are not limited to, CPU, disk 1/0, network 1/0, video, memory, keyboard, network resource, etc. As used herein, the term "computer job" or "job" includes, but is not limited to, a computer process, a thread, a micro-job (discussed herein below), or any portion of executable computer code. The RBS 100 may allow each computer job to receive its requirement of resources without colliding with any other jobs' requirement for resources. The RBS 100 is able to determine when a particular resource is only partially utilized, and to allocate the un-utilized portion to a selected job, with minimal impingement on any of the other jobs already utilizing the resource, in one embodiment. [0021] In one embodiment, the RBS 100 schedules computer jobs for a variety of different resources. The different resources may or may not be on the same computer system as the RBS 100. For example, the RBS 100 schedules computer jobs to utilize a resource that is accessed via a network, in one embodiment. As a particular example, the resource may be accessed via the Internet. [0022] Thus, in one embodiment, the RBS 100 receives computer jobs seeking access to a variety of resources and makes scheduling decisions as to which of the computer jobs should be scheduled to utilize which of the resources, as well as when the computer jobs should be scheduled. Computer Resource Utilization [0023] The RBS 100 inputs resource utilization information 105, which describes utilization of various computer resources. The utilization may pertain to an interval or a specified point. The utilization may be an average utilization over a specified interval. For example, a resource's utilization can be specified as an average utilization over a -4- WO 2007/149224 PCT/US2007/013394 specified time interval. As a particular example, a CPU resource at 30 percent utilization may pertain to average utilization over a recent time interval. The interval is measured in quanta, in one embodiment. [0024] The utilization is numeric-based, in one embodiment. For example, the utilization might be based on the number of operations waiting to execute at a specified point in time. For example, if a resource has "x" operations waiting to execute, the utilization could be "x". As another example of a numeric based utilization of a resource, the utilization is based on the number of requests to utilize a particular resource. For example, the utilization is based on the total number of requests that each process has to utilize a particular resource, in one embodiment. Numeric based utilization may pertain to a point or an interval. For example, numeric based resource utilization is based on an average number of operations waiting to execute over a particular time interval, in one embodiment. In another embodiment, numeric based resource utilization is based on the number of operations waiting to execute at a particular point. [0025] In one embodiment, the RBS 100 estimates future resource utilization, based on the resource utilization 105. For example, based on the number of requests to utilize a particular resource (and perhaps other factors), an estimate is made of future resource utilization. As another example, based on the percentage utilization of a particular resource (and perhaps other factors), an estimate is made of future resource utilization. Computer Job Utilization Criteria [0026] The RBS 100 also inputs utilization criteria 118 for the jobs to be scheduled. The utilization criteria 118 pertain to the resources. For example, resource thresholds may be used, wherein the RBS 100 only schedules a computerjob if resource utilization by other jobs is below the threshold. An example of using resource thresholds is that the RBS 100 only schedules a particular computer job to use disk 1/0 if the disk 1/0 has an available capacity of less than 60 percent. The RBS 100 has an Application Program Interface (API) for an application that owns the computer job to provide utilization criteria 118, in one embodiment. An example API is provided herein. [0027] The utilization criteria 118 are based on time, in one embodiment. Basing utilization criteria 118 on a percentage is an example of a time-based criteria. The utilization criteria 118 are numeric based, in one embodiment. An example of numeric based utilization criteria 118 is the number of operations that are waiting to execute on a particular resource. Another example of numeric based utilization criteria 118 is a number of requests that are received to use a resource. For example, the utilization criterion 118 of a particular process is based on the number of requests to use a particular resource. -5- WO 2007/149224 PCT/US2007/013394 [00281 The RBS 100 stores historic utilization information 116, in one embodiment. The historic utilization information 116 describes prior resource utilization by one or more of the jobs. For example, the fact that a computer job utilized 30 percent of the network 1/0 resource is stored for future reference. The RBS 100 uses the historic utilization information 116 to determine utilization criteria for a computer job to be scheduled, in one embodiment. For example, based on one or more previous times that the computer job utilized a particular resource, the RBS 100 determines that a computer job has a utilization criterion of "x" percent for the particular resource. The RBS 100 then uses this utilization criterion when scheduling the computer job. 10029] The RBS 100 also inputs executable code 108 of the jobs to be scheduled, which the RBS 100 analyzes to determine utilization criteria. DEFERRING SCHEDULING DECISIONS [00301 In one embodiment, the determination as to whether a computer job should be allowed to utilize a particular resource may be deferred. For example, a particular computer job may have a utilization criterion that allows a specified number of requests from other computer jobs to be serviced prior to the resource based scheduler even considering whether to schedule the particular computer job. In one embodiment, the RBS defers execution of the particular computer job if the utilization of the resource is above a threshold. The particular computer job specifies the number of requests from other computer jobs that can be serviced prior to scheduling the particular computer job. In one embodiment, after the number of requests has been serviced, the particular computer job is scheduled next. In another embodiment, after requests have been serviced, the RBS determines whether or not to allow the particular computer job to have the resource. [00311 In one embodiment, the resource based scheduler has a normal worklist for resource requests that should be satisfied without delay and a deferred worklist for resource requests that can be deferred. Each entry in the deferred list may be stamped with a current request number when the request is put on the deferred list. These deferred requests may be ordered based on which deferred request should be serviced first. In one embodiment, there are multiple deferred worklists. When the RBS determines to start a new request, the RBS might first look at the deferred list to see if any requests are expiring and then take them next, versus the normal list. -6- WO 2007/149224 PCT/US2007/013394 RESOURCE BASED SCHEDULING WORKLISTS [0032] The RBS 100 has a resource-based scheduling worklist 120 for each resource for which a computer job might be scheduled to use, in one embodiment. For example, the RBS 100 has a CPU resource worklist 120(1), a disk I/O worklist 120(2), a network I/O worklist 120(3) and other resource worklists 120(n). Examples of other resources worklists include, but are not limited to, a network resource worklist, a video resource worklist, a keyboard resource worklist. Each worklist 120 comprises jobs that are waiting to utilize the resource corresponding to that resource. So as to not obscure the diagram, not all possible resource-based scheduling worklists are depicted in FIG. 1A. In one embodiment, a particular worklist is for computer-jobs that utilize a combination of resources. In one embodiment, there are two or more worklists for a particular resource, wherein each worklist corresponds to a different priority. For example, computer jobs with a high priority go into one worklist, a medium priority on another, etc. A worklist may be ordered or not ordered. [00331 In one embodiment, the RBS 100 determines which worklist to place a computer job on based on analysis of the executable code 108. For example, the RBS 100 examines instructions in the executable code 108 of the computer job to determine what resources the computer job needs, in one embodiment. [00341 In one embodiment, when a sufficient amount of a computer resource is available, the RBS 100 gives the resource to one of the computer jobs in the worklist corresponding to that resource. For example, the scheduling logic 112 selects one of the computer jobs on the worklist based on available capacity of the resource and utilization criteria of the computer jobs. Other selection criteria can be used. The selection criteria include, but are not limited to, order in which jobs were added to the worklist, computer job priority (e.g., process priority, thread priority), and matching the resource's available capacity with resource needs of the jobs. The selection may be based on any combination of these criteria, as well as other criteria not specifically mentioned. RESOURCE BASED SCHEDULER WITHOUT RESOURCE-BASED SCHEDULING WORKLISTS [0035] In one embodiment, the RBS 100 does not use scheduling worklists that correspond to the various computer resources. Referring to FIG. IB, the RBS 100 receives requests to schedule computerjobs on an ongoing basis, in no particular order. For example, as an application desires to have a computer job or jobs executed, the application sends a request to the RBS 100 to schedule one or more computer jobs. The -7- WO 2007/149224 PCT/US2007/013394 RBS 100 determines or is informed what resource or resources are to be used by a particular computer job. Examples of resources include a processor 304, storage device 310, display 312, input device 314, and network communication interface 318, and a network resource 182 that is accessed via network 184. [0036] The RBS 100 determines or is informed as to the utilization of the particular resource. The RBS 100 also determines or is informed as to utilization criteria the computer job has pertaining to the particular resource. Based on utilization of the particular resource and the utilization criteria, the RBS 100 determines whether to schedule the particular computerjob to utilize the particular resource. For example, if the utilization of the network communication interface 318 meets the utilization criteria of a computerjob, the RBS 100 schedules the computer job to execute using the network communication interface 318. If not, the RBS 100 does not schedule the computer job to use the network communication interface 318. Rather, the RBS 100 may wait and schedule the computer job when utilization of the network communication interface 318 meets the utilization criteria of a computer job. PRIORITIZING RESOURCES [0037] The computer resources are prioritized for scheduling purposes, in one embodiment. For example, CPU, disk I/O, network I/O and other resources may be ranked based on relative speed of a resource. For example, a disk I/O might be designated as a slower resource than a CPU resource and therefore be given a higher priority. Similarly the network 1/0 resource may be faster than the disk 1/0 but slower than the CPU. The network resource would thus be given higher priority than the CPU but less than the disk 1/0. If the RBS 100 takes into account slower resources and schedules these resources with a higher priority, then delay to faster resources such as a CPU may be minimized. APPLICATION BASED EXAMPLE [0038] The following is an example of resource-based scheduling in an application. As examples, the application may be a defragmenter or a virus scanner. In this example, Computer job A is at the top of the disk I/O scheduling worklist 120(2) and has a utilization criterion of 60 percent disk 1/0 available capacity and Computer job B, which is next on the disk I/O scheduling worklist 120(2), has a utilization criterion of 20 percent disk I/O available capacity. If the disk I/O has 30 percent available capacity, then the RBS 100 would not schedule Computer job A because the disk 1/0 does not have enough -8- WO 2007/149224 PCT/US2007/013394 available capacity. However, Computer job B could be scheduled. The RBS 100, in this case, makes use of the 30 percent disk I/O available capacity by scheduling an appropriate computer job (Computer job B) to utilize the disk I/O instead of wasting it. If the RBS 100 had given the disk I/O resource to computer job A, which required 60 percent disk 1/0, this may have caused a computer job collision as more than one hundred percent of the disk I/O resource would have been allocated. KERNEL LEVEL RESOURCE BASED SCHEDULER [00391 In one embodiment, the RBS 100 is implemented in an O/S at the kernel level. In such an embodiment, the kernel level RBS 100 has full knowledge of the entire worklist of computer jobs (e.g., threads) that are requesting execution. Thus, the RBS 100 can order the execution based on resource availability without the need to determine the percentage utilization of any resource. This is because the RBS 100 already has full control to schedule the various resources. There is no need to measure, it already knows as it is the one allocating the resources. PROCESS FLOW [0040] FIG. 2 is a flowchart illustrating steps of a process 200 for resource-based scheduling, in accordance with an embodiment of the present invention. The steps of process 200 are described in a particular order for convenience of explanation. However, the steps may occur in a different order. Moreover, steps may be repeated many times. In step 202, the RBS 100 receives utilization criteria of one or more jobs. For example, an application program provides the utilization criteria to the RBS 100. The application can specify different criteria for different computer jobs. It is not necessary for the application to specify the utilization criteria for its computer jobs. [0041] The application sends parameters to the RBS 100 to control resource utilization, in accordance with an embodiment of the present invention. Control of resource utilization includes, but is not limited to; disk I/O, CPU and network. For example, the application can request a computer job be executed pending any combination of threshold levels of the above three resources, or other resources. Moreover, the application can specify different resource threshold levels for different computer jobs. For example, the application specifies a different resource threshold level with each computer job, in accordance with one embodiment. Therefore, fine-grained resource management is possible. -9- WO 2007/149224 PCT/US2007/013394 [00421 In step 204, the RBS 100 determines utilization criteria of one or more jobs. In one embodiment, the RBS 100 determines expected utilization of a particular resource by a computer job, wherein the utilization criterion is based on the expected resource utilization. In one embodiment, the RBS 100 determines the expected utilization of a particular computer job by examining instructions in the particular computer job. In one embodiment, the RBS 100 bases the expected utilization on a stored value that describes previous utilization of the resource by the particular computer job. [00431 In step 206, computer jobs are added to resource-based scheduling worklists. For example, a particular process, or at least threads of a particular process, are placed on at least one resource-based scheduling worklist. As more jobs request to be scheduled, they are added to the resource-based scheduling worklists. In one embodiment, requests from many different computer jobs are placed on the worklist. In one embodiment, the requests can be satisfied by a computer job being scheduled to utilize a particular resource. In one embodiment, a computer job is inserted to a location on the worklist based on the priority of the computer job. Because the order in which threads of a process execute may be important, computer jobs (e.g., threads) are located on the worklist to preserve desired execution order, in one embodiment. 100441 In step 208, a particular computer resource is selected to be utilized. For example, the RBS 100 may determine that a computer resource that has been designated as slower should have a higher priority. As a particular example, the RBS 100 selects the disk 1/0 resource. However, the selection can be based on other factors such as thread flow, computer job priority, (e.g., process priority, thread priority), the number ofjobs (e.g., threads) waiting to execute on each worklist and the availability of other resources required by ajob (e.g., thread or process). [00451 In step 210, the RBS 100 determines utilization of the selected computer resource. The utilization may be based on activity over a recent interval. For example, the utilization may be based on any convenient measure such as, the last "x" quanta, or a recent time period. However, the utilization does not need to be based on an interval. For example, the utilization might be based on the number of operations that are waiting to execute at a particular point in time. [00461 When the RBS 100 calculates resource utilization, it is the resource utilization ofjobs other than the jobs associated with a particular application that is measured, in accordance with one embodiment of the present invention. The following example in which the CPU utilization threshold is set to 20 percent is used to illustrate. If CPU utilization is below 20 percent prior to allowing computer jobs associated with a -10- WO 2007/149224 PCT/US2007/013394 particular application to execute, CPU utilization may increase to over 20 percent when the computer jobs execute. This increase beyond 20 percent is not considered a CPU resource utilization violation, in this example. Similar principles may apply to network, disk I/O resources, and other resources. [00471 In one embodiment, the RBS 100 estimates utilization of the selected computer resource over a future time interval. The RBS 100 may estimate future utilization based on recent utilization of the resource. As a particular example, recent utilization of a network I/O is measured (or otherwise learned). Based on the recent network I/O utilization, an estimate is made of network I/O utilization for the near future. The estimate may be based on other factors as well. [00481 In step 212, The RBS 100 selects one of the computer jobs on the worklist corresponding to the selected resource. The RBS 100 schedules the selected computer job for execution to use the selected computer resource. The RBS 100 makes the selection based on the utilization of the particular resource and the utilization criterion of at least one of the computer jobs on the worklist for the particular computer resource. For example, if the utilization of the selected resource is 60 percent and the utilization criterion of a particular computer job is that the selected resource has less than 40 percent utilization, then the particular computer job is not scheduled, as the utilization criterion is not satisfied. In this case, the RBS 100 could select another computer job and schedule that computer job if its utilization criterion allows the selected resource to have 60 percent utilization. [00491 By repeating at least some of the steps of process 200 in order to schedule one computer job after another, the RBS determines a scheduling order for computer jobs. [0050] In step 214, the RBS 100 stores resource utilization information for the computer job that just executed. The RBS 100 may later use this resource utilization information to determine utilization criteria for the computer job. MICRO-JOBS [0051] In one embodiment, the RBS 100 schedules micro-jobs. Micro-jobs have a size that allows a particular micro-job to complete within an allotted time for which the particular micro-job owns a resource used to execute the processing job, in one embodiment. In one embodiment, each micro-job is such a size that it will complete within its allotted time. However, it may be that some of the micro-jobs are too large to complete execution within their allotted time. -11- WO 2007/149224 PCT/US2007/013394 [00521 In one embodiment, the allotted time is a quantum. A quantum is a time slice given to a portion of computer code (e.g., a thread) during which time that code portion owns the CPU resource. Different operating systems used different quanta. Moreover, the quantum assigned to a particular code portion may change based on circumstances during runtime. For example, an O/S might increase or decrease the size of the quantum allotted to a thread. In one embodiment, a computer job is divided into micro-jobs based on the size of the quantum that is expected to be allocated to the computer job. In another embodiment, a computer job is divided into micro-jobs based on the size of the quantum that has been allocated to the computer job. The determination as to what portions of the computer job should be split off as micro-jobs may be made either prior to runtime or during runtime. [00531 The micro-jobs are substantially smaller (for example, the smallest) work units that can be completed as a single unit while safely allowing for a pause in execution until the next micro-job executes, in accordance with one embodiment. By safely allowing for a pause in execution, it is meant that the execution of a particular micro-job can be delayed without affecting the outcome that results from execution of the all of the micro jobs. [0054] A micro-job may be a part of a thread. For example, a thread may be divided into multiple micro-jobs. These micro-jobs may be scheduled similar to how a thread is scheduled. However, as previously stated, a micro-job will complete its execution if allowed to execute for a quantum or other time period for which it owns a processing resource, in one embodiment. [00551 A micro-job may only need a very small amount of resources (e.g., CPU time, memory allocation) at any one time. Such minimal use of resources at any one time may result in a stealthy process. Keeping the micro-jobs small allows the computer job to use only a small amount of computer resources at one time. Thus, execution of a micro-job consumes a sufficiently small amount of resources so as to not significantly impact performance of other applications in the computer system, in accordance with one embodiment of the present invention. EXAMPLE API [0056] An embodiment of the present invention is an API for allowing an application to specify various resource utilization parameters. The RBS 100 has such an API, in one embodiment. Applications can use the API to specify utilization criteria for computer jobs (e.g., processes, threads, micro-jobs, or other code portions). The example API has the following resource threshold parameters for CPU, disk, and network. -12- WO 2007/149224 PCT/US2007/013394 " CPU Utilization threshold " Pending Disk 1/0 Count threshold e Network Utilization threshold [00571 The above parameters can be specified for each computer job. For example, for a computer job that uses the network, a network threshold may be used. However, the network threshold could be zero for computer jobs that do not use the network. Thus, fine-grained resource management is provided for, in accordance with an embodiment of the present invention. [00581 As a particular example, the application can request that a particular computer job be executed only if the CPU utilization is below 50 percent, and the 1/0 Disk Utilization is below 40 percent, and network traffic is below 60 percent. Any combination of the resource threshold factors can be used, including none at all. The CPU utilization threshold differentiates between RBS use of the CPU as opposed to that of any otherjob, in accordance with an embodiment of the present invention. [0059] The following two parameters are used to specify an interval over which utilization should be measured. " CPU Utilization Window e Network Utilization Window [0060] The CPU Utilization Window parameter defines a time window over which CPU utilization is calculated. For example, CPU utilization over the last n milliseconds is averaged. The network utilization window defines a time window over which network utilization is calculated. These parameters may be internal to the RBS 100. However, an application may override these parameters. The pending disk 1/0 is absolute at any point in time and thus it does not have to be calculated. [00611 A mandatory idle time parameter may be passed from the application to the RBS to control how the computer jobs are spread out over time. The mandatory idle time parameter is optional. Furthermore, when used, the mandatory idle parameter may have a value of zero. e Mandatory Idle Time -13- WO 2007/149224 PCT/US2007/013394 [0062] The RBS keeps track of "idle time," which is defined as system idle time after all computer jobs have executed. For example, application(s) can worklist up computer jobs with the RBS. When there are no computer jobs on the resource-based worklists 120, the RBS waits for the specified Mandatory Idle Time and then wakes up and authorizes the application(s) to perform additional work. For example, a defragmenter might first execute a number of computer jobs to defragment a disk drive, then be paused by the RBS computer job scheduler. After the specified Mandatory Idle Time, the RBS calls the defragmenter to authorize additional work. For example, the defragmenter might execute a clean-up job, such as releasing memory. Mandatory Idle Time can be a default parameter that can be adjusted by an application. [0063] The following parameters relate to waiting to execute a computer job when resource utilization is above a threshold. * Wait Time " Maximum Wait Time [0064] If the RBS determines that resource utilization is currently too high to execute a computer job, the RBS waits for the specified Wait Time and then re-checks resource utilization. The Wait Time parameter can be increased each time the RBS determines that resource utilization is too high. For example, the RBS can increase the Wait Time parameter until the Max Wait Time is reached. These parameters can be specified by the application when it is first started. An application can adjust these parameters during its run time. [0065] While the example API uses time (e.g., ms) to specify various parameters, other measures such as quanta may be used. VARIATIONS [0066] The RBS 100 is a part of an operating system, in one embodiment. In another embodiment, the RBS 100 is a stand-alone application that facilitates scheduling of computer jobs for other applications. In still another embodiment, the RBS 100 is integrated into an application program and schedules for that particular application. For example, the RBS 100 may be integrated into a disk defragmentation program or a virus detection program. -14- WO 2007/149224 PCT/US2007/013394 [00671 If the RBS 100 executes outside of the operating system, the RBS 100 self limits in its own resource utilization, in one embodiment. For example, the RBS 100 monitors its own resource utilization and if its own resource utilization gets too high, the RBS 100 makes a request to the operating system to stop scheduling the RBS 100 for a period of time. FURTHER EXAMPLES [0068] The following examples illustrate ways in which the RBS 100 may operate. However, the RBS 100 is not required to operate as described in these examples. As one example, the RBS 100 schedules multiple computer jobs for a particular application. Some of these computer jobs may require disk 1/0. The RBS 100 may analyze disk utilization as a condition for scheduling the computer jobs. If disk utilization is too high, then the RBS 100 waits until disk utilization drops to schedule a particular computer job of the application. The RBS 100 continues to monitor the disk 1/0 utilization, and allows another computer job to be scheduled if no other application is seeking access to disk 1/0. However, if another application seeks utilization of disk 1/0, then the RBS 100 does not allow another computer job to be scheduled, in this embodiment. Thus, other application(s) can utilize the disk I/O. [00691 As another example, the RBS 100 may analyze network activity. If network traffic is too high, the RBS 100 will not schedule any computer job of the application until traffic slows. If network traffic is low enough, then the RBS 100 schedules a computer job for execution. The RBS 100 continues to check to make sure that network traffic stays low enough. If network traffic stays low enough, another computer job may be scheduled. However, if traffic gets too high, no further computer jobs are scheduled to execute. In these last two examples, the computer job may be a micro-job. However, that is not required. HARDWARE OVERVIEW [00701 FIG. 3 is a block diagram that illustrates a computer system 300 upon which an embodiment of the invention may be implemented. Steps of process 200 are stored as instructions one or more of the computer-readable media of system 300 and executed on the processor of computer system 300. Computer system 300 includes a bus 302 or other communication mechanism for communicating information, and a processor 304 coupled with bus 302 for processing information. Computer system 300 also includes a main memory 306, such as a random access memory (RAM) or other dynamic storage device, -15- WO 2007/149224 PCT/US2007/013394 coupled to bus 302 for storing information and instructions to be executed by processor 304. Main memory 306 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 304. Computer system 300 further includes a read only memory (ROM) 308 or other static storage device coupled to bus 302 for storing static information and instructions for processor 304. A storage device 310, such as a magnetic disk or optical disk, is provided and coupled to bus 302 for storing information and instructions. The computer system 300 can have any number of processors 304. For example, computer system 300 is a multi-processor system, in one embodiment. The processor 304 can have any number of cores. In one embodiment, the processor 304 is a multi-core processor 304. Computer system 300 can be used in a hyper threaded machine. [0071] Computer system 300 may be coupled via bus 302 to a display 312, such as a cathode ray tube (CRT), for displaying information to a computer user. An input device 314, including alphanumeric and other keys, is coupled to bus 302 for communicating information and command selections to processor 304. Another type of user input device is cursor control 316, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 304 and for controlling cursor movement on display 312. This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane. [0072] The invention is related to the use of computer system 300 for implementing the techniques described herein. According to one embodiment of the invention, those techniques are performed by computer system 300 in response to processor 304 executing one or more sequences of one or more instructions contained in main memory 306. Such instructions may be read into main memory 306 from another machine-readable medium, such as storage device 310. Execution of the sequences of instructions contained in main memory 306 causes processor 304 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention. Thus, embodiments of the invention are not limited to any specific combination of hardware circuitry and software. [00731 The term "machine-readable medium" as used herein refers to any medium that participates in providing data that causes a machine to operate in a specific fashion. In an embodiment implemented using computer system 300, various machine-readable media are involved, for example, in providing instructions to processor 304 for execution. Such a medium may take many forms, including but not limited to, non-volatile media, -16- WO 2007/149224 PCT/US2007/013394 volatile media, and transmission media. Non-volatile media includes, for example, optical or magnetic disks, such as storage device 310. Volatile media includes dynamic memory, such as main memory 306. Transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise bus 302. Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infrared data communications. All such media must be tangible to enable the instructions carried by the media to be detected by a physical mechanism that reads the instructions into a machine. [0074] Common forms of machine-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD ROM, any other optical medium, punchcards, papertape, any other physical medium with patterns of holes, a RAM, a PROM, an EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read. [00751 Various forms of machine-readable media may be involved in carrying one or more sequences of one or more instructions to processor 304 for execution. For example, the instructions may initially be carried on a magnetic disk of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 300 can receive the data on the telephone line and use an infrared transmitter to convert the data to an infrared signal. An infrared detector can receive the data carried in the infrared signal and appropriate circuitry can place the data on bus 302. Bus 302 carries the data to main memory 306, from which processor 304 retrieves and executes the instructions. The instructions received by main memory 306 may optionally be stored on storage device 310 either before or after execution by processor 304. [0076] Computer system 300 also includes a communication interface 318 coupled to bus 302. Communication interface 318 provides a two-way data communication coupling to a network link 320 that is connected to a local network 322. For example, communication interface 318 may be an integrated services digital network (ISDN) card or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 318 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links may also be implemented. In any such implementation, communication interface 318 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information. -17- WO 2007/149224 PCT/US2007/013394 [00771 'Network link 320 typically provides data communication through one or more networks to other data devices. For example, network link 320 may provide a connection through local network 322 to a host computer 324 or to data equipment operated by an Internet Service Provider (ISP) 326. ISP 326 in turn provides data communication services through the worldwide packet data communication network now commonly referred to as the "Internet" 328. Local network 322 and Internet 328 both use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on network link 320 and through communication interface 318, which carry the digital data to and from computer system 300, are exemplary forms of carrier waves transporting the information. [00781 Computer system 300 can send messages and receive data, including program code, through the network(s), network link 320 and communication interface 318. In the Internet example, a server 330 might transmit a requested code for an application program through Internet 328, ISP 326, local network 322 and communication interface 318. [0079] The received code may be executed by processor 304 as it is received, and/or stored in storage device 310, or other non-volatile storage for later execution. In this manner, computer system 300 may obtain application code in the form of a carrier wave. [0080] In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details that may vary from implementation to implementation. Thus, the sole and exclusive indicator of what is the invention, and is intended by the applicants to be the invention, is the set of claims that issue from this application, in the specific form in which such claims issue, including any subsequent correction. Any definitions expressly set forth herein for terms contained in such claims shall govern the meaning of such terms as used in the claims. Hence, no limitation, element, property, feature, advantage or attribute that is not expressly recited in a claim should limit the scope of such claim in any way. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. -18-

Claims (41)

1. A machine-implemented method comprising the steps: based on utilization of a particular resource and a utilization criterion pertaining to the particular resource and associated with a particular computer job, determining whether to schedule the particular computerjob to utilize the particular resource.

2. The method of Claim 1, further comprising: determining expected utilization of the particular resource by the computer job, wherein the utilization criterion is based on the expected utilization.

3. The method of Claim 2, wherein the step of determining expected utilization comprises examining instructions in the computer job.

4. The method of Claim 2, wherein the step of determining expected utilization comprises: accessing a value that describes previous utilization of the particular resource by the computer job; and basing the expected utilization on the stored value that describes previous utilization.

5. The method of Claim 1, further comprising receiving the utilization criterion from an application program owning the computer job.

6. The method of Claim 1, further comprising placing the computer job on at least one of a plurality of resource-based scheduling worklists.

7. The method of Claim 1, further comprising: for each of a plurality of computer jobs to be scheduled for execution, placing the computer job on at least one of a plurality of resource-based scheduling worklists that correspond to a plurality of computer resources, and wherein -19- WO 2007/149224 PCT/US2007/013394 a given resource-based scheduling worklist comprises computer jobs that are waiting to utilize the given computer resource.

8. The method of Claim 7, further comprising: selecting a particular computer resource to be utilized by one of the plurality of computer jobs.

9. The method of Claim 1, wherein determining whether to schedule the particular computer job to utilize the particular resource is performed by an operating system.

10. The method of Claim 1, wherein determining whether to schedule the particular computer job to utilize the particular resource is performed by a resource-based scheduler outside of an operating system.

11. The method of Claim 1, further comprising determining the utilization of the particular resource.

12. The method of Claim 1, further comprising estimating projected utilization of the particular resource over an interval.

13. The method of Claim 12, wherein determining whether to schedule the particular computer job to utilize the particular resource is based on the projected utilization of the particular resource over the interval and the utilization criteria.

14. The method of Claim 1, wherein the step of determining whether to schedule the particular computer job to utilize the particular resource is further based on utilization of another resource and a utilization criterion associated with the particular computer job that pertains to the other resource.

15. The method of Claim 1, wherein the utilization of the particular resource is based on an amount of time of utilization of the particular resource.

16. The method of Claim 1, wherein the utilization of the particular resource is based on a number of requests to use the particular resource. -20- WO 2007/149224 PCT/US2007/013394

17. A machine-implemented method comprising the steps: placing each of a plurality of computer jobs on at least one of a plurality of resource-based scheduling worklists that correspond to a plurality of computer resources, wherein a given resource-based scheduling worklist comprises computer jobs each having a utilization criterion pertaining to the given resource; selecting a particular computer resource to be utilized by one of the plurality of computer jobs; and based on utilization of the particular computer resource and the utilization criterion of at least one of the computer jobs on the worklist for the particular computer resource, selecting one of the computer jobs to utilize the particular computer resource.

18. The method of Claim 17, further comprising assigning a priority to each of the plurality of computer resources.

19. The method of Claim 18, wherein the step of selecting a particular computer resource to be utilized by one of the plurality of computer jobs is based on the priority assigned to each of the plurality of computer resources.

20. The method of Claim 18, wherein assigning a priority to the plurality of computer resources is based on relative speed of the plurality of computer resources.

21. The method of Claim 17, wherein the step of selecting one of the computer jobs to utilize the particular computer resource is further based on a priority at least one of the computer jobs on the worklist for the particular computer resource.

22. The method of Claim 17, wherein selecting one of the computer jobs to utilize the particular computer resource during the interval comprises: identifying one of the computerjobs on the worklist for the particular resource; and if the estimated utilization of the particular resource indicates sufficient available capacity to satisfy the utilization criterion of the particular computer job, scheduling the particular computer job for execution using the particular resource. -21- WO 2007/149224 PCT/US2007/013394

23. The method of Claim 17, further comprising: determining expected utilization of the particular resource by a first computer job, wherein the utilization criterion for first computer job is based on the expected utilization.

24. The method of Claim 23, wherein the step of determining expected utilization comprises examining instructions in the first computerjob.

25. The method of Claim 23, wherein the step of determining expected utilization comprises: accessing a value that describes previous utilization of the particular resource by the first computer job; and basing the expected utilization on the stored value that describes previous utilization.

26. The method of Claim 17, further comprising receiving the utilization criterion from an application that owns the first computer job.

27. The method of Claim 17, wherein selecting one of the computer jobs to utilize the particular computer resource is performed by an operating system.

28. The method of Claim 17, wherein the step of selecting one of the computer jobs to utilize the particular computer resource is performed by a resource-based scheduler outside of an operating system.

29. The method of Claim 17, further comprising determining the utilization of the particular resource.

30. The method of Claim 17, wherein the step of determining whether to schedule the particular computer job to utilize the particular resource is further based on utilization of another resource and a utilization criterion associated with the particular computer job that pertains to the other resource. -22- WO 2007/149224 PCT/US2007/013394

31. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more computer processors, cause the one or more computer processors to carry out the steps of: based on utilization of a particular resource and a utilization criterion pertaining to the particular resource and associated with a particular computer job, determining whether to schedule the particular computer job to utilize the particular resource.

32. The method of Claim 31, wherein the step of determining whether to schedule the particular computer job to utilize the particular resource is further based on utilization of another resource and a utilization criterion associated with the particular computer job that pertains to the other resource.

33. A system, comprising: one or more computer processors, and; a computer-readable medium communicatively coupled to the one or more computer processors, wherein the computer-readable medium has stored thereon one or more stored sequences of instructions which, when executed by the one or more computer processors, cause the one or more computer processors to perform: based on utilization of a particular resource and a utilization criterion pertaining to the particular resource and associated with a particular computer job, determining whether to schedule the particular computer job to utilize the particular resource.

34. The method of Claim 33, wherein the step of determining whether to schedule the particular computer job to utilize the particular resource is further based on utilization of another resource and a utilization criterion associated with the particular computer job that pertains to the other resource.

35. A machine-implemented method comprising the steps: receiving requests from a plurality of computer jobs, wherein each request requires utilization of a particular resource to satisfy the request, and -23- WO 2007/149224 PCT/US2007/013394 based on utilization of the particular resource and utilization criterion that each of the computer jobs have pertaining to the particular resource, determining a scheduling order for the computer jobs to utilize the particular resource.

36. The method of Claim 35, wherein at least a portion of the requests require utilization of another resource to satisfy the request; and wherein the step of determining a scheduling order for the computer jobs to utilize the particular resource is further based on utilization criterion that the computer jobs associated with the portion of requests have pertaining to the other resource.

37. The method of Claim 35, wherein the utilization of the particular resource is based on an amount of time of utilization of the particular resource.

38. The method of Claim 35, wherein the utilization of the particular resource is based on the number of requests that require utilization of the particular resource to satisfy the request.

39. The method of Claim 35, wherein determining a scheduling order for the computerjobs to utilize the particular resource is performed by an operating system.

40. The method of Claim 35, wherein determining a scheduling order for the computer jobs to utilize the particular resource is performed by a resource-based scheduler outside of an operating system.

41. A machine-implemented method comprising the steps: receiving requests from a plurality of computer jobs, wherein each request requires utilization of one or more of a plurality of resources to satisfy the request, selecting a particular resource to have a computer job scheduled to utilize; and based on utilization of the particular resource and utilization criterion that each of the computer jobs that require utilization of the particular resource have pertaining to the particular resource, determining a scheduling order for the computer jobs that require utilization of the particular resource. -24-