CN109189581B - Job scheduling method and device - Google Patents
Job scheduling method and device Download PDFInfo
- Publication number
- CN109189581B CN109189581B CN201811102507.2A CN201811102507A CN109189581B CN 109189581 B CN109189581 B CN 109189581B CN 201811102507 A CN201811102507 A CN 201811102507A CN 109189581 B CN109189581 B CN 109189581B
- Authority
- CN
- China
- Prior art keywords
- virtual machine
- idle
- target
- overloaded
- queue
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000012545 processing Methods 0.000 claims abstract description 85
- 230000008569 process Effects 0.000 claims abstract description 18
- 230000002618 waking effect Effects 0.000 claims description 5
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
Abstract
The embodiment of the invention discloses a job scheduling method and a job scheduling device, wherein when an overload virtual machine and an idle virtual machine appear in the task processing process, the overload virtual machine in an overload virtual machine queue can request the job scheduler to allocate the idle virtual machine from the idle virtual machine queue to assist the overload virtual machine to complete the rest of jobs according to a preset rule through an auxiliary processing request sent to the job scheduler. Therefore, even if part of the virtual machines are overloaded in the process of processing the tasks, the idle virtual machines which have completed the tasks can be flexibly and dynamically scheduled by the task scheduler to assist the overloaded virtual machines to complete the rest of the tasks, so that the resources of the virtual machines are fully utilized, the task processing time is saved, the task processing efficiency is improved, and the user experience is improved.
Description
Technical Field
The invention relates to the technical field of cloud computing, in particular to a job scheduling method and device.
Background
At present, resource (including network, server, storage, application software and service) platforms can provide various useful and convenient services for users, for example, cloud computing service platforms, which can rapidly provide the needed resources for users with little management effort or little interaction with service providers.
However, a resource platform, such as a cloud computing service platform, generally responds, allocates, and retransmits tasks received by the resource platform through a job scheduler. Specifically, when the resource platform receives a task request, the job scheduler needs to allocate a plurality of jobs in the task to each virtual machine for processing according to the load capabilities (e.g., memory size, processor configuration, and other capabilities of the virtual machine) of the current plurality of virtual machines.
However, when processing the task, some virtual machines may complete the allocated job early, and some virtual machines may cause an overload phenomenon (that is, a ratio between the current remaining number of jobs of the virtual machine and the total number of jobs allocated to the virtual machine exceeds a preset threshold) due to a slow speed of processing the job or a failure, so that the resource platform is prone to have a long time for processing the task, low efficiency, or even unable to complete the task due to the overload phenomenon of the part of virtual machines.
Therefore, it is desirable to provide a method for flexibly and dynamically scheduling jobs, so as to improve the processing efficiency of a resource platform, such as a cloud computing service platform, on tasks.
Disclosure of Invention
In order to solve the above technical problems, embodiments of the present invention provide a job scheduling method and apparatus, so that even if some virtual machines are overloaded during a task processing process, a job scheduler flexibly and dynamically schedules idle virtual machines that have completed a job to assist the overloaded virtual machines to complete the remaining job, thereby fully utilizing resources of the virtual machines, saving task processing time, and improving task processing efficiency, thereby improving user experience.
In a first aspect, a job scheduling method is provided, including:
responding to an auxiliary processing request sent by a target overload virtual machine in an overload virtual machine queue, and acquiring an idle virtual machine in an idle virtual machine queue;
determining a target idle virtual machine corresponding to the target overload virtual machine from the idle virtual machines according to a preset rule, and awakening the target idle virtual machine;
and controlling the target idle virtual machine to assist the target overload virtual machine to complete the rest operation.
Optionally, the overloaded virtual machines in the overloaded virtual machine queue are sorted according to the number of the remaining jobs; and the idle virtual machines in the idle virtual machine queue are sorted according to the morning and evening of the idle time;
the preset rule is as follows: and allocating the idle virtual machines with the idle time from morning to evening to the overloaded virtual machine according to the sequence of the number of the remaining jobs.
Optionally, the auxiliary processing request includes a sequence number of the target overloaded virtual machine in the overloaded virtual machine queue.
Optionally, the determining, according to a preset rule, a target idle virtual machine corresponding to the target overloaded virtual machine from the idle virtual machines includes:
analyzing the auxiliary processing request to obtain a first sequence number of the target overload virtual machine in the overload virtual machine queue;
and searching the idle virtual machine with the sequence number being the first sequence number from the idle virtual machine queue, and recording the idle virtual machine as the target idle virtual machine.
Optionally, the controlling the target idle virtual machine to assist the target overloaded virtual machine to complete the remaining job includes:
allocating a first residual job with a preset proportion in the residual jobs of the target overload virtual machine to the target idle virtual machine for processing; wherein the remaining jobs include the first remaining job and a second remaining job;
and controlling the target overload virtual machine to process the second residual operation.
Optionally, after waking up the target idle virtual machine, the method further includes:
and removing the target idle virtual machine from the idle virtual machine queue, and updating the idle virtual machine queue.
Optionally, the condition for responding to the auxiliary processing request includes: and the number of idle virtual machines in the idle virtual machine queue is not less than the number of overloaded virtual machines in the overloaded virtual machine queue.
In a second aspect, there is also provided a job scheduling apparatus, including:
the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for responding to an auxiliary processing request sent by a target overload virtual machine in an overload virtual machine queue and acquiring an idle virtual machine in an idle virtual machine queue;
a determining unit, configured to determine, according to a preset rule, a target idle virtual machine corresponding to the target overloaded virtual machine from the idle virtual machines, and wake up the target idle virtual machine;
and the control unit is used for controlling the target idle virtual machine to assist the target overload virtual machine to finish the rest operation.
Optionally, the overloaded virtual machines in the overloaded virtual machine queue are sorted according to the number of the remaining jobs; and the idle virtual machines in the idle virtual machine queue are sorted according to the morning and evening of the idle time;
the preset rule is as follows: and allocating the idle virtual machines with the idle time from morning to evening to the overloaded virtual machine according to the sequence of the number of the remaining jobs.
Optionally, the auxiliary processing request includes a sequence number of the target overloaded virtual machine in the overloaded virtual machine queue.
Optionally, the determining unit includes:
the analysis subunit is configured to analyze the auxiliary processing request to obtain a first sequence number of the target overloaded virtual machine in the overloaded virtual machine queue;
and the searching subunit is configured to search, from the idle virtual machine queue, an idle virtual machine with a sequence number of the first sequence number, and record the idle virtual machine as the target idle virtual machine.
Optionally, the control unit includes:
the allocation subunit is configured to allocate a first remaining job in a preset proportion of remaining jobs of the target overloaded virtual machine to the target idle virtual machine for processing; wherein the remaining jobs include the first remaining job and a second remaining job;
and the control subunit is used for controlling the target overload virtual machine to process the second residual operation.
Optionally, after waking up the target idle virtual machine, the method further includes:
and the updating unit is used for removing the target idle virtual machine from the idle virtual machine queue and updating the idle virtual machine queue.
Optionally, the condition for responding to the auxiliary processing request includes: and the number of idle virtual machines in the idle virtual machine queue is not less than the number of overloaded virtual machines in the overloaded virtual machine queue.
In the embodiment of the present invention, when an overloaded virtual machine and an idle virtual machine appear in a task processing process, the overloaded virtual machine in the overloaded virtual machine queue may request the job scheduler to allocate the idle virtual machine from the idle virtual machine queue to assist the overloaded virtual machine to complete the remaining job through an auxiliary processing request sent to the job scheduler. Therefore, even if part of the virtual machines are overloaded in the process of processing the tasks, the idle virtual machines which have completed the tasks can be flexibly and dynamically scheduled by the task scheduler to assist the overloaded virtual machines to complete the rest of the tasks, so that the resources of the virtual machines are fully utilized, the task processing time is saved, the task processing efficiency is improved, and the user experience is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments described in the present invention, and it is obvious for those skilled in the art to obtain other drawings according to these drawings.
Fig. 1 is a schematic flowchart of a job scheduling method according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a job scheduling apparatus according to an embodiment of the present invention.
Detailed Description
Currently, resource platforms such as cloud computing provide various convenient services to users, and therefore, these resource platforms are gaining favor of users and are rapidly developing. However, the inventor has found that, taking a cloud computing service platform as an example, as the number of users and the user requirements increase, tasks received by the cloud computing service platform increase in an exponential manner, which requires that a job scheduler can flexibly schedule each job in the tasks to improve task processing efficiency.
Currently, a cloud computing server platform includes a plurality of virtual machines capable of processing jobs, and after the cloud computing server platform receives a task, a job scheduler allocates a plurality of jobs in the task to each virtual machine for processing. However, when processing the task, some virtual machines may complete the allocated task in advance, and some virtual machines may overload due to a slow speed of processing the task or a failure, so that the virtual machines that complete the task in advance need to wait for other virtual machines, especially the overloaded virtual machine, to complete the task before performing the subsequent task, thereby causing a problem that the resource platform has a long task processing time, low efficiency, or even cannot complete the task.
Based on this, in the embodiment of the present invention, when an overloaded virtual machine and an idle virtual machine occur in a task processing process, the overloaded virtual machine in the overloaded virtual machine queue may request, through an auxiliary processing request sent to the job scheduler, that the job scheduler allocates an idle virtual machine from the idle virtual machine queue according to a preset rule to assist the overloaded virtual machine in completing the remaining job. Therefore, even if part of the virtual machines are overloaded in the process of processing the tasks, the idle virtual machines which have completed the tasks can be flexibly and dynamically scheduled by the task scheduler to assist the overloaded virtual machines to complete the rest of the tasks, so that the resources of the virtual machines are fully utilized, the task processing time is saved, the task processing efficiency is improved, and the user experience is improved.
It should be noted that the cloud computing service platform is only one scenario to which the job scheduling method provided in the embodiment of the present invention is applicable, and the description of the job scheduling method of the cloud computing service platform is only to clearly describe the implementation manner of the embodiment of the present invention, and the resource platform to which the embodiment of the present invention can be applied is not limited in detail in the present invention.
The following describes a specific implementation manner of a job scheduling method and apparatus according to an embodiment of the present invention in detail by using an embodiment with reference to the accompanying drawings.
Referring to fig. 1, a flowchart of a job scheduling method according to an embodiment of the present invention is shown. The method specifically comprises the following steps:
It is understood that the overloaded virtual machine refers to a virtual machine in which the ratio of the number of remaining jobs that are currently unprocessed to the total number of jobs that are allocated exceeds a preset threshold. The preset threshold may be a fixed threshold determined by a technician according to experience or experiments, and when a ratio of the number of remaining jobs currently not processed by the virtual machine to the total number of allocated jobs exceeds the preset threshold, the virtual machine is marked as an overloaded virtual machine and added to the overloaded virtual machine queue.
It can be understood that the overloaded virtual machine queue refers to a queue composed of the overloaded virtual machines at the current time, and elements in the queue are all the overloaded virtual machines at the current time.
For example: assuming that the preset threshold is 50%, the total number of jobs assigned to the virtual machine a is 10, and the number of remaining jobs unprocessed by the virtual machine a at time T1 is 7, the ratio of the number of remaining jobs unprocessed by the virtual machine a at time T1 to the total number of assigned jobs is 70%, which exceeds the preset threshold of 50%, and the virtual machine a is added to the overloaded virtual machine queue.
When the number of remaining jobs unprocessed by the virtual machine a is 4 at time T2, the ratio of the number of remaining jobs unprocessed by the virtual machine a to the total number of allocated jobs at time T2 is 40%, the preset threshold value is not exceeded by 50%, and the virtual machine a is removed from the overloaded virtual machine queue.
That is, the overloaded virtual machine is not a fixed name of the virtual machine, but is determined at the present time according to the actual situation of the virtual machine. As the number of jobs remaining for the virtual machine changes, the virtual machine changes from an overloaded virtual machine to a non-overloaded virtual machine, and similarly, the overloaded virtual machine queue also changes dynamically as the state of the overloaded virtual machine changes.
The idle virtual machine refers to all jobs which have been allocated by the job scheduler at the current time, and is marked as an idle virtual machine. The idle virtual machine queue refers to a queue formed by idle virtual machines at the current time, and elements in the queue are all the idle virtual machines at the current time.
In some possible implementation manners, in order that the job scheduler can flexibly and orderly schedule the overloaded virtual machines and the idle virtual machines again, the overloaded virtual machines in the overloaded virtual machine queue can be sorted according to the number of the remaining jobs; and the idle virtual machines in the idle virtual machine queue can also be sorted according to the morning and evening of the idle time.
For example: assuming that the current time is T1, the ratio of the number of remaining jobs unprocessed by the current overloaded virtual machine a, the current overloaded virtual machine B, and the current overloaded virtual machine C to the total number of allocated jobs is: 70%, 80%, and 60%, then the order of the overloaded virtual machines in the overloaded virtual machine queue is: b → A → C; the idle times for currently idle virtual machine D, idle virtual machine E, and idle virtual machine F are t1, t2, and t3, respectively (where t2 > t3 > t1), then the order of the idle virtual machines in the idle virtual machine queue is: f → E → D.
It is to be understood that the secondary processing request is a request generated by an overloaded virtual machine (i.e., a target overloaded virtual machine) having a secondary processing job demand and sent to a job scheduler, for requesting the job scheduler to allocate a free virtual machine for the target overloaded virtual machine to assist in completing a remaining job.
In specific implementation, when the job scheduler receives an auxiliary processing request sent by a target overloaded virtual machine, the idle virtual machine in the idle virtual machine queue at the current time may be acquired as an option for allocating remaining jobs to assist in completing the remaining jobs for the target overloaded virtual machine.
It should be noted that, in order to enable each overloaded virtual machine to request an idle virtual machine that assists the overloaded virtual machine to complete the remaining tasks, the job scheduler may respond to an auxiliary processing request received from a target overloaded virtual machine in the overloaded virtual machine queue only if the number of idle virtual machines in the idle virtual machine queue is not less than the number of overloaded virtual machines in the overloaded virtual machine queue; when the number of the idle virtual machines in the idle virtual machine queue is smaller than the number of the overloaded virtual machines in the overloaded virtual machine queue, the job scheduler considers that the idle virtual machines which assist in completing the remaining jobs cannot be allocated to all the overloaded virtual machines currently, and ignores the auxiliary processing request sent by the target overloaded virtual machine.
The job scheduler may be an independent server for allocating and scheduling jobs to the resource platform, which is independent from the resource platform, may be a virtual machine in the resource platform for allocating and scheduling jobs to the resource platform, may be another server having a function of allocating and scheduling jobs to the resource platform, and is not particularly limited in this embodiment.
Therefore, the idle virtual machines in the idle virtual machine queue obtained in step 101 provide a data basis for subsequently determining the target idle virtual machine of the auxiliary target overload virtual machine.
It can be understood that the preset rule refers to a basis which is set in advance and determines a target idle virtual machine for a target overloaded virtual machine.
In some possible implementations, if the overloaded virtual machines in the overloaded virtual machine queue are sorted according to the number of remaining jobs; and the idle virtual machines in the idle virtual machine queue are sorted according to the morning and evening of the idle time; then, the preset rule may specifically be: and allocating the idle virtual machines with the idle time from early to late to the overloaded virtual machine according to the sequence of the number of the residual jobs.
It can be understood that the preset rule in this implementation is also referred to as a water filling algorithm, that is, an idle virtual machine with the earliest idle time in the idle virtual machine queue is allocated to the overloaded virtual machine with the most jobs remaining in the overloaded virtual machine queue, and an idle virtual machine with the latest idle time in the idle virtual machine queue is allocated to the overloaded virtual machine with the least jobs remaining in the overloaded virtual machine queue.
As an example, the preset rule is assumed to be the water filling algorithm, and in order to implement scheduling more accurately and orderly and improve task processing efficiency of the resource platform to the greatest extent, the auxiliary processing request may include sequence numbers of the target overloaded virtual machine sorted in the overloaded virtual machine queue.
Then, in step 102, determining a target idle virtual machine corresponding to the target overloaded virtual machine from the idle virtual machines according to a preset rule, which may specifically include: analyzing the auxiliary processing request to obtain a first sequence number of the target overload virtual machine in the overload virtual machine queue; and searching the idle virtual machine with the sequence number being the first sequence number from the idle virtual machines, and recording the idle virtual machine as the target idle virtual machine.
For example, assuming that the current time is T1, the overloaded virtual machine queue is sorted from large to small according to the ratio between the number of unprocessed remaining jobs and the total number of allocated jobs, and includes: 1-overload virtual machine B, 2-overload virtual machine A and 3-overload virtual machine C; after the idle virtual machine queues are sorted from morning to evening according to the idle time, the method comprises the following steps: 1-idle virtual machine F, 2-idle virtual machine E, and 3-idle virtual machine D. After the operation scheduler receives an auxiliary processing request sent by a target overload virtual machine B, acquiring a first sequence number of the target overload virtual machine B as 2 by analyzing the auxiliary processing request, and acquiring an idle virtual machine F, an idle virtual machine E and an idle virtual machine D in a current idle virtual machine queue; at this time, the job scheduler searches for an idle virtual machine with a sequence number of 2, that is, an idle virtual machine E, from the acquired 3 idle virtual machines, and determines that the idle virtual machine E is a target idle virtual machine.
It is understood that the target idle virtual machine is currently in an idle state and may be sleeping, and then, in order to quickly invest in assisting the target overloaded virtual machine to complete the remaining jobs, after the target idle virtual machine is determined, the target idle virtual machine needs to be woken up, that is, the target idle virtual machine is woken up from a sleeping state to a running state.
As an example, when a target idle virtual machine is allocated to a corresponding target overloaded virtual machine and the target idle virtual machine is awakened, the target idle virtual machine may enter a working state, and then the current idle virtual machine queue needs to be updated, that is, the embodiment of the present invention may further include: and removing the target idle virtual machine from the idle virtual machine queue, and updating the idle virtual machine queue.
For example, when it is determined that the target idle virtual machine is the idle virtual machine E and the target idle virtual machine E is woken up, the idle virtual machine E may be further queued from the idle virtual machine queue: and eliminating the 1-idle virtual machine F, the 2-idle virtual machine E and the 3-idle virtual machine D, wherein the currently updated idle virtual machine queue comprises: idle virtual machine F and idle virtual machine D.
And 103, controlling the target idle virtual machine to assist the target overload virtual machine to complete the rest operation.
It can be understood that, in order that the target overloaded virtual machine can accurately communicate with the target idle virtual machine allocated by the job scheduler, the job scheduler may further obtain a physical address of the target idle virtual machine, and send the physical address to the target overloaded virtual machine, so that the target overloaded virtual machine can send the job needing to be allocated to the target idle virtual machine to the corresponding target idle virtual machine according to the physical address.
In a specific implementation, the controlling, in step 103, the target idle virtual machine to assist the target overloaded virtual machine to complete the remaining operation may specifically include: allocating a first residual job with a preset proportion in the residual jobs of the target overload virtual machine to the target idle virtual machine for processing; wherein the remaining jobs include the first remaining job and a second remaining job; and controlling the target overload virtual machine to process the second residual operation.
In one case, the preset ratio may be a ratio determined by a skilled person according to experience or experiment; in another case, the preset ratio may also be a ratio set by the job scheduler according to the processing speed of the target overloaded virtual machine, and specifically may be: when the processing speed of the target overload virtual machine is lower, the preset proportion is larger, so that the efficiency of finishing the rest operation by the target idle virtual machine and the target overload virtual machine is improved to the maximum extent.
For example, assuming that the number of remaining jobs of the target overloaded virtual machine is 10 and the current processing efficiency is 0, the preset ratio may be 100%, i.e., all (i.e., 10) remaining jobs are allocated to the target idle virtual machine; when the current processing efficiency is consistent with the processing efficiency of the target idle virtual machine, then the preset ratio may be 50%, i.e., half (i.e., 5) of the remaining jobs are allocated to the target idle virtual machine.
Therefore, in the embodiment of the present invention, when an overloaded virtual machine and an idle virtual machine occur in a task processing process, the overloaded virtual machine in the overloaded virtual machine queue may request the job scheduler to allocate the idle virtual machine from the idle virtual machine queue to assist the overloaded virtual machine to complete the remaining job through an auxiliary processing request sent to the job scheduler. Therefore, even if part of the virtual machines are overloaded in the process of processing the tasks, the idle virtual machines which have completed the tasks can be flexibly and dynamically scheduled by the task scheduler to assist the overloaded virtual machines to complete the rest of the tasks, so that the resources of the virtual machines are fully utilized, the task processing time is saved, the task processing efficiency is improved, and the user experience is improved.
In order to make the job scheduling method provided by the embodiment of the present invention clearer, a specific example is provided below to explain the embodiment of the present invention.
Assume that after the cloud computing service platform distributes jobs for 5 virtual machines n1, n2, n3, n4, and n5 through the job scheduler while executing a certain task, the number of jobs in each corresponding virtual machine task queue is 12, 16, 8, 6, and 10. It is assumed that the time consumption for each virtual machine to process the same job is the same, and an average method is adopted as the time consumption for the virtual machine to process each task for each job in the own job queue. The idle virtual machine queues Q, Q record the idle time and physical address of each idle virtual machine. Suppose that the virtual machines n1, n2, n3, n4, and n5 individually complete their respective jobs for 2 minutes, 3 minutes, 4 minutes, 10 minutes, and 20 minutes, respectively. Assume that the preset threshold is 50%. The preset ratio is assumed to be 50%.
After 4 minutes, the virtual machines n1, n2, and n3 all have completed corresponding work, and then Q in the current idle virtual machine queue is specifically: { free virtual machine n1, free virtual machine n2, free virtual machine n3 }. At this time, the number of jobs completed by n4 and n5 is 2, and then the ratios of the remaining number of jobs to the total number of jobs of n4 and n5 are both greater than 50%, and then the virtual machine queue P specifically includes: { free virtual machine n4, free virtual machine n5 }.
At this time, according to the water filling algorithm provided by the embodiment of the present invention, the idle virtual machine n1 that completes the processing of the job earliest in Q is found as the overloaded virtual machine n5 that completes the remaining jobs most, to assist in completing the remaining jobs; finding the idle virtual machine n2 that completed job processing in the second order in Q assists completion of the remaining jobs for the overloaded virtual machine n4 that has the most jobs remaining.
Wherein the determined idle virtual machines n1 and n2 need to be woken up, which takes about 1 minute.
Then, the overloaded virtual machine n5 has the current number of remaining jobs of (10-2) ═ 8, and half (i.e., 4) of them are allocated as the idle virtual machine n1 processes. The idle virtual machine n1 and the overloaded virtual machine n5 finish the respective 4 job needs: (20/10) × 4 ═ 8 minutes, i.e., a total of 4+1+8 ═ 13 minutes was required. Similarly, if the number of jobs currently remaining in the overloaded virtual machine n4 is (6-2) ═ 4, half (i.e., 2) of the jobs are allocated to be processed by the idle virtual machine n 2. The idle virtual machine n2 and the overloaded virtual machine n4 finish processing respective 2 job needs: (10/6) × 2 ═ 4 minutes, i.e., a total of 4+1+4 ═ 9 minutes was required.
It can be understood that, by using the job scheduling method provided by the embodiment of the present invention, the task can be completed in 13 minutes at most, and compared with 20 minutes required by the slowest virtual machine n5 to complete all jobs independently, 35% of the task processing time is saved, thereby greatly improving the task processing efficiency of the cloud computing service platform.
Correspondingly, an embodiment of the present invention further provides a job scheduling apparatus, as shown in fig. 2, the apparatus may specifically include:
an obtaining unit 201, configured to obtain an idle virtual machine in an idle virtual machine queue in response to an auxiliary processing request sent by a target overloaded virtual machine in the overloaded virtual machine queue;
a determining unit 202, configured to determine, according to a preset rule, a target idle virtual machine corresponding to the target overloaded virtual machine from the idle virtual machines, and wake up the target idle virtual machine;
a control unit 203, configured to control the target idle virtual machine to assist the target overloaded virtual machine to complete the remaining operations.
Optionally, the overloaded virtual machines in the overloaded virtual machine queue are sorted according to the number of the remaining jobs; and the idle virtual machines in the idle virtual machine queue are sorted according to the morning and evening of the idle time;
the preset rule is as follows: and allocating the idle virtual machines with the idle time from morning to evening to the overloaded virtual machine according to the sequence of the number of the remaining jobs.
Optionally, the auxiliary processing request includes a sequence number of the target overloaded virtual machine in the overloaded virtual machine queue.
Optionally, the determining unit 202 includes:
the analysis subunit is configured to analyze the auxiliary processing request to obtain a first sequence number of the target overloaded virtual machine in the overloaded virtual machine queue;
and the searching subunit is configured to search, from the idle virtual machine queue, an idle virtual machine with a sequence number of the first sequence number, and record the idle virtual machine as the target idle virtual machine.
Optionally, the control unit 203 includes:
the allocation subunit is configured to allocate a first remaining job in a preset proportion of remaining jobs of the target overloaded virtual machine to the target idle virtual machine for processing; wherein the remaining jobs include the first remaining job and a second remaining job;
and the control subunit is used for controlling the target overload virtual machine to process the second residual operation.
Optionally, after waking up the target idle virtual machine, the method further includes:
and the updating unit is used for removing the target idle virtual machine from the idle virtual machine queue and updating the idle virtual machine queue.
Optionally, the condition for responding to the auxiliary processing request includes: and the number of idle virtual machines in the idle virtual machine queue is not less than the number of overloaded virtual machines in the overloaded virtual machine queue.
The above description is related to a job scheduling apparatus, wherein specific implementation manners and achieved effects may refer to the description of an embodiment of a job scheduling method shown in fig. 1, and are not described herein again.
The "first" in the names such as "first serial number", "first remaining job" and the like mentioned in the embodiments of the present invention is used merely as a name identification, and does not represent the first in sequence. The same applies to "second" etc.
As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the above embodiment methods can be implemented by software plus a general hardware platform. With this understanding, the technical solution of the present invention can be embodied in the form of a software product, which can be stored in a storage medium, such as a read-only memory (ROM)/RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network communication device such as a router, etc.) to execute the method according to the embodiments or some parts of the embodiments of the present invention.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus embodiment, since it is substantially similar to the method embodiment, it is relatively simple to describe, and reference may be made to some descriptions of the method embodiment for relevant points. The above-described embodiments of the apparatus are merely illustrative, and the modules described as separate parts may or may not be physically separate, and the parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
The above description is only for the preferred embodiment of the present invention and is not intended to limit the scope of the present invention. It should be noted that, for a person skilled in the art, several modifications and refinements can be made without departing from the invention, and these modifications and refinements should be regarded as the protection scope of the present invention.
Claims (12)
1. A job scheduling method, comprising:
responding to an auxiliary processing request sent by a target overload virtual machine in an overload virtual machine queue, and acquiring an idle virtual machine in an idle virtual machine queue;
determining a target idle virtual machine corresponding to the target overload virtual machine from the idle virtual machines according to a preset rule, and awakening the target idle virtual machine;
controlling the target idle virtual machine to assist the target overload virtual machine to complete the rest operation;
the overload virtual machines in the overload virtual machine queue are sorted according to the number of the residual jobs; and the idle virtual machines in the idle virtual machine queue are sorted according to the morning and evening of the idle time;
the preset rule is as follows: and allocating the idle virtual machines with the idle time from morning to evening to the overloaded virtual machine according to the sequence of the number of the remaining jobs.
2. The method of claim 1, wherein the secondary processing request includes a sequence number of the target overloaded virtual machine in the queue of overloaded virtual machines.
3. The method according to claim 2, wherein the determining, according to a preset rule, a target idle virtual machine corresponding to the target overloaded virtual machine from the idle virtual machines comprises:
analyzing the auxiliary processing request to obtain a first sequence number of the target overload virtual machine in the overload virtual machine queue;
and searching the idle virtual machine with the sequence number being the first sequence number from the idle virtual machine queue, and recording the idle virtual machine as the target idle virtual machine.
4. The method of claim 1, wherein the controlling the target idle virtual machine to assist the target overloaded virtual machine in completing remaining jobs comprises:
allocating a first residual job with a preset proportion in the residual jobs of the target overload virtual machine to the target idle virtual machine for processing; wherein the remaining jobs include the first remaining job and a second remaining job;
and controlling the target overload virtual machine to process the second residual operation.
5. The method of claim 1, after waking the target idle virtual machine, further comprising:
and removing the target idle virtual machine from the idle virtual machine queue, and updating the idle virtual machine queue.
6. The method of claim 1, wherein the condition responsive to the secondary processing request comprises: and the number of idle virtual machines in the idle virtual machine queue is not less than the number of overloaded virtual machines in the overloaded virtual machine queue.
7. A job scheduling apparatus comprising:
the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for responding to an auxiliary processing request sent by a target overload virtual machine in an overload virtual machine queue and acquiring an idle virtual machine in an idle virtual machine queue;
a determining unit, configured to determine, according to a preset rule, a target idle virtual machine corresponding to the target overloaded virtual machine from the idle virtual machines, and wake up the target idle virtual machine;
the control unit is used for controlling the target idle virtual machine to assist the target overload virtual machine to complete the rest operation;
the overload virtual machines in the overload virtual machine queue are sorted according to the number of the residual jobs; and the idle virtual machines in the idle virtual machine queue are sorted according to the morning and evening of the idle time;
the preset rule is as follows: and allocating the idle virtual machines with the idle time from morning to evening to the overloaded virtual machine according to the sequence of the number of the remaining jobs.
8. The apparatus of claim 7, wherein the secondary processing request includes a sequence number of the target overloaded virtual machine in the queue of overloaded virtual machines.
9. The apparatus of claim 8, wherein the determining unit comprises:
the analysis subunit is configured to analyze the auxiliary processing request to obtain a first sequence number of the target overloaded virtual machine in the overloaded virtual machine queue;
and the searching subunit is configured to search, from the idle virtual machine queue, an idle virtual machine with a sequence number of the first sequence number, and record the idle virtual machine as the target idle virtual machine.
10. The apparatus of claim 7, wherein the control unit comprises:
the allocation subunit is configured to allocate a first remaining job in a preset proportion of remaining jobs of the target overloaded virtual machine to the target idle virtual machine for processing; wherein the remaining jobs include the first remaining job and a second remaining job;
and the control subunit is used for controlling the target overload virtual machine to process the second residual operation.
11. The apparatus of claim 7, further comprising, after waking the target idle virtual machine:
and the updating unit is used for removing the target idle virtual machine from the idle virtual machine queue and updating the idle virtual machine queue.
12. The apparatus of claim 7, wherein the condition responsive to the secondary processing request comprises: and the number of idle virtual machines in the idle virtual machine queue is not less than the number of overloaded virtual machines in the overloaded virtual machine queue.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811102507.2A CN109189581B (en) | 2018-09-20 | 2018-09-20 | Job scheduling method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811102507.2A CN109189581B (en) | 2018-09-20 | 2018-09-20 | Job scheduling method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109189581A CN109189581A (en) | 2019-01-11 |
| CN109189581B true CN109189581B (en) | 2021-08-31 |
Family
ID=64909180
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811102507.2A Active CN109189581B (en) | 2018-09-20 | 2018-09-20 | Job scheduling method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109189581B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111930514B (en) * | 2020-09-14 | 2021-09-10 | 四川中电启明星信息技术有限公司 | Resource optimization allocation method and system |
| CN115328666B (en) * | 2022-10-14 | 2023-07-14 | 浪潮电子信息产业股份有限公司 | Device scheduling method, system, electronic device and computer readable storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101488098A (en) * | 2009-02-13 | 2009-07-22 | 华中科技大学 | Multi-core computing resource management system based on virtual computing technology |
| CN102317917A (en) * | 2011-06-30 | 2012-01-11 | 华为技术有限公司 | Hot field virtual machine cpu dispatching method and virtual machine system (vms) |
| CN102508718A (en) * | 2011-11-22 | 2012-06-20 | 杭州华三通信技术有限公司 | Method and device for balancing load of virtual machine |
| CN103377092A (en) * | 2012-04-12 | 2013-10-30 | 韩国电子通信研究院 | Two-level resource management method and appratus for dynamic resource management |
| CN104125166A (en) * | 2014-07-31 | 2014-10-29 | 华为技术有限公司 | Queue scheduling method and computing system |
| CN104239123A (en) * | 2014-09-05 | 2014-12-24 | 北方工业大学 | Campus-cloud-platform-oriented virtual machine management dispatching method and system |
| CN106980532A (en) * | 2016-01-18 | 2017-07-25 | 西安中兴新软件有限责任公司 | A kind of job scheduling method and device |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9092250B1 (en) * | 2006-10-27 | 2015-07-28 | Hewlett-Packard Development Company, L.P. | Selecting one of plural layouts of virtual machines on physical machines |
| US9529613B2 (en) * | 2012-12-12 | 2016-12-27 | Vmware, Inc. | Methods and apparatus to reclaim resources in virtual computing environments |
-
2018
- 2018-09-20 CN CN201811102507.2A patent/CN109189581B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101488098A (en) * | 2009-02-13 | 2009-07-22 | 华中科技大学 | Multi-core computing resource management system based on virtual computing technology |
| CN102317917A (en) * | 2011-06-30 | 2012-01-11 | 华为技术有限公司 | Hot field virtual machine cpu dispatching method and virtual machine system (vms) |
| CN102508718A (en) * | 2011-11-22 | 2012-06-20 | 杭州华三通信技术有限公司 | Method and device for balancing load of virtual machine |
| CN103377092A (en) * | 2012-04-12 | 2013-10-30 | 韩国电子通信研究院 | Two-level resource management method and appratus for dynamic resource management |
| CN104125166A (en) * | 2014-07-31 | 2014-10-29 | 华为技术有限公司 | Queue scheduling method and computing system |
| CN104239123A (en) * | 2014-09-05 | 2014-12-24 | 北方工业大学 | Campus-cloud-platform-oriented virtual machine management dispatching method and system |
| CN106980532A (en) * | 2016-01-18 | 2017-07-25 | 西安中兴新软件有限责任公司 | A kind of job scheduling method and device |
Non-Patent Citations (2)
| Title |
|---|
| 云计算环境下基于改进粒子群算法的任务调度;张照胜，李蜀瑜;《电子设计工程》;20160805;第24卷(第15期);第5-8,12页 * |
| 张照胜，李蜀瑜.云计算环境下基于改进粒子群算法的任务调度.《电子设计工程》.2016,第24卷(第15期),第5-8,12页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109189581A (en) | 2019-01-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11582166B2 (en) | Systems and methods for provision of a guaranteed batch | |
| EP3675434B1 (en) | Distributed system resource allocation method, device and system | |
| CN111381950A (en) | Task scheduling method and system based on multiple copies for edge computing environment | |
| CN106557369A (en) | A kind of management method and system of multithreading | |
| CN111767134A (en) | Multitask dynamic resource scheduling method | |
| CN106775948B (en) | Cloud task scheduling method and device based on priority | |
| CN109086135B (en) | Resource scaling method and device, computer equipment and storage medium | |
| US10778807B2 (en) | Scheduling cluster resources to a job based on its type, particular scheduling algorithm,and resource availability in a particular resource stability sub-levels | |
| CN112486642B (en) | Resource scheduling method, device, electronic equipment and computer readable storage medium | |
| US10606650B2 (en) | Methods and nodes for scheduling data processing | |
| CN112783659A (en) | Resource allocation method and device, computer equipment and storage medium | |
| CN109189581B (en) | Job scheduling method and device | |
| CN104820616A (en) | Task scheduling method and device | |
| CN106775975B (en) | Process scheduling method and device | |
| CN114721818A (en) | Kubernetes cluster-based GPU time-sharing method and system | |
| CN114816709A (en) | Task scheduling method, device, server and readable storage medium | |
| CN110175078B (en) | Service processing method and device | |
| CN113626173A (en) | Scheduling method, device and storage medium | |
| CN111158875B (en) | Multi-module-based multi-task processing method, device and system | |
| Wu et al. | Abp scheduler: Speeding up service spread in docker swarm | |
| CN110413393B (en) | Cluster resource management method and device, computer cluster and readable storage medium | |
| CN113849295A (en) | Model training method and device and computer readable storage medium | |
| CN111143210A (en) | Test task scheduling method and system | |
| CN113051063B (en) | Task scheduling method and device for distributed tasks and electronic equipment | |
| CN112130979B (en) | Method, device, terminal and medium for scheduling task and training neural network model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |