CN109684070B - Scheduling method in cloud computing parallel operation - Google Patents

Scheduling method in cloud computing parallel operation Download PDF

Info

Publication number
CN109684070B
CN109684070B CN201810997296.7A CN201810997296A CN109684070B CN 109684070 B CN109684070 B CN 109684070B CN 201810997296 A CN201810997296 A CN 201810997296A CN 109684070 B CN109684070 B CN 109684070B
Authority
CN
China
Prior art keywords
task
virtual machine
virtual machines
time
tasks
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810997296.7A
Other languages
Chinese (zh)
Other versions
CN109684070A (en
Inventor
马建峰
张兆一
李辉
张世哲
李金库
姚青松
宁建斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xidian University
Original Assignee
Xidian University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xidian University filed Critical Xidian University
Priority to CN201810997296.7A priority Critical patent/CN109684070B/en
Publication of CN109684070A publication Critical patent/CN109684070A/en
Application granted granted Critical
Publication of CN109684070B publication Critical patent/CN109684070B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/5011—Pool
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/5017—Task decomposition
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention discloses a scheduling method in parallel operation of cloud computing, which is characterized in that 1,2,3, 8230n virtual machines are started simultaneously after the configuration of the virtual machines is determined, and the average time t of starting each virtual machine in each test is counted 1 ,t 2 ,t 3 ,…,t n Obtaining the maximum likelihood value of a and b according to a formula ti = ai + b; then calculating the task time tau, and starting m virtual machines to execute the task in parallel to form an optimal scheme; and secondly, a task window approximately and evenly divides the tasks into n parts (n > m) to form a task set, the task set is placed in a task pool, a task scheduler sequentially creates m virtual machines, simultaneously starts a parallel processing task process, allocates the tasks to the virtual machines when a new virtual machine is started or the tasks on the virtual machines are completed, and finally all the virtual machines complete all the tasks and return task results. The optimal scheduling strategy is designed according to the characteristics of the starting time of the virtual machines, can realize the approximately optimal speed-up ratio, ensures that each virtual machine task is almost completed at the same time, greatly reduces the task execution time, improves the task efficiency and reduces the waste of system resources.

Description

Scheduling method in cloud computing parallel operation
Technical Field
The invention belongs to the technical field of computers, and further relates to a scheduling method in cloud computing parallel operation in the technical field of cloud computing.
Background
Due to the advantages of convenient storage service, flexible charging mode and the like, more and more enterprises and individuals store the local data on the cloud server to reduce the storage burden and management overhead of the local data. Considerable awareness has been gained in recent years in a considerable number of businesses and individuals. Users do not need to reserve their own hardware and software resources, and they can send requests to the cloud at any time to compute jobs. When a large-scale computing task or a transmission task is executed, a Virtual Machine (VM) on the cloud platform can be started to execute parallel computing or parallel transmission. For example, to protect the privacy of data, we encrypt a large amount of plaintext into ciphertext and transmit a large amount of ciphertext from the server to the client, which takes a lot of time to complete. Generally, we can break a task into multiple subtasks and start a large number of virtual machines to execute the subtasks to shorten the completion time.
However, there are many problems to be solved. Since the start of a virtual machine also requires time, the more virtual machines are not, the shorter the total completion time of the task. How many virtual machines should be started, how should the virtual machines be assigned tasks? Therefore, an optimal scheduling scheme is determined, so that the cloud platform can map the tasks to the virtual machines in an optimal mode, and the total task completion time is further shortened to be a hot spot problem in the current cloud computing field.
A parallel computing method and a system (application number: 201310591160.3, publication number: 103617086B) applied by Donghou group sharps corporation disclose a parallel computing method, which comprises the following steps: monitoring each computing node to obtain node monitoring data; calculating a load capacity of the compute node based on the node monitoring data; and distributing the tasks to be distributed according to the load capacity of the computing nodes. The method has the following disadvantages: scheduling is only performed on platform resources, and a task scheduling method is not optimized.
The patent applied to Shandong university based on a dynamic load balancing method of a Linux parallel computing platform (application number: 201310341592.9, publication number: 103399800B) discloses a dynamic load balancing method of a Linux-based parallel computing platform, and the method content is as follows: in the parallel computing process, the total computing task is divided into a plurality of stages with equal execution time to be executed. By using a routine work scheduling technology in the system, before the parallel computation of each time stage is started, the current resource utilization rate of each node is read, and the computation tasks of the nodes are dynamically allocated according to the computation performance and the computation complexity of each node, so that the computation time of each node in each stage is basically equal, and the delay of system synchronization waiting is reduced. By the dynamic adjustment strategy, the total calculation task can be completed with higher resource utilization rate. The method has the following defects: the method is limited to the Linux platform, and the influence of the system starting time on the execution efficiency is not considered.
The essence of the main strategies widely studied and practiced at present, such as dynamically adjusting the CPU frequency, shutting down or putting idle machines into a sleep state, migration and merging of virtual machines, etc., is to maximize the energy utilization rate by dynamically scheduling and integrating data center resources, however, there is no optimal acceleration strategy that minimizes the total time of tasks when the task volume and system resources are fixed.
Therefore, when system resources are fixed, how to shorten the total task completion time as much as possible when tasks are executed in parallel, and seek for an appropriate number of virtual machines, so that maximizing the optimal speed-up ratio becomes an urgent problem to be solved.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a scheduling method in parallel operation of cloud computing, which is characterized in that before a task is executed, the number of virtual machines capable of realizing optimal acceleration is determined through system resources and task amount, then the task is divided, and the virtual machines are sequentially started to execute the task.
The invention is realized by the following technical scheme:
a scheduling method in parallel operation of cloud computing comprises the following steps:
s1, calculating cloud platform parameters:
after the configuration of the virtual machines is determined, 1,2,3, \ 8230, n virtual machines are started simultaneously, and the average time t used for starting each virtual machine in each test is counted 1 ,t 2 ,t 3 ,…,t n Substituting the two sets of data into t i In the = ai + b, the likelihood values of a and b are calculated and recorded;
wherein, t i When i virtual machines are started simultaneously, the average time for starting each virtual machine, and a and b system characteristic parameters;
s2, determining an optimal acceleration strategy:
executing tasks in one virtual machine, and starting m virtual machines to execute the tasks in parallel to obtain an optimal scheme when the task time tau is calculated;
Figure GDA0001983095090000021
s3, splitting a task:
when the task reaches the task window, the task window approximately and averagely divides the task into n parts (n > m) to form a task set, and the task set is placed in a task pool;
s4, creating a virtual machine:
the task scheduler sequentially creates m virtual machines and simultaneously starts a parallel processing task process;
s5, task allocation:
when a new virtual machine is started or a task on the virtual machine is completed, a task scheduler allocates a task to the virtual machine;
s6, completing a task:
and completing all tasks and returning task results.
Preferably, in step S5, the specific method for the task scheduler to allocate the tasks is that the task scheduler takes out 1 task from the task pool, maps the task onto the virtual machine, starts to execute the task, checks the task pool after the task on the virtual machine is executed, continues to allocate the task to the virtual machine if the task pool is not empty, and returns the virtual machine to the initial state if the task pool is empty.
Compared with the prior art, the invention has the following beneficial technical effects:
1) The optimal scheduling strategy is designed according to the characteristics of the starting time of the virtual machines, can realize the approximately optimal speed-up ratio, ensures that each virtual machine task is almost completed at the same time, greatly reduces the task execution time, improves the task efficiency and reduces the waste of system resources.
2) The optimal acceleration strategy in the invention is realized by controlling the number of virtual machines and planning tasks. Once the calculation task is given, the near optimal acceleration ratio system can calculate the optimal number of virtual machines and automatically plan the task. The application range is wider, and the method can be used together with the existing dynamic resource scheduling scheme, so that the system efficiency is greatly improved.
Drawings
FIG. 1 is a block flow diagram of a scheduling method;
FIG. 2 is a graph of total time for startup of multiple virtual machines;
FIG. 3 is a schematic diagram of the task time τ determination process;
FIG. 4 is a task allocation diagram.
Detailed Description
The invention is further described with reference to the following figures and detailed description.
As shown in fig. 1, which is a schematic diagram of an optimal solution step in parallel operation of cloud computing according to the present invention, a specific flow of the system is described as follows:
the method comprises the following steps: calculating cloud platform parameters: after the configuration of the virtual machines is determined, 1,2,3, \ 8230, n virtual machines are started simultaneously, and the average time t used for starting each virtual machine in each test is counted 1 ,t 2 ,t 3 ,…,t n Substituting the two sets of data into t i In = ai + b, (i =1,2, \8230;, n), the likelihood values of a and b are calculated and recorded;
step two: determining an optimal acceleration strategy: executing task in a virtual machine, and starting when computing task time tau
Figure GDA0001983095090000031
The parallel execution tasks of the virtual machines are the optimal scheme;
step three: splitting tasks: when the task reaches the task window, the task window approximately and averagely divides the task into n parts (n > m) to form a task set, and the task set is placed in a task pool;
step four: creating a virtual machine: the task scheduler sequentially creates m virtual machines and simultaneously starts a parallel processing task process;
step five: and (3) task allocation: when a new virtual machine is started or a task on the virtual machine is completed, a task scheduler allocates a task to the virtual machine;
step six: and (3) completing the task: and completing all tasks and returning task results.
In the first step, when a plurality of virtual machines are started simultaneously, the calculation method of the average starting time of each virtual machine T = AN + B (N is the number of virtual machines, and T is the average starting time), and the calculation formula is obtained through a large number of experimental tests. The experimental contents are as follows: starting m virtual machines on the OpenStack platform simultaneously, and recording the starting time T of the first virtual machine min Total time to start T max And calculating the average time T for starting each virtual machine avg As shown in fig. 2, it can be found that the total starting time of multiple virtual machines approximately conforms to the curve of T = AN + B.
In step two, a schematic diagram of the process of determining the task time τ is shown in fig. 3, when the started virtual machine continuously fetches the task from the task pool and executes the task, it can be considered that all the virtual machines finish the task approximately at the same time, so the total task completion time is equal to the task completion time of each virtual machine, and is also equal to the sum of the average start time of each virtual machine and the average task execution time, i.e. T all ＝T avg + τ/m = am + b + τ/m, and it is observed that when m is taken as
Figure GDA0001983095090000041
The total time takes the minimum value.
In step five, fig. 4 is a schematic diagram of task allocation, and specifically includes the following steps:
the first step is as follows: when the virtual machine is started, taking 1 part of task out of the task pool, mapping the task to the virtual machine, and starting to execute the task;
the second step: and after the task on the virtual machine is executed, checking the task pool, if the task pool is not empty, continuing the first step, and if the task pool is empty, restoring the virtual machine to an initial state.
The above description is only one specific example of the present invention and should not be construed as limiting the invention in any way. It will be apparent to those skilled in the art that modifications and improvements to the algorithm can be made without departing from the principles and structures of the invention, after understanding the present disclosure and principles, but such modifications and improvements are intended to be within the scope of the invention as defined by the appended claims.

Claims (2)

1. A scheduling method in parallel operation of cloud computing is characterized by comprising the following steps:
s1, calculating cloud platform parameters:
after the configuration of the virtual machines is determined, 1,2,3, \ 8230, n virtual machines are started simultaneously, and average time t1, t2 and t for starting each virtual machine in each test is counted 3 ,…,t n Respectively substituting n data obtained by statistics into t i In = ai + b, the maximum likelihood value of a and b is calculated;
wherein, t i When i virtual machines are started simultaneously, the average time for starting each virtual machine is used, and a and b are system characteristic parameters;
s2, determining an optimal acceleration strategy:
executing tasks in one virtual machine, and starting m virtual machines to execute the tasks in parallel to obtain an optimal scheme when the task time tau is calculated;
when the started virtual machines continuously take tasks from the task pool to execute, all the virtual machines finish the tasks approximately at the same time, so that the total task completion time is equal to the task completion time of each virtual machine and also equal to the sum of the average starting time of each virtual machine and the average task execution time, and the formula is as follows:
T all ＝T avg +τ/m＝am+b+τ/m
when m is taken
Figure FDA0003849607490000011
When the time is short, the total time is the minimum value;
wherein, T avg For starting the average time of use of each virtual machine, T all Total task completion time;
s3, splitting tasks:
when the task reaches the task window, the task window approximately and averagely divides the task into n parts (n > m) to form a task set, and the task set is placed in a task pool;
s4, creating a virtual machine:
the task scheduler sequentially creates m virtual machines and simultaneously starts a parallel processing task process;
s5, task allocation:
when a new virtual machine is started or a task on the virtual machine is completed, a task scheduler allocates a task to the virtual machine;
s6, completing a task:
and all the virtual machines complete all the tasks and return task results.
2. The scheduling method in the parallel operation of the cloud computing according to claim 1, wherein the task scheduler allocates the tasks in step S5 by taking 1 task out of the task pool by the task scheduler, mapping the task onto the virtual machine, starting the virtual machine to execute the task, checking the task pool after the task on the virtual machine is executed, if the task pool is not empty, continuing to allocate the task to the virtual machine, and if the task pool is empty, restoring the virtual machine to the initial state.
CN201810997296.7A 2018-08-29 2018-08-29 Scheduling method in cloud computing parallel operation Active CN109684070B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810997296.7A CN109684070B (en) 2018-08-29 2018-08-29 Scheduling method in cloud computing parallel operation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810997296.7A CN109684070B (en) 2018-08-29 2018-08-29 Scheduling method in cloud computing parallel operation

Publications (2)

Publication Number Publication Date
CN109684070A CN109684070A (en) 2019-04-26
CN109684070B true CN109684070B (en) 2022-12-13

Family

ID=66184433

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810997296.7A Active CN109684070B (en) 2018-08-29 2018-08-29 Scheduling method in cloud computing parallel operation

Country Status (1)

Country Link
CN (1) CN109684070B (en)

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2012104898A1 (en) * 2011-01-31 2012-08-09 トヨタ自動車株式会社 Safety control device and safety control method
JP5632403B2 (en) * 2012-01-24 2014-11-26 日本電信電話株式会社 Task management system, task management server, task management method, and task management program
CN104536806B (en) * 2014-12-26 2017-11-03 东南大学 A kind of workflow application flexible resource Supply Method under cloud environment
CN105159752B (en) * 2015-09-22 2018-03-30 中国人民解放军国防科学技术大学 Virtualize the real-time task and resource regulating method of machine startup Time Perception in cloud
KR101707601B1 (en) * 2015-12-31 2017-02-16 숭실대학교산학협력단 Virtual machine monitor and schduling method of virtual machine monitor
CN105912406B (en) * 2016-05-05 2018-01-12 中国人民解放军国防科学技术大学 The Independent Task Scheduling and resource allocation method of a kind of low energy consumption
CN107357641A (en) * 2017-06-21 2017-11-17 西安电子科技大学 Method for scheduling task in a kind of cloud computing

Also Published As

Publication number Publication date
CN109684070A (en) 2019-04-26

Similar Documents

Publication Publication Date Title
Aslam et al. Load balancing algorithms in cloud computing: A survey of modern techniques
Di et al. Dynamic optimization of multiattribute resource allocation in self-organizing clouds
EP3281359B1 (en) Application driven and adaptive unified resource management for data centers with multi-resource schedulable unit (mrsu)
US20140282503A1 (en) Weight-based collocation management
Mishra et al. Time efficient dynamic threshold-based load balancing technique for Cloud Computing
Mosa et al. Dynamic virtual machine placement considering CPU and memory resource requirements
Domanal et al. Load balancing in cloud environment using a novel hybrid scheduling algorithm
Nair et al. Efficient resource arbitration and allocation strategies in cloud computing through virtualization
Liu et al. Dynamically negotiating capacity between on-demand and batch clusters
Wu et al. Dynamically adjusting scale of a kubernetes cluster under qos guarantee
Katangur et al. Priority weighted round robin algorithm for load balancing in the cloud
CN111813558B (en) Resource scheduling method for hybrid cloud
CN109684070B (en) Scheduling method in cloud computing parallel operation
CN109032779A (en) Task processing method, device, computer equipment and readable storage medium storing program for executing
CN110069319B (en) Multi-target virtual machine scheduling method and system for cloud resource management
CN109189581B (en) Job scheduling method and device
Koneru et al. Resource allocation method using scheduling methods for parallel data processing in cloud
CN117112222A (en) Request processing method and device, electronic equipment and storage medium
CN109144664B (en) Dynamic migration method of virtual machine based on user service quality demand difference
Swarnakar et al. A novel improved hybrid model for load balancing in cloud environment
EP3982258A1 (en) Method and apparatus for reducing power consumption of virtual machine cluster
Darji et al. Dynamic load balancing for cloud computing using heuristic data and load on server
Chen et al. A new heuristic scheduling strategy lbmm in cloud computing
Zhang et al. Speeding up vm startup by cooperative vm image caching
Anjiki et al. Performance Improvement by Controlling VM Migration between Edge Nodes in a Multi-Stage Information Processing System

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant