CN111240811A - Cluster scheduling method, device and system and electronic equipment - Google Patents
Cluster scheduling method, device and system and electronic equipment Download PDFInfo
- Publication number
- CN111240811A CN111240811A CN201811436068.9A CN201811436068A CN111240811A CN 111240811 A CN111240811 A CN 111240811A CN 201811436068 A CN201811436068 A CN 201811436068A CN 111240811 A CN111240811 A CN 111240811A
- Authority
- CN
- China
- Prior art keywords
- cluster
- instance
- scheduling
- aging
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
Abstract
The embodiment of the invention provides a cluster scheduling method, a cluster scheduling device, a cluster scheduling system and electronic equipment. The method comprises the following steps: acquiring the aging information of an aging example in a cluster; determining a scheduling strategy according to the aging information of the aging example; and distributing tasks to the computing instances in the cluster according to the scheduling strategy. According to the embodiment of the invention, the corresponding scheduling strategy is determined by monitoring the timeliness information of the timeliness instance so as to perform task allocation on the computing instance in the cluster, thereby exerting the elastic advantage of timeliness computing resources, further optimizing cluster scheduling and improving the resource utilization rate.
Description
Technical Field
The invention relates to the technical field of cloud computing, in particular to a cluster scheduling method, a cluster scheduling device, a cluster scheduling system and electronic equipment.
Background
In the field of cloud computing, in order to improve the utilization rate of cloud computing resources, a cloud service provider provides a bidding instance service, namely, a user can purchase a bidding instance at a price far lower than that of a common instance within a certain time efficiency range, and the bidding instance can be released at any time after the time efficiency is exceeded. Users can use a large amount of flexible computing resources in a planned way by selecting a bidding instance service, so as to save cost.
In the clustered cloud computing service, a cluster scheduler is arranged to perform scheduling management on computing resources (various instances). The existing cluster scheduler carries out scheduling management aiming at known fixed computing resources, and the computing resources which are added into a cluster can be obtained in a non-fault condition by default.
In the process of implementing the invention, the inventor finds that the prior art has at least the following problems: when the existing cluster scheduler schedules the computing resources, whether the scheduled resources have timeliness or not is not considered, so that the timeliness of the example cannot be sensed, and the elastic advantage of the timeliness example is difficult to be exerted.
Disclosure of Invention
Embodiments of the present invention provide a cluster scheduling method, apparatus, and system, and an electronic device, so as to solve a defect in the prior art that it is difficult to exert an elastic advantage of a time-efficient example.
To achieve the above object, an embodiment of the present invention provides a cluster scheduling method, including:
acquiring the aging information of an aging example in a cluster;
determining a scheduling strategy according to the aging information of the aging example;
and distributing tasks to the computing instances in the cluster according to the scheduling strategy.
An embodiment of the present invention further provides a cluster scheduling apparatus, including:
the information acquisition module is used for acquiring the aging information of the aging instances in the cluster;
the strategy determining module is used for determining a scheduling strategy according to the timeliness information of the timeliness example;
and the execution module is used for distributing tasks to the computing examples in the cluster according to the scheduling strategy.
The embodiment of the present invention further provides a cluster scheduling system, including: a cluster resource management component and a cluster scheduler,
the cluster resource management component is used for acquiring the aging information of an aging instance in a cluster and sending the aging information to the cluster scheduler;
the cluster scheduler is used for receiving the aging information of the aging example sent by the cluster resource management component, determining a scheduling strategy according to the aging information of the aging example, and distributing tasks to the calculation examples in the cluster according to the scheduling strategy.
An embodiment of the present invention further provides an electronic device, including:
a memory for storing a program;
a processor for executing the program stored in the memory for:
acquiring the aging information of an aging example in a cluster;
determining a scheduling strategy according to the aging information of the aging example;
and distributing tasks to the computing instances in the cluster according to the scheduling strategy.
An embodiment of the present invention further provides a computer-readable storage medium, where instructions are stored on the computer-readable storage medium, where the instructions include:
acquiring the aging information of an aging example in a cluster;
determining a scheduling strategy according to the aging information of the aging example;
and distributing tasks to the computing instances in the cluster according to the scheduling strategy.
According to the cluster scheduling method, the cluster scheduling device, the cluster scheduling system and the electronic equipment, the corresponding scheduling strategy is determined by monitoring the timeliness information of the timeliness instances, so that task allocation is carried out on the computing instances in the cluster, the elastic advantage of timeliness computing resources is brought into play, cluster scheduling can be further optimized, and the resource utilization rate is improved.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the application. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
fig. 1 is a system block diagram of an embodiment of a cluster scheduling system provided in the present invention;
FIG. 2 is a timing diagram of an embodiment of a cluster scheduling system provided in the present invention;
fig. 3 is a flowchart of an embodiment of a cluster scheduling method provided in the present invention;
fig. 4 is a flowchart of another embodiment of a cluster scheduling method provided in the present invention;
fig. 5 is a schematic structural diagram of an embodiment of a cluster scheduling apparatus provided in the present invention;
fig. 6 is a schematic structural diagram of another embodiment of a cluster scheduling apparatus provided in the present invention;
fig. 7 is a schematic structural diagram of an embodiment of an electronic device provided in the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
In the prior art, a cluster scheduler performs scheduling management on known fixed computing resources, and computing resources that are added to a cluster are all available under a non-fault condition by default. That is, when scheduling a computing resource, the conventional cluster scheduler does not consider whether the scheduled resource is a bidding instance, and therefore, cannot sense the timeliness of the bidding instance, and cannot exert the advantage of flexibility of the bidding instance. Therefore, aiming at the defects of the prior art, the present application proposes a cluster scheduling scheme, whose main principle is: the method comprises the steps of obtaining timeliness information of timeliness computing resources (namely timeliness instances), and determining a scheduling strategy according to the timeliness information so as to distribute tasks to the computing resources, thereby exerting the elastic advantage of the timeliness computing resources, further optimizing cluster scheduling and improving the resource utilization rate.
The above embodiments are illustrations of technical principles of the embodiments of the present invention, and specific technical solutions of the embodiments of the present invention are further described in detail below by using a plurality of embodiments.
Example one
Fig. 1 is a system block diagram of an embodiment of a cluster scheduling system provided by the present invention, and the structure shown in fig. 1 is only one example of a service system to which the technical solution of the present invention can be applied. The cluster scheduling system provided by the embodiment of the invention can be applied to scenes of clustered cloud Computing services, such as High Performance Computing (HPC) cluster cloud-on-scene and other scenes. HPC is a field with a long history of development, playing an important role in scientific engineering calculations. With the rise of cloud computing and artificial intelligence, the cloud demand on an HPC cluster is more and more, the scale of a single cluster is larger and larger, and in the process of migrating to the cloud, the construction of an HPC cluster with the best cost performance by using characteristic services (such as bidding instance services) of resources on the cloud becomes an important subject. As shown in fig. 1, a cluster scheduling system provided in an embodiment of the present invention includes: a cluster resource management component 11 and a cluster scheduler 12.
The cluster resource management component 11 is configured to obtain aging information of an aging instance in a cluster, and send the aging information to the cluster scheduler 12; the cluster scheduler 12 is configured to receive the aging information of the aging example sent by the cluster resource management component 11, determine a scheduling policy according to the aging information of the aging example, and perform task allocation on each computation example in the cluster according to the scheduling policy.
In the embodiment of the present invention, the cluster resource management component 11 may be responsible for managing the life cycles of creating and deleting resources, adding and exiting computing resources, and the like of the whole cluster. The cluster resource management component 11 interacts with the cluster scheduler 12 to inform the cluster scheduler 12 of the joining, exiting of computing resources (compute instances) so that the cluster scheduler 12 can determine or update the scheduling policy. Unlike the composition of conventional supercomputing systems, the cloud-based time-sensitive instances are flexibly applicable and releasable, and therefore the cluster resource management component 11 can be dedicated to the lifecycle management of cluster resources. Of course, in the embodiment of the present invention, the cluster resource management component 11 may be an independently deployed system, may be deployed together with the cluster scheduler 12, and may also be an internal component of the cluster scheduler 12.
The cluster scheduler 12 may be responsible for receiving a computing job (computing task) submitted by a user, allocating the task to a specific computing resource (computing instance) of the computing cluster according to different scheduling policies for computing, and monitoring the running state of the task to obtain a computing execution result. Different tasks may be configured with different parameters such as resource requirements, priorities, execution times, etc., and the cluster scheduler 12 may generally adopt different scheduling strategies according to the configuration parameters of the different tasks. When there are more tasks on a compute instance, the cluster scheduler 12 may perform task queuing and queue management. The cluster scheduler 12 may also monitor the results of task execution and tasks that failed execution may resubmit execution.
The compute cluster is responsible for performing specific compute tasks. High-performance computing application environments, such as scientific computing software for meteorology, mechanics, molecular dynamics and the like, are deployed on computing resources of the cluster.
Specifically, in the embodiment of the present invention, the aging information of the aging example includes the expiration time of the aging example. The cluster scheduler 12 may be specifically configured to determine an upcoming expiration instance according to the expiration time of the aging instance, where the upcoming expiration instance is an instance whose time difference between the expiration time and the current time is smaller than a preset time threshold; the cluster scheduler 12 may also be configured to determine a scheduling policy based on the instance information of the upcoming expired instance and send the scheduling policy to the cluster resource management component 11. The cluster resource management component 11 may also be configured to perform a release operation on the upcoming expired instance according to the scheduling policy sent by the cluster scheduler.
In addition, the cluster resource management component 11 may also be used to apply for a computation instance and create a cluster, and in the embodiment of the present invention, the computation instance created by the cluster resource management component 11 includes a time-efficient instance, and may also include some common instances without time-efficiency.
Further, the cluster resource management component 11 may be further configured to apply for a new time-sensitive instance when the number of time-sensitive instances in the cluster is lower than a preset number threshold.
That is, in the embodiment of the present invention, a resource pool manner may also be adopted for the application of the resource. For example, a batch of bidding instance resources may be pre-applied while maintaining the number of bidding instance resources above a certain water line (i.e., a preset number threshold). When the released bidding instances are gradually increased, so that the number of the bidding instance resources in the resource pool is lower than the water level line, the application of a new batch of bidding instance resources is triggered. This eliminates the need for detecting and outdated replenishment operations on a time-sensitive instance-by-time basis. In addition, when applying for the bidding instance resources, a batch time-sharing application mode can be adopted to divide the application of a batch of bidding instance resources into several batches, and each batch is applied at staggered time, so that the released time of the bidding instances is staggered to balance available resources.
Fig. 2 is a timing diagram of an embodiment of a cluster scheduling system provided in the present invention. As shown in fig. 2, in the cluster scheduling system provided in the embodiment of the present invention, each component and device may perform corresponding operations according to the following timing sequence:
s201, the cluster resource management component applies for a calculation example and creates a calculation cluster. For example, the examples applied in FIG. 2 include the time-sensitive example (e.g., Bid example A) and the general example without time-sensitive.
S202, the cluster resource management component notifies the cluster scheduler of the instance information of the computing instance and maintains the life cycle of all the computing instances of the cluster when the resource application is successful.
And S203, after the cluster is established, the cluster scheduler can receive the tasks submitted by the users and distribute the tasks submitted by the users to each computing instance for computing according to the scheduling strategy.
S204, when the time limit of the bidding instance is about to expire (the time is configurable, for example, 5 minutes before the expiration), the cluster resource management component informs the cluster scheduler of the instance information of the about to expire instance.
S205, after receiving the notice of the cluster resource management component, the cluster scheduler marks the computing resource as a resource to be expired and returns a scheduling strategy to the cluster resource management component.
S206, the cluster resource management component releases the bidding instance A according to the scheduling strategy returned by the cluster scheduler.
And S207, after the bidding example A is released, the cluster resource management component continues to apply for a new bidding example B to supplement the lost computing resource.
And S208, after applying for the bidding case B, informing the cluster scheduler of the case information of the new resource.
S209, the cluster scheduler distributes the tasks submitted by the users to each computing instance for computing according to the scheduling strategy.
In the embodiment of the invention, the whole cluster can continuously receive and execute the computing tasks submitted by the user.
The cluster scheduling system provided by the embodiment of the invention determines a corresponding scheduling strategy by monitoring the timeliness information of the timeliness instance so as to distribute tasks to the calculation instances in the cluster; and corresponding releasing operation is carried out on the example which is about to expire according to the scheduling strategy, so that the elastic advantage of time-efficient computing resources is exerted, the cluster scheduling can be further optimized, and the resource utilization rate is improved.
Example two
Fig. 3 is a flowchart of an embodiment of the cluster scheduling method provided by the present invention, and an execution main body of the method may be the cluster scheduling system, various terminals or server devices having a software cluster scheduling function, or a system or chip integrated on these devices. As shown in fig. 3, the cluster scheduling method includes the following steps:
s301, acquiring aging information of an aging example in the cluster.
S302, according to the aging information of the aging example, a scheduling strategy is determined.
In the embodiment of the invention, after the cluster is created, each calculation example in the cluster needs to be monitored, the aging information of each aging example is obtained, and the corresponding scheduling side strategy is determined according to the aging information.
S303, distributing tasks to the computing instances in the cluster according to the scheduling strategy.
In the embodiment of the invention, according to the determined scheduling policy, the computing tasks submitted by the users are distributed to each computing instance (including time-efficient instances and also including non-time-efficient common instances) to perform computing operation.
According to the cluster scheduling method provided by the embodiment of the invention, the corresponding scheduling strategy is determined by monitoring the timeliness information of the timeliness examples so as to perform task allocation on the calculation examples in the cluster, so that the elastic advantages of timeliness calculation resources are exerted, the cluster scheduling can be further optimized, the resource utilization rate is improved, meanwhile, the cluster can obtain resources with lower cost, and the reliability of calculation is guaranteed.
EXAMPLE III
Fig. 4 is a flowchart of another embodiment of a cluster scheduling method provided in the present invention. As shown in fig. 4, on the basis of the embodiment shown in fig. 3, the cluster scheduling method provided in this embodiment may further include the following steps:
s401, applying for a calculation example and creating a cluster.
In the embodiment of the present invention, when creating a cluster, the computing instance of the application may be completely composed of time-sensitive instances, or may contain a certain number of non-time-sensitive common instances. When all instances are released, task scheduling may be interrupted and may continue until a new instance joins the cluster.
S402, acquiring aging information of aging instances in the cluster.
In the embodiment of the present invention, the aging information of the aging example includes the expiration time of the aging example.
And S403, determining an upcoming due instance according to the due time of the aging instance.
In the embodiment of the present invention, the upcoming expiration instance is an instance whose time difference between the expiration time and the current time is smaller than a preset time threshold.
S404, determining a scheduling strategy according to the instance information of the upcoming expiration instance.
In the embodiment of the invention, the appropriate scheduling policy is determined according to the instance detailed information of the upcoming expired instance, such as the computing task on the instance.
Specifically, the scheduling policy determined according to the instance detail information of the upcoming expired instance in the embodiment of the present invention may include:
1. immediate failure strategy: when the upcoming expired instance does not execute the computing task at the current time or fails to execute the computing task, the upcoming expired instance is immediately released.
Due to the existence of non-time-sensitive common instances in the cluster, the rescheduled task can always find schedulable resources.
2. Task expiration invalidation policy: when the upcoming due instance is executing the computing task at the current time and the current computing task is finished, releasing the upcoming due instance.
The strategy allows the executed task to continue to be executed and completed, and since the bidding instance is not released immediately after the aging period is over, the bidding instance can still be executed for a period of time after the aging period is over, and the current task can be executed and completed normally. No new tasks are scheduled to the instance, regardless of whether the task is eventually completed.
3. The strategy is used as much as possible: and continuing to schedule the task to the upcoming due instance until the upcoming due instance is released by the system.
4. The user manually selects the policy: releasing the upcoming due instance immediately according to a policy selection instruction input by a user; or, when the current computing task is finished, releasing the upcoming due instance; alternatively, the task continues to be scheduled to the upcoming expired instance until the upcoming expired instance is released by the system.
S405, according to the scheduling strategy, task allocation is carried out on the calculation examples in the cluster.
In the embodiment of the invention, after the scheduling policy is determined according to the instance information of the upcoming expiration instance or the instruction input by the user, the computing task submitted by the user is distributed to each computing instance (including the time-efficient instance and also including the ordinary instance without time-efficiency) according to the determined scheduling policy so as to perform computing operation.
In particular, when an immediate invalidation policy is used, the upcoming expired instance is released directly. When a task due invalidation policy is used, the due instance is released after completion or failure of execution of the task on the due instance. When the strategy of using as much as possible is used, the instance is reapplied after the instance is released by the system so as to supplement the cluster resources.
In addition, in the embodiment of the present invention, a resource pool mode may also be adopted for the application of the resource (computing instance). For example, a batch of bidding instance resources may be pre-applied while maintaining the number of bidding instance resources above a certain water line (i.e., a preset number threshold). When the released bidding instances are gradually increased, so that the number of the bidding instance resources in the resource pool is lower than the water level line, the application of a new batch of bidding instance resources is triggered. This eliminates the need for detecting and outdated replenishment operations on a time-sensitive instance-by-time basis. Therefore, the cluster scheduling method provided by the embodiment of the present invention may further include the following steps.
S406, when the number of the time-efficient instances in the cluster is lower than a preset number threshold, applying for a new time-efficient instance.
In the real-time process of the present invention, step S406 may be executed as long as the condition is met (i.e., the number of the time-sensitive instances in the cluster is lower than the preset number threshold), which is not in sequence with the execution process of steps S402 to S405.
In addition, when applying for the bidding instance resources, a batch time-sharing application mode can be adopted to divide the application of a batch of bidding instance resources into several batches, and each batch is applied at staggered time, so that the released time of the bidding instances is staggered to balance available resources.
According to the cluster scheduling method provided by the embodiment of the invention, the corresponding scheduling strategy is determined by monitoring the timeliness information of the timeliness instance so as to distribute tasks to the calculation instance in the cluster; corresponding releasing operation is carried out on the examples which are about to expire according to the scheduling strategy, so that the elastic advantage of time-efficient computing resources is exerted, cluster scheduling can be further optimized, the resource utilization rate is improved, meanwhile, the cluster can obtain resources with lower cost, and the reliability of computing is guaranteed.
Example four
Fig. 5 is a schematic structural diagram of an embodiment of a cluster scheduling apparatus provided in the present invention, which may be used to execute the method steps shown in fig. 3. As shown in fig. 5, a cluster scheduling apparatus provided in an embodiment of the present invention includes: an information acquisition module 51, a policy determination module 52 and an execution module 53.
The information obtaining module 51 is configured to obtain aging information of an aging example in a cluster; the strategy determining module 52 is used for determining a scheduling strategy according to the aging information of the aging example; the execution module 53 is configured to perform task allocation on the computing instances in the cluster according to the scheduling policy.
In the embodiment of the present invention, after a cluster is created, the information obtaining module 51 needs to monitor each computing instance in the cluster, obtain aging information of each aging instance, and the policy determining module 52 determines a corresponding scheduling policy according to the aging information. Then, the execution module 53 distributes the computing tasks submitted by the users to each computing instance (including the time-efficient instance, or including the ordinary instance without time-efficiency) according to the determined scheduling policy, so as to perform the computing operation.
The cluster scheduling device provided by the embodiment of the invention determines the corresponding scheduling strategy by monitoring the timeliness information of the timeliness example so as to perform task allocation on the calculation example in the cluster, thereby exerting the elastic advantage of timeliness calculation resources, further optimizing cluster scheduling and improving the resource utilization rate, and meanwhile, the cluster can obtain resources with lower cost and ensure the reliability of calculation.
EXAMPLE five
Fig. 6 is a schematic structural diagram of another embodiment of the cluster scheduling apparatus provided in the present invention. As shown in fig. 6, on the basis of the embodiment shown in fig. 5, in the cluster scheduling apparatus provided in this embodiment, the policy determining module 52 may include: a first determination unit 521 and a second determination unit 522.
The first determining unit 521 may be configured to determine an upcoming expiration instance according to an expiration time of the aging instance; the second determining unit 522 may be configured to determine the scheduling policy according to the instance information of the upcoming expiration instance.
In the embodiment of the present invention, the upcoming expiration instance is an instance whose time difference between the expiration time and the current time is smaller than a preset time threshold. When determining the scheduling policy, first, the first determining unit 521 determines an upcoming expiration instance according to the expiration time of the time-sensitive instance. Then, the second determining unit 522 determines an appropriate scheduling policy according to the instance information (e.g., the computing task on the instance, etc.) of the upcoming due instance determined by the first determining unit 521.
Specifically, the scheduling policy determined according to the instance detail information of the upcoming expired instance in the embodiment of the present invention may include:
1. immediate failure strategy: when the upcoming expired instance does not execute the computing task at the current time or fails to execute the computing task, the upcoming expired instance is immediately released.
2. Task expiration invalidation policy: when the upcoming due instance is executing the computing task at the current time and the current computing task is finished, releasing the upcoming due instance.
3. The strategy is used as much as possible: and continuing to schedule the task to the upcoming due instance until the upcoming due instance is released by the system.
4. The user manually selects the policy: releasing the upcoming due instance immediately according to a policy selection instruction input by a user; or, when the current computing task is finished, releasing the upcoming due instance; alternatively, the task continues to be scheduled to the upcoming expired instance until the upcoming expired instance is released by the system.
Further, the cluster scheduling apparatus provided in the embodiment of the present invention may further include: a common module 61. The creation module 61 may be configured to apply for a computing instance to create a cluster.
In the embodiment of the present invention, when the creating module 61 creates a cluster, the computing instance applied for may be completely composed of time-sensitive instances, or may include a certain number of non-time-sensitive common instances. When all instances are released, task scheduling may be interrupted and may continue until a new instance joins the cluster.
Further, the creating module 61 may be further configured to apply for a new aging example when the number of aging examples in the cluster is lower than a preset number threshold.
In the embodiment of the present invention, a resource pool manner may also be adopted for the application of the resource (computing instance). For example, creation module 61 may pre-apply for a batch of bidding instance resources while maintaining the number of bidding instance resources above a certain water line (i.e., a preset number threshold). When the released bid instances are gradually increased such that the number of bid instance resources in the resource pool is below the water line, the creation module 61 may trigger a re-application for a new batch of bid instance resources. This eliminates the need for detecting and outdated replenishment operations on a time-sensitive instance-by-time basis. In addition, when the creating module 61 applies for the bidding instance resources, it may adopt a batch time-sharing application mode to divide a batch of applications for the bidding instance resources into several batches, and each batch is time-staggered to apply, so that the time for releasing the bidding instances is also staggered to balance the available resources.
The cluster scheduling device provided by the embodiment of the invention determines a corresponding scheduling strategy by monitoring the timeliness information of the timeliness instance so as to distribute tasks to the calculation instances in the cluster; corresponding releasing operation is carried out on the examples which are about to expire according to the scheduling strategy, so that the elastic advantage of time-efficient computing resources is exerted, cluster scheduling can be further optimized, the resource utilization rate is improved, meanwhile, the cluster can obtain resources with lower cost, and the reliability of computing is guaranteed.
EXAMPLE six
The internal functions and structure of the cluster scheduling system, which can be implemented as an electronic device, are described above. Fig. 7 is a schematic structural diagram of an embodiment of an electronic device provided in the present invention. As shown in fig. 7, the electronic device includes a memory 71 and a processor 72.
The memory 71 stores programs. In addition to the above-described programs, the memory 71 may also be configured to store other various data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phonebook data, messages, pictures, videos, and so forth.
The memory 71 may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
A processor 72, coupled to the memory 71, that executes programs stored by the memory 71 to:
acquiring the aging information of an aging example in a cluster;
determining a scheduling strategy according to the aging information of the aging example;
and according to the scheduling strategy, carrying out task allocation on the computing instances in the cluster.
Further, as shown in fig. 7, the electronic device may further include: communication components 73, power components 74, audio components 75, a display 76, and the like. Only some of the components are schematically shown in fig. 7, and the electronic device is not meant to include only the components shown in fig. 7.
The communication component 73 is configured to facilitate wired or wireless communication between the electronic device and other devices. The electronic device may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 73 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 73 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
A power supply component 74 provides power to the various components of the electronic device. The power components 74 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for an electronic device.
The audio component 75 is configured to output and/or input audio signals. For example, the audio component 75 includes a Microphone (MIC) configured to receive external audio signals when the electronic device is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal may further be stored in a memory 71 or transmitted via a communication component 73. In some embodiments, audio assembly 75 also includes a speaker for outputting audio signals.
The display 76 includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.
Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (16)
1. A method for cluster scheduling, comprising:
acquiring the aging information of an aging example in a cluster;
determining a scheduling strategy according to the aging information of the aging example;
and distributing tasks to the computing instances in the cluster according to the scheduling strategy.
2. The cluster scheduling method according to claim 1, wherein the aging information of the aging instance includes an expiration time of the aging instance, and the determining the scheduling policy according to the aging information of the aging instance comprises:
determining an upcoming expiration instance according to the expiration time of the timeliness instance, wherein the upcoming expiration instance is an instance of which the time difference between the expiration time and the current time is smaller than a preset time threshold;
and determining a scheduling policy according to the instance information of the upcoming expired instance.
3. The cluster scheduling method of claim 2, wherein the scheduling policy comprises:
and when the upcoming expired instance does not execute the computing task at the current moment or fails to execute the computing task, immediately releasing the upcoming expired instance.
4. The cluster scheduling method of claim 2, wherein the scheduling policy comprises:
and when the upcoming expired instance is executing the computing task at the current moment and the current computing task is finished, releasing the upcoming expired instance.
5. The cluster scheduling method of claim 2, wherein the scheduling policy comprises:
continuing to schedule tasks to the upcoming expired instance until the upcoming expired instance is released by the system.
6. The cluster scheduling method of claim 2, wherein the scheduling policy comprises:
releasing the upcoming due instance immediately according to a policy selection instruction input by a user; or, when the current computing task is finished, releasing the upcoming due instance; or continuing to schedule tasks to the upcoming expired instance until the upcoming expired instance is released by the system.
7. The cluster scheduling method according to any of claims 1 to 6, further comprising, before said obtaining age information of an age-effective instance in a cluster:
applying for a computing instance, and creating a cluster, wherein the computing instance comprises a time-sensitive instance.
8. The cluster scheduling method of claim 7, further comprising:
and applying for a new time-sensitive example when the number of the time-sensitive examples in the cluster is lower than a preset number threshold.
9. A cluster scheduling apparatus, comprising:
the information acquisition module is used for acquiring the aging information of the aging instances in the cluster;
the strategy determining module is used for determining a scheduling strategy according to the timeliness information of the timeliness example;
and the execution module is used for distributing tasks to the computing examples in the cluster according to the scheduling strategy.
10. The cluster scheduling device according to claim 9, wherein the aging information of the aging instance includes an expiration time of the aging instance, and the policy determining module comprises:
the first determining unit is used for determining an upcoming expiration example according to the expiration time of the timeliness example, wherein the upcoming expiration example is an example of which the time difference between the expiration time and the current time is smaller than a preset time threshold;
and the second determining unit is used for determining a scheduling policy according to the instance information of the upcoming expiration instance.
11. The cluster scheduling device according to claim 9 or 10, further comprising:
and the creating module is used for applying for a calculation example and creating a cluster, wherein the calculation example comprises a timeliness example.
12. A cluster scheduling system, comprising: a cluster resource management component and a cluster scheduler,
the cluster resource management component is used for acquiring the aging information of an aging instance in a cluster and sending the aging information to the cluster scheduler;
the cluster scheduler is used for receiving the aging information of the aging example sent by the cluster resource management component, determining a scheduling strategy according to the aging information of the aging example, and distributing tasks to the calculation examples in the cluster according to the scheduling strategy.
13. The cluster scheduling system of claim 12 wherein the age information of the age instances includes the expiration time of the age instances,
the cluster scheduler is specifically configured to determine an upcoming expiration instance according to an expiration time of the timeliness instance, where the upcoming expiration instance is an instance in which a time difference between the expiration time and a current time is smaller than a preset time threshold; determining a scheduling policy according to the instance information of the upcoming expiration instance; sending the scheduling strategy to the cluster resource management component;
the cluster resource management component is further configured to perform a release operation on the upcoming expired instance according to the scheduling policy sent by the cluster scheduler.
14. The cluster scheduling system of claim 12 wherein the cluster resource management component is further configured to,
applying for a computing instance and creating a cluster, wherein the computing instance comprises a timeliness instance;
and applying for a new timeliness example under the condition that the number of timeliness examples in the cluster is lower than a preset number threshold.
15. An electronic device, comprising:
a memory for storing a program;
a processor for executing the program stored in the memory for:
acquiring the aging information of an aging example in a cluster;
determining a scheduling strategy according to the aging information of the aging example;
and distributing tasks to the computing instances in the cluster according to the scheduling strategy.
16. A computer-readable storage medium having instructions stored thereon, the instructions comprising:
acquiring the aging information of an aging example in a cluster;
determining a scheduling strategy according to the aging information of the aging example;
and distributing tasks to the computing instances in the cluster according to the scheduling strategy.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811436068.9A CN111240811A (en) | 2018-11-28 | 2018-11-28 | Cluster scheduling method, device and system and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811436068.9A CN111240811A (en) | 2018-11-28 | 2018-11-28 | Cluster scheduling method, device and system and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111240811A true CN111240811A (en) | 2020-06-05 |
Family
ID=70871006
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811436068.9A Pending CN111240811A (en) | 2018-11-28 | 2018-11-28 | Cluster scheduling method, device and system and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111240811A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117707794A (en) * | 2024-02-05 | 2024-03-15 | 之江实验室 | Heterogeneous federation-oriented multi-class job distribution management method and system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130047259A1 (en) * | 2011-08-15 | 2013-02-21 | Bank Of America Corporation | Method and apparatus for token-based virtual machine recycling |
| CN103477323A (en) * | 2011-01-05 | 2013-12-25 | 阿尔卡特朗讯 | Seamless scaling of enterprise applications |
| CN103761147A (en) * | 2014-01-15 | 2014-04-30 | 清华大学 | Method and system for managing calculation examples in cloud platforms |
| US20140173614A1 (en) * | 2012-12-18 | 2014-06-19 | International Business Machines Corporation | Sending tasks between virtual machines based on expiration times |
| CN104737132A (en) * | 2012-09-12 | 2015-06-24 | 萨勒斯福斯通讯有限公司 | Auction-based resource sharing for message queues in an on-demand services environment |
| CN105242957A (en) * | 2015-09-28 | 2016-01-13 | 广州云晫信息科技有限公司 | Method and system for cloud computing system to allocate GPU resources to virtual machine |
| CN107209682A (en) * | 2014-12-05 | 2017-09-26 | 亚马逊技术有限公司 | The automatic management of resource adjustment |
-
2018
- 2018-11-28 CN CN201811436068.9A patent/CN111240811A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103477323A (en) * | 2011-01-05 | 2013-12-25 | 阿尔卡特朗讯 | Seamless scaling of enterprise applications |
| US20130047259A1 (en) * | 2011-08-15 | 2013-02-21 | Bank Of America Corporation | Method and apparatus for token-based virtual machine recycling |
| CN104737132A (en) * | 2012-09-12 | 2015-06-24 | 萨勒斯福斯通讯有限公司 | Auction-based resource sharing for message queues in an on-demand services environment |
| US20140173614A1 (en) * | 2012-12-18 | 2014-06-19 | International Business Machines Corporation | Sending tasks between virtual machines based on expiration times |
| CN103761147A (en) * | 2014-01-15 | 2014-04-30 | 清华大学 | Method and system for managing calculation examples in cloud platforms |
| CN107209682A (en) * | 2014-12-05 | 2017-09-26 | 亚马逊技术有限公司 | The automatic management of resource adjustment |
| CN105242957A (en) * | 2015-09-28 | 2016-01-13 | 广州云晫信息科技有限公司 | Method and system for cloud computing system to allocate GPU resources to virtual machine |
Non-Patent Citations (1)
| Title |
|---|
| 王泽武;孙磊;郭松辉;: "基于滚动优化的密码云实时任务阈值调度方法" * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117707794A (en) * | 2024-02-05 | 2024-03-15 | 之江实验室 | Heterogeneous federation-oriented multi-class job distribution management method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110489228B (en) | Resource scheduling method and electronic equipment | |
| US11294714B2 (en) | Method and apparatus for scheduling task, device and medium | |
| KR101475550B1 (en) | Aggregating recurrent schedules to optimize resource consumption | |
| CN102567110B (en) | Idle time service | |
| CN112988362B (en) | Task processing method and device, electronic equipment and storage medium | |
| CN109656725B (en) | Message consumer switching method and device, storage medium and electronic equipment | |
| KR20170033121A (en) | Method for processing service and electronic device for the same | |
| CN112463535A (en) | Multi-cluster exception handling method and device | |
| US8799474B2 (en) | Apparatus and method to allocate limited resources | |
| EP3756093B1 (en) | Dynamic processor power management | |
| CN111143045A (en) | Task scheduling method and device of smart home operating system and storage medium | |
| CN112445575A (en) | Multi-cluster resource scheduling method, device and system | |
| US10275007B2 (en) | Performance management for a multiple-CPU platform | |
| US20190086986A1 (en) | Achieving a consistent computing device battery drain rate | |
| CN111240811A (en) | Cluster scheduling method, device and system and electronic equipment | |
| CN114598665A (en) | Resource scheduling method and device, computer readable storage medium and electronic equipment | |
| CN112486658A (en) | Task scheduling method and device for task scheduling | |
| CN115858175B (en) | Asynchronous I/O request priority scheduling method, device, medium and control equipment | |
| US10042414B2 (en) | Concurrent network application scheduling for reduced power consumption | |
| CN109753339B (en) | Resource allocation method and device and electronic equipment | |
| CN112052077A (en) | Method, device, equipment and medium for software task management | |
| KR20120047352A (en) | Method of updating software in real-time system and apparatus for the same | |
| CN113114731B (en) | Task processing method, device, server and storage medium | |
| CN115048186A (en) | Method and device for processing expansion and contraction of service container, storage medium and electronic equipment | |
| CN110737320B (en) | Power consumption detection method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40031316 Country of ref document: HK |