CN113608834A - Resource scheduling method, device and equipment based on super-fusion and readable medium - Google Patents
Resource scheduling method, device and equipment based on super-fusion and readable medium Download PDFInfo
- Publication number
- CN113608834A CN113608834A CN202110862295.3A CN202110862295A CN113608834A CN 113608834 A CN113608834 A CN 113608834A CN 202110862295 A CN202110862295 A CN 202110862295A CN 113608834 A CN113608834 A CN 113608834A
- Authority
- CN
- China
- Prior art keywords
- queue
- virtual machine
- host
- super
- task request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a resource scheduling method based on super-fusion, which comprises the following steps: creating a super-fusion cluster, dividing a plurality of hosts in the super-fusion cluster into a plurality of host groups, and creating a plurality of virtual machines in each host group; in response to receiving a queue task request, selecting a target virtual machine based on the number of queue tags of the host cluster and the resource use condition, executing the queue task request by scheduling the target virtual machine, and adding a queue tag on the target virtual machine; detecting the sum of the number of queue tags of all the host computer groups, and judging whether to enter a high concurrency mode or not based on the resource use condition of the host computer groups; and if the virtual machine enters the high concurrency mode, performing multiple increase on the virtual core granularity of the virtual machine according to the priority of the queue task request executed on the virtual machine. The invention also discloses a resource scheduling device, computer equipment and a readable storage medium based on the super-fusion.
Description
Technical Field
The invention relates to the technical field of resource scheduling, in particular to a resource scheduling method, device, equipment and readable medium based on super-fusion.
Background
In the big data era, in order to store and process massive data, a large-scale server cluster or a data center is needed, generally, a large number of types of numerous and diverse application programs and services are operated on the clusters, the number of cluster nodes of a large manufacturer is hundreds of nodes and thousands of nodes at present, the number of nodes of a big data cluster is increased to a new height by combining cloud data in recent two years, and part of domestic leading manufacturers pass the big data cluster authentication of tens of thousands of nodes on the cloud. The big data cluster built on the super-fusion cloud is a popular trend of nearly one year and two years, and the number of big data nodes on the super-fusion cloud is large and the growth speed is high.
Yarn is the mainstream resource scheduling component of big data cluster, has adopted master/slave architecture, and master is responsible for carrying out unified management and dispatch to the resource on each node, and Yarn's basic architecture contains: resource manager, ApplicationMaster, NodeManager, Container. Resource scheduler is one of the most core components of Yarn, a plug-in service component in RM.
In the prior art, the scheduling modes of a resource scheduler mainly include the following: the FIFO Scheduler arranges the applications into a queue according to the submitted sequence and then allocates resources according to the queue sequence; the whole Capacity Scheduler cluster can provide services for a plurality of organizations or units by setting a plurality of queues, and prevent resources from being excessively occupied or unavailable by configuring maximum and minimum resource parameters; the design goal of the Fair Scheduler is to allocate Fair resources for all applications, and how to allocate fairly is through parameter setting; label scheduling sets a plurality of labels for each Node and a queue in a scheduler so as to limit the queue to only occupy resource nodes containing corresponding labels, and can be mixed with other scheduling strategies for use; the resource preemption model sets a minimum resource quantity and a maximum resource quantity in the resource scheduler, the resource scheduler can temporarily allocate the resource of the queue with lower load to the queue with heavy load, the queue resource with lower load meets the minimum resource quantity per se, and other resources are temporarily preempted for use.
However, Yarn's resource scheduling aspect is out-of-order scheduling, and in the case of a small job, a task is selected to run in its own JVM (Java Virtual Machine), and in the case of a large job, the task is scheduled according to data information of the task and allocated to a node storing data. When the task information is evaluated, the self resources are consumed for operation, and the overall task performance is reduced. When the task is scheduled, a proper node is selected to operate according to the resources required by the resource allocation unit, the allocated resources are determined at the moment, and the resources can be released only after the task is completed, so that the resource calculation power of the low-priority task cannot be fully utilized.
Disclosure of Invention
In view of this, an object of the embodiments of the present invention is to provide a resource scheduling method, device, equipment and readable medium based on super-fusion, which optimize task resource allocation and task monitoring, allocate task resources reasonably and orderly, and reduce resource loss during allocation operation under a super-fusion framework. And judging a high concurrency threshold value according to the task monitoring index, performing priority division on the high concurrency task, and distributing resource calculation power to the high priority task under the condition of not influencing the normal running of the low priority task, so that the high priority task can be effectively and quickly executed. The task adjustment is realized through the control of a high concurrency mode, the overall operation efficiency is improved and the operation period is shortened under the condition of limited time and resources.
Based on the above object, an aspect of the embodiments of the present invention provides a resource scheduling method based on super-fusion, including the following steps: creating a super-fusion cluster, dividing a plurality of hosts in the super-fusion cluster into a plurality of host groups, and creating a plurality of virtual machines in each host group; in response to receiving a queue task request, selecting a target virtual machine based on the number of queue tags of the host cluster and the resource use condition, executing the queue task request by scheduling the target virtual machine, and adding a queue tag on the target virtual machine; detecting the sum of the number of queue tags of all the host computer groups, and judging whether to enter a high concurrency mode or not based on the resource use condition of the host computer groups; and if the virtual machine enters the high concurrency mode, performing multiple increase on the virtual core granularity of the virtual machine according to the priority of the queue task request executed on the virtual machine.
In some embodiments, further comprising: periodically detecting the sum of the number of queue tags of all the host computer groups, and judging whether to exit from a high concurrency mode or not based on the resource use condition of the host computer groups; if the virtual machine exits the high concurrency mode, setting the virtual machine for performing virtual kernel granularity adjustment to be in a state of not receiving a new queue task request; and in response to the completion of the execution of the queue task request which is being executed on the virtual machine, restoring the virtual core granularity of the virtual machine to the initial configuration, and setting the virtual core granularity to be in a state of receiving a new queue task request and adding the new queue task request.
In some embodiments, further comprising: and in response to the completion of the queue task execution request of the virtual machine, deleting the corresponding queue tag.
In some embodiments, further comprising: and setting the priority of the queue task request according to the priority of the user submitting the queue task request.
In some embodiments, selecting a target virtual machine based on the number of queue tags and resource usage for the host cluster comprises: traversing according to the sequence of the number of the queue tags of the host cluster from small to large, and selecting a virtual machine with available resources larger than the resources required by the queue task request in the host cluster as a target virtual machine.
In some embodiments, detecting a sum of the number of queue tags of all the host clusters, and determining whether to enter a high concurrency mode based on resource usage of the host clusters includes: detecting the number of queue tags of all the host computer groups, summing the number of queue tags to obtain the sum of the number of queue tags, and judging whether the sum of the number of queue tags exceeds a preset number or not; if the sum of the number of the queue tags exceeds a preset number, further judging whether the resource use condition index of the host computer group exceeds a preset index; and if the resource use condition index of the host computer group exceeds a preset index, confirming to enter a high concurrency mode.
In some embodiments, multiplying the virtual core granularity of the virtual machine according to the priority of the queued task requests executing on the virtual machine comprises: judging whether a queue task request submitted by a user with low priority exists; if the queue task request submitted by the user with low priority exists, returning a virtual machine list used by the queue task request; and performing multiple increase on the virtual core granularity of the virtual machines in the virtual machine list based on the priority.
In another aspect of the embodiments of the present invention, a resource scheduling apparatus based on super-fusion is further provided, including: the system comprises a first module, a second module and a third module, wherein the first module is configured to create a super-fusion cluster, divide a plurality of hosts in the super-fusion cluster into a plurality of host groups, and create a plurality of virtual machines in each host group; a second module, configured to, in response to receiving a queue task request, select a target virtual machine based on the number of queue tags and resource usage of the host cluster, execute the queue task request by scheduling the target virtual machine, and add a queue tag to the target virtual machine; the third module is configured to detect the sum of the number of queue tags of all the host computer groups and judge whether to enter a high concurrency mode or not based on the resource use condition of the host computer groups; and the fourth module is configured to, if the virtual machine enters the high concurrency mode, multiply increase the virtual core granularity of the virtual machine according to the priority of the queue task request executed on the virtual machine.
In another aspect of the embodiments of the present invention, there is also provided a computer device, including: at least one processor; and a memory storing computer instructions executable on the processor, the instructions when executed by the processor implementing the steps of the method.
In a further aspect of the embodiments of the present invention, a computer-readable storage medium is also provided, in which a computer program for implementing the above method steps is stored when the computer program is executed by a processor.
The invention has the following beneficial technical effects: under the super-fusion framework, the task resource allocation and the task monitoring are optimized, the task resources are allocated reasonably and orderly, and the resource loss during allocation operation is reduced. And judging a high concurrency threshold value according to the task monitoring index, performing priority division on the high concurrency task, and distributing resource calculation power to the high priority task under the condition of not influencing the normal running of the low priority task, so that the high priority task can be effectively and quickly executed. The task adjustment is realized through the control of a high concurrency mode, the overall operation efficiency is improved and the operation period is shortened under the condition of limited time and resources.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.
Fig. 1 is a schematic diagram of an embodiment of a resource scheduling method based on super-fusion according to the present invention;
fig. 2 is a schematic diagram of an embodiment of a resource scheduling apparatus based on super-fusion according to the present invention;
FIG. 3 is a schematic diagram of an embodiment of a computer device provided by the present invention;
FIG. 4 is a schematic diagram of an embodiment of a computer-readable storage medium provided by the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.
It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are used for distinguishing two entities with the same name but different names or different parameters, and it should be noted that "first" and "second" are merely for convenience of description and should not be construed as limitations of the embodiments of the present invention, and they are not described in any more detail in the following embodiments.
In view of the above, a first aspect of the embodiments of the present invention provides an embodiment of a resource scheduling method based on super-fusion. Fig. 1 is a schematic diagram illustrating an embodiment of a resource scheduling method based on super-fusion according to the present invention. As shown in fig. 1, the embodiment of the present invention includes the following steps:
s01, creating a super fusion cluster, dividing a plurality of hosts in the super fusion cluster into a plurality of host clusters, and creating a plurality of virtual machines in each host cluster;
s02, responding to the received queue task request, selecting a target virtual machine based on the number of queue tags of the host group and the resource use condition, executing the queue task request by scheduling the target virtual machine, and adding the queue tag on the target virtual machine;
s03, detecting the sum of the number of queue labels of all the host clusters, and judging whether to enter a high concurrency mode or not based on the resource use condition of the host clusters; and
and S04, if the virtual machine enters the high concurrency mode, multiplying the virtual core granularity of the virtual machine according to the priority of the queue task request executed on the virtual machine.
In this embodiment, the queue is reasonably scheduled by adding the queue tag of the virtual machine and requesting the resource condition through the physical resource and the task. The problem of out-of-order resource allocation of the yarn scheduler is solved. By optimizing the detection function of the ics monitoring module, resource indexes and detailed information of all queues and tasks can be obtained, and the index information is judged whether the threshold value of the high concurrent task is reached. And scheduling the task resources under the condition of high concurrent tasks, and scheduling the computing resources of the CPUs of the tasks with medium and lower priorities into the tasks with high priorities under the condition of not changing the task resources of the tasks. The problems of low operation efficiency and long operation period during high concurrent tasks are solved. And planning user group types, and judging task levels according to user group priorities. And executing the virtualized core particle size modification on the tasks with low levels in the high concurrency mode, and distributing the calculation power to the tasks with higher priorities.
In this embodiment, the yann super-fusion cluster establishment and queue generation setting includes: building a cluster on the hyper-fusion cloud, and deploying a virtual machine on the hyper-fusion platform; and setting the priority of the virtual machine in the yann queue. Wherein, the building of the super-fusion cloud cluster comprises the following steps: selecting a super-fusion all-in-one machine to construct a cloud platform, and converting the super-fusion all-in-one machine into a computing management node and a computing node; deploying a local storage pool and sds nodes, establishing the local storage pool by the storage devices of all the server nodes in batch, and establishing sds virtual machines on each server host node on the super-fusion platform to ensure data intercommunication and data balance. Deploying virtual machines on a hyper-converged platform comprises: adding a queue tag on each virtual machine, wherein the queue tag is triggered to take effect when issued by a horn queue, and the virtual machines in the same queue group can be controlled in batch on the super-fusion cloud platform; the yarn queue virtual machine priority presetting comprises the following steps: the virtual machines which are subordinate to different hosts and divided into the same queue have higher priority than the virtual machines which are divided into the same host by vm of the same host and divided into the same queue.
In some embodiments of the invention, further comprising: periodically detecting the sum of the number of queue tags of all the host computer groups, and judging whether to exit from a high concurrency mode or not based on the resource use condition of the host computer groups; if the virtual machine exits from the high concurrency mode, setting the virtual machine for performing virtual kernel granularity adjustment to be in a state of not receiving a new queue task request; and in response to the completion of the execution of the queue task request which is being executed on the virtual machine, restoring the virtual core granularity of the virtual machine to the initial configuration, and setting the virtual core granularity to be in a state of receiving a new queue task request and adding.
In this embodiment, when it is detected that the number of queue tags is smaller than the preset number or the resource utilization index is smaller than the preset index, the high concurrency mode exits, the virtual machine performing virtual core particle size adjustment does not receive the addition of a new queue after executing the queue of the current round, the configuration recovery is executed, and the new queue is normally received after the configuration recovery.
In some embodiments of the invention, further comprising: and in response to the completion of the queue task execution request of the virtual machine, deleting the corresponding queue tag.
In some embodiments of the invention, further comprising: and setting the priority of the queue task request according to the priority of the user submitting the queue task request.
In this embodiment, a user group is established, different users are selected for each submission queue task, and the tasks are classified into a high priority, a higher priority, a medium priority and a low priority according to the task priority.
In some embodiments of the present invention, selecting the target virtual machine based on the number of queue tags and the resource usage of the host cluster comprises: traversing according to the sequence of the number of the queue tags of the host cluster from small to large, and selecting a virtual machine as a target virtual machine, wherein the available resource of the host cluster is larger than the resource required by the queue task request.
In this embodiment, n physical server hosts which are super-converged are divided into n multiple virtual machines according to actual requirements, if each physical server host creates m virtual machines, the total number of the virtual machines is m × n, and the virtual cpu resource allocated to each virtual machine is x. After the queue is started, the queue resource preferentially utilizes m1 of a virtual machine resource n 1.
When the same queue is started at m1 of n1, whether the ratio of x to the number of queue tasks is smaller than the cpu resource required by a single task is judged, if the ratio of x to the number of queue tasks is smaller than the cpu resource required by the single task, the queue is preferably started on a non-n 1 virtual machine, for example, the m1 virtual machine resource of n2 is scheduled.
When a new queue is initiated, the queue labels of the virtual machines of the n hosts are traversed, and the new queue is preferentially initiated on the host with the least queue label.
In some embodiments of the present invention, detecting a sum of the number of queue tags of all host clusters, and determining whether to enter the high concurrency mode based on the resource usage of the host clusters includes: detecting the number of queue tags of all host groups, summing the number of queue tags to obtain the total number of queue tags, and judging whether the total number of queue tags exceeds a preset number or not; if the sum of the number of the queue tags exceeds the preset number, further judging whether the resource use condition index of the host computer group exceeds a preset index; and if the resource use condition index of the host computer group exceeds a preset index, confirming to enter a high concurrency mode.
In this embodiment, all the virtual machines are monitored by the ics monitoring module, and index information required by the adjustment queue is acquired. Presetting a threshold value of the number of queue tags as m, and detecting the sum of the number of queue tags on all virtual machines of a platform; when the sum of the number of the queue tags is larger than m, starting to detect the CPU service condition and the memory service condition of each physical server host; and when the CPU service condition and the memory service condition exceed the threshold value, judging to enter a high concurrency mode.
In this embodiment, when the total number of queue tags is less than m, the polling monitoring module is not turned on. Resources are used by existing queues.
In some embodiments of the invention, multiplying the virtual core granularity of the virtual machine according to the priority of the queue task request executing on the virtual machine comprises: judging whether a queue task request submitted by a user with low priority exists; if the queue task request submitted by the user with low priority exists, returning a virtual machine list used by the queue task request; and multiplying the virtual core granularity of the virtual machines in the virtual machine list based on the priority.
In this embodiment, after entering the high concurrency mode, it is determined whether a task queue submitted by a low-priority user exists, and if so, a virtual machine host list used by the queue is returned; and adjusting the configuration groups of different virtual machines, taking the same queues of the returned virtual machine list as a group, and adjusting the virtual granularity of the virtual cores through virtual core technology adjustment, wherein the virtual granularity of the virtual machine where the medium priority queue is located is 2 times of the initial virtual granularity, and the virtual granularity of the virtual machine where the low priority queue is located is 3 times of the initial granularity. The virtual core granularity configuration is effective, the number of cores used by the virtual machine in the queue is increased, and partial virtual machines in the queue are automatically released. The computational power of the medium and low priority queues is reduced while keeping the queues unchanged.
In some embodiments of the invention, further comprising: and adding an optimization identifier to the virtual machine for adjusting the virtual core granularity, wherein the virtual machine cannot be optimized again before executing the task.
It should be particularly noted that, the steps in the above embodiments of the resource scheduling method based on super-fusion may be mutually intersected, replaced, added, and deleted, and therefore, these reasonable permutation and combination transformations should also belong to the scope of the present invention for the resource scheduling method based on super-fusion, and should not limit the scope of the present invention to the embodiments.
In view of the above object, a second aspect of the embodiments of the present invention provides a resource scheduling apparatus based on super-fusion. Fig. 2 is a schematic diagram illustrating an embodiment of a resource scheduling apparatus based on super-fusion according to the present invention. As shown in fig. 2, the embodiment of the present invention includes the following modules: a first module S11 configured to create a super-fusion cluster, divide a plurality of hosts in the super-fusion cluster into a plurality of host clusters, and create a plurality of virtual machines in each host cluster; a second module S12, configured to, in response to receiving the queue task request, select a target virtual machine based on the number of queue tags and the resource usage of the host cluster, execute the queue task request by scheduling the target virtual machine, and add the queue tag to the target virtual machine; a third module S13, configured to detect the sum of the number of queue tags of all host clusters, and determine whether to enter a high concurrency mode based on the resource usage of the host cluster; and a fourth module S14, configured to, if the virtual machine enters the high concurrency mode, multiply increase the virtual core granularity of the virtual machine according to the priority of the queue task request executed on the virtual machine.
In view of the above object, a third aspect of the embodiments of the present invention provides a computer device. Fig. 3 is a schematic diagram of an embodiment of a computer device provided by the present invention. As shown in fig. 3, an embodiment of the present invention includes the following means: at least one processor S21; and a memory S22, the memory S22 storing computer instructions S23 executable on the processor, the instructions when executed by the processor implementing the steps of the above method.
The invention also provides a computer readable storage medium. FIG. 4 is a schematic diagram illustrating an embodiment of a computer-readable storage medium provided by the present invention. As shown in fig. 4, the computer readable storage medium stores S31 a computer program that, when executed by a processor, performs the method as described above S32.
Finally, it should be noted that, as one of ordinary skill in the art can appreciate that all or part of the processes of the methods of the above embodiments can be implemented by a computer program to instruct related hardware, and the program of the resource scheduling method based on super-fusion can be stored in a computer readable storage medium, and when executed, the program can include the processes of the embodiments of the methods as described above. The storage medium of the program may be a magnetic disk, an optical disk, a Read Only Memory (ROM), a Random Access Memory (RAM), or the like. The embodiments of the computer program may achieve the same or similar effects as any of the above-described method embodiments.
Furthermore, the methods disclosed according to embodiments of the present invention may also be implemented as a computer program executed by a processor, which may be stored in a computer-readable storage medium. Which when executed by a processor performs the above-described functions defined in the methods disclosed in embodiments of the invention.
Further, the above method steps and system elements may also be implemented using a controller and a computer readable storage medium for storing a computer program for causing the controller to implement the functions of the above steps or elements.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments of the present invention.
In one or more exemplary designs, the functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, includes Compact Disc (CD), laser disc, optical disc, Digital Versatile Disc (DVD), floppy disk, blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.
It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.
The numbers of the embodiments disclosed in the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, of embodiments of the invention is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.
Claims (10)
1. A resource scheduling method based on super-fusion is characterized by comprising the following steps:
creating a super-fusion cluster, dividing a plurality of hosts in the super-fusion cluster into a plurality of host groups, and creating a plurality of virtual machines in each host group;
in response to receiving a queue task request, selecting a target virtual machine based on the number of queue tags of the host cluster and the resource use condition, executing the queue task request by scheduling the target virtual machine, and adding a queue tag on the target virtual machine;
detecting the sum of the number of queue tags of all the host computer groups, and judging whether to enter a high concurrency mode or not based on the resource use condition of the host computer groups; and
and if the virtual machine enters the high concurrency mode, performing multiple increase on the virtual core granularity of the virtual machine according to the priority of the queue task request executed on the virtual machine.
2. The method for resource scheduling based on super-fusion according to claim 1, further comprising:
periodically detecting the sum of the number of queue tags of all the host computer groups, and judging whether to exit from a high concurrency mode or not based on the resource use condition of the host computer groups;
if the virtual machine exits the high concurrency mode, setting the virtual machine for performing virtual kernel granularity adjustment to be in a state of not receiving a new queue task request;
and in response to the completion of the execution of the queue task request which is being executed on the virtual machine, restoring the virtual core granularity of the virtual machine to the initial configuration, and setting the virtual core granularity to be in a state of receiving a new queue task request and adding the new queue task request.
3. The method for resource scheduling based on super-fusion according to claim 1, further comprising:
and in response to the completion of the queue task execution request of the virtual machine, deleting the corresponding queue tag.
4. The method for resource scheduling based on super-fusion according to claim 1, further comprising:
and setting the priority of the queue task request according to the priority of the user submitting the queue task request.
5. The resource scheduling method based on super fusion of claim 1, wherein selecting a target virtual machine based on the number of queue tags and the resource usage of the host cluster comprises:
traversing according to the sequence of the number of the queue tags of the host cluster from small to large, and selecting a virtual machine with available resources larger than the resources required by the queue task request in the host cluster as a target virtual machine.
6. The resource scheduling method based on super fusion of claim 1, wherein detecting the sum of the number of queue tags of all the host clusters, and determining whether to enter a high concurrency mode based on the resource usage of the host clusters comprises:
detecting the number of queue tags of all the host computer groups, summing the number of queue tags to obtain the sum of the number of queue tags, and judging whether the sum of the number of queue tags exceeds a preset number or not;
if the sum of the number of the queue tags exceeds a preset number, further judging whether the resource use condition index of the host computer group exceeds a preset index;
and if the resource use condition index of the host computer group exceeds a preset index, confirming to enter a high concurrency mode.
7. The method according to claim 4, wherein the multiplying the virtual core granularity of the virtual machine according to the priority of the queue task request executed on the virtual machine comprises:
judging whether a queue task request submitted by a user with low priority exists;
if the queue task request submitted by the user with low priority exists, returning a virtual machine list used by the queue task request;
and performing multiple increase on the virtual core granularity of the virtual machines in the virtual machine list based on the priority.
8. A resource scheduling apparatus based on super convergence, comprising:
the system comprises a first module, a second module and a third module, wherein the first module is configured to create a super-fusion cluster, divide a plurality of hosts in the super-fusion cluster into a plurality of host groups, and create a plurality of virtual machines in each host group;
a second module, configured to, in response to receiving a queue task request, select a target virtual machine based on the number of queue tags and resource usage of the host cluster, execute the queue task request by scheduling the target virtual machine, and add a queue tag to the target virtual machine;
the third module is configured to detect the sum of the number of queue tags of all the host computer groups and judge whether to enter a high concurrency mode or not based on the resource use condition of the host computer groups; and
and if the virtual machine enters the high concurrency mode, performing multiple increase on the virtual core granularity of the virtual machine according to the priority of the queue task request executed on the virtual machine.
9. A computer device, comprising:
at least one processor; and
a memory storing computer instructions executable on the processor, the instructions when executed by the processor implementing the steps of the method of any one of claims 1 to 7.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110862295.3A CN113608834A (en) | 2021-07-29 | 2021-07-29 | Resource scheduling method, device and equipment based on super-fusion and readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110862295.3A CN113608834A (en) | 2021-07-29 | 2021-07-29 | Resource scheduling method, device and equipment based on super-fusion and readable medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113608834A true CN113608834A (en) | 2021-11-05 |
Family
ID=78305944
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110862295.3A Pending CN113608834A (en) | 2021-07-29 | 2021-07-29 | Resource scheduling method, device and equipment based on super-fusion and readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113608834A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114500401A (en) * | 2022-01-21 | 2022-05-13 | 上海金融期货信息技术有限公司 | Resource scheduling method and system for dealing with burst traffic |
| CN115103004A (en) * | 2022-06-30 | 2022-09-23 | 浪潮电子信息产业股份有限公司 | Session establishing method, device, equipment and storage medium |
| CN115277813A (en) * | 2022-07-19 | 2022-11-01 | 北京志凌海纳科技有限公司 | Super-fusion cluster host resource control method, system, equipment and readable medium |
| CN117435142A (en) * | 2023-12-12 | 2024-01-23 | 苏州元脑智能科技有限公司 | IO request scheduling method and storage device |
| CN117472516A (en) * | 2023-12-27 | 2024-01-30 | 苏州元脑智能科技有限公司 | Virtual resource scheduling method, device, cluster system, electronic equipment and medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105635311A (en) * | 2016-01-22 | 2016-06-01 | 广东亿迅科技有限公司 | Method for synchronizing resource pool information in cloud management platform |
| US20190294479A1 (en) * | 2017-02-06 | 2019-09-26 | Tencent Technology (Shenzhen) Company Limited | Resource scheduling method, system, server, and storage medium |
| CN110704186A (en) * | 2019-09-25 | 2020-01-17 | 国家计算机网络与信息安全管理中心 | Computing resource allocation method and device based on hybrid distribution architecture and storage medium |
| CN111338802A (en) * | 2020-02-29 | 2020-06-26 | 苏州浪潮智能科技有限公司 | Method, system, equipment and medium for optimizing performance of big data cluster |
-
2021
- 2021-07-29 CN CN202110862295.3A patent/CN113608834A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105635311A (en) * | 2016-01-22 | 2016-06-01 | 广东亿迅科技有限公司 | Method for synchronizing resource pool information in cloud management platform |
| US20190294479A1 (en) * | 2017-02-06 | 2019-09-26 | Tencent Technology (Shenzhen) Company Limited | Resource scheduling method, system, server, and storage medium |
| CN110704186A (en) * | 2019-09-25 | 2020-01-17 | 国家计算机网络与信息安全管理中心 | Computing resource allocation method and device based on hybrid distribution architecture and storage medium |
| CN111338802A (en) * | 2020-02-29 | 2020-06-26 | 苏州浪潮智能科技有限公司 | Method, system, equipment and medium for optimizing performance of big data cluster |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114500401A (en) * | 2022-01-21 | 2022-05-13 | 上海金融期货信息技术有限公司 | Resource scheduling method and system for dealing with burst traffic |
| CN114500401B (en) * | 2022-01-21 | 2023-11-14 | 上海金融期货信息技术有限公司 | Resource scheduling method and system for coping with burst traffic |
| CN115103004A (en) * | 2022-06-30 | 2022-09-23 | 浪潮电子信息产业股份有限公司 | Session establishing method, device, equipment and storage medium |
| CN115277813A (en) * | 2022-07-19 | 2022-11-01 | 北京志凌海纳科技有限公司 | Super-fusion cluster host resource control method, system, equipment and readable medium |
| CN117435142A (en) * | 2023-12-12 | 2024-01-23 | 苏州元脑智能科技有限公司 | IO request scheduling method and storage device |
| CN117435142B (en) * | 2023-12-12 | 2024-03-01 | 苏州元脑智能科技有限公司 | IO request scheduling method and storage device |
| CN117472516A (en) * | 2023-12-27 | 2024-01-30 | 苏州元脑智能科技有限公司 | Virtual resource scheduling method, device, cluster system, electronic equipment and medium |
| CN117472516B (en) * | 2023-12-27 | 2024-03-29 | 苏州元脑智能科技有限公司 | Virtual resource scheduling method, device, cluster system, electronic equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113608834A (en) | Resource scheduling method, device and equipment based on super-fusion and readable medium | |
| EP3761170B1 (en) | Virtual machine creation method and apparatus | |
| US20190324819A1 (en) | Distributed-system task assignment method and apparatus | |
| US9419859B2 (en) | Service allocation in a distributed computing platform | |
| CN111399970B (en) | Reserved resource management method, device and storage medium | |
| CN111338774A (en) | Distributed timing task scheduling system and computing device | |
| CN110221920B (en) | Deployment method, device, storage medium and system | |
| JP2013506179A (en) | Execution management system combining instruction threads and management method | |
| CN111338802B (en) | Method, system, equipment and medium for optimizing performance of big data cluster | |
| US11740921B2 (en) | Coordinated container scheduling for improved resource allocation in virtual computing environment | |
| US11403149B2 (en) | Management of a virtual network function | |
| CN111338785A (en) | Resource scheduling method and device, electronic equipment and storage medium | |
| CN111427675A (en) | Data processing method and device and computer readable storage medium | |
| WO2023174037A1 (en) | Resource scheduling method, apparatus and system, device, medium, and program product | |
| CN114610497A (en) | Container scheduling method, cluster system, device, electronic equipment and storage medium | |
| CN111464331B (en) | Control method and system for thread creation and terminal equipment | |
| US20220129294A1 (en) | Virtualization platform and virtualization platform scaling management method | |
| CN110795234A (en) | Resource scheduling method and device | |
| CN113301087B (en) | Resource scheduling method, device, computing equipment and medium | |
| CN113626173A (en) | Scheduling method, device and storage medium | |
| WO2023020177A1 (en) | Task scheduling method, game engine, device and storage medium | |
| KR102014246B1 (en) | Mesos process apparatus for unified management of resource and method for the same | |
| CN112015515B (en) | Instantiation method and device of virtual network function | |
| JP6339978B2 (en) | Resource allocation management device and resource allocation management method | |
| Lavacca | Scheduling Jobs on Federation of Kubernetes Clusters |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20211105 |
|
| RJ01 | Rejection of invention patent application after publication |