CN114090267A - Resource allocation method, device, equipment and medium based on dynamic resource view - Google Patents
Resource allocation method, device, equipment and medium based on dynamic resource view Download PDFInfo
- Publication number
- CN114090267A CN114090267A CN202111504156.XA CN202111504156A CN114090267A CN 114090267 A CN114090267 A CN 114090267A CN 202111504156 A CN202111504156 A CN 202111504156A CN 114090267 A CN114090267 A CN 114090267A
- Authority
- CN
- China
- Prior art keywords
- dynamic
- resource
- cluster
- level
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000013468 resource allocation Methods 0.000 title claims abstract description 32
- 230000003068 static effect Effects 0.000 claims description 8
- 238000001514 detection method Methods 0.000 claims description 3
- 238000004891 communication Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5015—Service provider selection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5021—Priority
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5022—Workload threshold
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application relates to a resource allocation method, a device, electronic equipment and a storage medium based on a dynamic resource view, which are applied to an off-line hybrid deployment task, wherein the method comprises the following steps: defining the level of the task; customizing the dynamic actually available resources of the cluster; calculating the real-time resource utilization rate of each service to predict the utilization rate of the host, and calculating the dynamic actual available resources of the cluster according to the utilization rate of the host; determining a dynamic resource view of the cluster according to the dynamic actual available resources of the cluster; and allocating resources according to the dynamic resource view and the level of the task. According to the method and the system, the dynamic resource sensor is arranged at each physical node, the resource utilization rate can be sensed in real time, so that the distributable dynamic resources of each node can be calculated, the dynamic resource view of the cluster is determined according to the dynamic resources of the cluster, and the scheduler of the control center distributes resources for tasks of different levels according to the dynamic resource view, so that mixed deployment of offline tasks and online tasks is realized, and the resource utilization rate is effectively improved.
Description
Technical Field
The present application relates to the field of resource allocation technologies, and in particular, to a method and an apparatus for resource allocation based on a dynamic resource view, an electronic device, and a storage medium.
Background
Currently cluster resource usage is dynamic, while quotas are static limits. The online service can estimate a Request and a Limit according to a peak value used by the online service, the Quota cannot be modified after application, but the resource usage is dynamic, the usage in the day and the usage in the night are possibly different, the resource utilization rate of the whole cluster is in a lower state when the online service is idle, and the resource waste condition is serious; the cluster native scheduler built based on kubernets does not sense the usage of real resources of the service on the cluster.
Disclosure of Invention
In view of the foregoing, the present application provides a resource allocation method based on a dynamic resource view, an electronic device, and a storage medium.
In a first aspect, an embodiment of the present application provides a resource allocation method based on a dynamic resource view, which is applied to an off-line hybrid deployment task, and includes:
defining the level of the task;
customizing the dynamic actually available resources of the cluster;
calculating the real-time resource utilization rate of each service to predict the utilization rate of the host, and calculating the dynamic actual available resources of the cluster according to the utilization rate of the host;
determining a dynamic resource view of the cluster according to the dynamic actual available resources of the cluster;
and allocating resources according to the dynamic resource view and the level of the task.
Further, in the resource allocation method based on the dynamic resource view, the defining the level of the task includes:
defining the grades of the tasks as three grades, namely p1, p2 and p3 according to the priorities of the tasks;
where the p1 rating is highest for online traffic; p2 is used for offline tasks, and p3 is the lowest level used for offline tasks.
Further, in the above method for allocating resources based on a dynamic resource view, the customizing of the dynamic actually available resources of the cluster includes:
the dynamic actual available resources are graded and registered to the API-Server of the cluster; including the four p 2-cpu-allocates, p 2-mem-allocates, p 3-cpu-allocates and p 3-mem-allocates.
Further, in the above method for resource allocation based on a dynamic resource view, calculating a real-time resource utilization rate of each service to predict a utilization rate of a host, and calculating a dynamic actual available resource of a cluster according to the utilization rate of the host, the method includes:
deploying a dynamic resource detection plug-in at each node in the cluster, wherein the plug-in is used for calculating the utilization rate of a real-time resource utilization rate prediction host of each service;
calculating the dynamic actual available resources of the cluster according to the utilization rate of the host according to the following formula:
p2-CPU-allocate is the node total CPU resource-p 1 level task CPU real-time utilization rate-safety threshold;
p3-CPU-allocate ═ node total CPU resources-task CPU real-time utilization at the p1 level-task real-time utilization at the p2 level-security threshold.
Further, in the above method for allocating resources based on a dynamic resource view, allocating resources according to the dynamic resource view and the level of the task includes:
if the level of the task is the level p1, calling a native static resource map of the cluster to allocate resources;
if the task level is a p2 level, judging whether the task request is a dynamic resource request, if so, calling a dynamic resource map for scheduling;
if the task level is the p3 level, judging whether the task request is a dynamic resource request, if so, calling a dynamic resource map for scheduling.
Further, the above method for allocating resources based on dynamic resource views further includes: and allocating proper nodes for the offline tasks according to the existing resources, scheduling the tasks to the nodes, and starting the tasks.
Further, the above method for allocating resources based on dynamic resource views further includes: further comprising: the dynamic adjustment of p2 level and p3 level resources is performed through the Cgroup mechanism.
In a second aspect, an embodiment of the present application further provides a resource allocation apparatus based on a dynamic resource view, which is applied to an off-line hybrid deployment task, and includes:
a definition module: for defining a level of a task;
a self-defining module: the method is used for customizing the dynamic actually available resources of the cluster;
a calculation module: the system comprises a plurality of servers, a plurality of clusters and a plurality of servers, wherein the servers are used for calculating real-time resource utilization rate of each service to predict the utilization rate of a host, and calculating dynamic actual available resources of the clusters according to the utilization rate of the host;
a determination module: the dynamic resource view is used for determining the cluster according to the dynamic actual available resources of the cluster;
a distribution module: and the system is used for allocating resources according to the dynamic resource view and the level of the task.
In a third aspect, an embodiment of the present application further provides an electronic device, including: a processor and a memory;
the processor is used for executing the resource allocation method based on the dynamic resource view by calling the program or the instruction stored in the memory.
In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores a program or instructions, and the program or instructions cause a computer to perform the above method for resource allocation based on a dynamic resource view.
The embodiment of the application has the advantages that: the application relates to a resource allocation method, a device, electronic equipment and a storage medium based on a dynamic resource view, which are applied to an off-line hybrid deployment task, wherein the method comprises the following steps: defining the level of the task; customizing the dynamic actually available resources of the cluster; calculating the real-time resource utilization rate of each service to predict the utilization rate of the host, and calculating the dynamic actual available resources of the cluster according to the utilization rate of the host; determining a dynamic resource view of the cluster according to the dynamic actual available resources of the cluster; and allocating resources according to the dynamic resource view and the level of the task. According to the method and the system, the dynamic resource sensor is arranged at each physical node, the resource utilization rate can be sensed in real time, so that the distributable dynamic resources of each node can be calculated, the dynamic resource view of the cluster is determined according to the dynamic resources of the cluster, and the scheduler of the control center distributes resources for tasks of different levels according to the dynamic resource view, so that mixed deployment of offline tasks and online tasks is realized, and the resource utilization rate is effectively improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments or the conventional technologies of the present application, the drawings used in the descriptions of the embodiments or the conventional technologies will be briefly introduced below, it is obvious that the drawings in the following descriptions are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a first schematic view illustrating a resource allocation method based on a dynamic resource view according to an embodiment of the present application;
fig. 2 is a schematic diagram of a resource allocation apparatus based on a dynamic resource view according to an embodiment of the present application;
fig. 3 is a schematic block diagram of an electronic device provided in an embodiment of the present application.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present application more comprehensible, embodiments accompanying the present application are described in detail below with reference to the accompanying drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is capable of embodiment in many different forms than that described herein and those skilled in the art will be able to make similar modifications without departing from the spirit of the application and therefore should not be limited to the specific embodiments disclosed below.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the description of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
Fig. 1 is a first schematic view of a resource allocation method based on a dynamic resource view according to an embodiment of the present application.
In a first aspect, an embodiment of the present application provides a resource allocation method based on a dynamic resource view, which is applied to an off-line hybrid deployment task, and with reference to fig. 1, includes five steps S101 to S1103:
s101: the level of the task is defined.
Specifically, in the embodiment of the present application, the defined task levels are classified into a p1 level, a p2 level, and a p3 level according to the priority of the task.
S102: and customizing the dynamically and actually available resources of the cluster.
Specifically, in the embodiment of the application, a resource expansion mode is adopted, dynamic actual available resources are graded and defined, and are registered to the API-Server of the cluster, and the cluster can sense the change of the dynamic actual available resources of the cluster.
S103: and calculating the real-time resource utilization rate of each service to predict the utilization rate of the host, and calculating the dynamic actual available resources of the cluster according to the utilization rate of the host.
Specifically, in the embodiment of the present application, dynamic-resource-plug-in is deployed at each node in the cluster, the plug-in calculates the real-time resource utilization rate of each service to predict the utilization rate of the host, and calculates the number of resources that can be dynamically allocated according to the utilization rate of the host.
S104: and determining a dynamic resource view of the cluster according to the dynamic actual available resources of the cluster.
Specifically, in the embodiment of the present application, after the number of dynamically allocatable resources is calculated, the plug-in of each node periodically sends the resources to the controller, and the controller collects the resources and arranges the resources into the dynamic resource view of the whole cluster.
S105: and allocating the resources according to the dynamic resource view and the level of the task.
Specifically, in the embodiment of the application, the scheduler of the control center allocates resources according to the dynamic resource view and the level of the task, so that the off-line task and the on-line task are mixed, and the utilization rate of the resources is effectively improved.
Further, in the resource allocation method based on the dynamic resource view, the defining the level of the task includes:
defining the grades of the tasks as three grades, namely p1, p2 and p3 according to the priorities of the tasks;
where the p1 rating is highest for online traffic; p2 is used for offline tasks, and p3 is the lowest level used for offline tasks.
Specifically, in the embodiment of the present application, the level of p1 is the highest, and is used for online services, the service of the service needs to specify the requested resource and the maximum resource at the time of starting, that is, the Request and limit fields, and the level of p3 is the lowest, and may be kill at any time.
Further, in the above method for allocating resources based on a dynamic resource view, the customizing of the dynamic actually available resources of the cluster includes:
the dynamic actual available resources are graded and registered to the API-Server of the cluster; including the four p 2-cpu-allocates, p 2-mem-allocates, p 3-cpu-allocates and p 3-mem-allocates.
Specifically, in the embodiment of the present application, a resource extension manner is adopted, dynamic actually available resources are classified and registered in the API-Server of the cluster, so that the cluster can sense the change of the user-defined resources, and 4 types of user-defined resources are defined in the present application: p2-cpu-allocate, p2-mem-allocate, p3-cpu-allocate and p 3-mem-allocate.
Further, in the above method for resource allocation based on a dynamic resource view, calculating a real-time resource utilization rate of each service to predict a utilization rate of a host, and calculating a dynamic actual available resource of a cluster according to the utilization rate of the host, the method includes:
deploying a dynamic resource detection plug-in at each node in the cluster, wherein the plug-in is used for calculating the utilization rate of a real-time resource utilization rate prediction host of each service;
specifically, in the embodiment of the present application, each node in the cluster deploys a dynamic-resource-plugin, and the plugin calculates the utilization rate of the real-time resource utilization rate prediction host of each service to calculate the number of dynamically allocatable resources.
Calculating the dynamic actual available resources of the cluster according to the utilization rate of the host according to the following formula:
p2-CPU-allocate is the node total CPU resource-p 1 level task CPU real-time utilization rate-safety threshold;
p3-CPU-allocate ═ node total CPU resources-task CPU real-time utilization at the p1 level-task real-time utilization at the p2 level-security threshold.
Specifically, in the embodiment of the present application, the plug-in of each node periodically sends the dynamic actually available resources of the cluster to the controller, and the controller collects and arranges the dynamic actually available resources of the cluster into the dynamic resource view of the whole cluster.
Further, in the above method for allocating resources based on a dynamic resource view, allocating resources according to the dynamic resource view and the level of the task includes:
if the level of the task is the level p1, calling a native static resource map of the cluster to allocate resources;
if the task level is a p2 level, judging whether the task request is a dynamic resource request, if so, calling a dynamic resource map for scheduling;
if the task level is the p3 level, judging whether the task request is a dynamic resource request, if so, calling a dynamic resource map for scheduling.
Specifically, in the embodiment of the present application, because the online tasks all use the p1 level, the online tasks request native resource fields, and the scheduler schedules resources by invoking a native static resource map of the cluster; the offline task has the characteristics of tolerance to errors and restarting, so that the offline task can use a p2 or p3 level, when a user of the cluster submits a task of a p2 level, the requested fields are p2-cpu-allocate and p2-mem-allocate, the request passes through an API-Server and reaches a scheduler, the scheduler judges that the request is a dynamic resource request and calls a dynamic resource map for scheduling, when the user of the cluster submits the task of a p3 level, the requested fields are p3-cpu-allocate and p3-mem-allocate, the request passes through the API-Server and reaches the scheduler, and the scheduler judges that the request is a dynamic resource request and calls the dynamic resource map for scheduling.
Further, the above method for allocating resources based on dynamic resource views further includes: and allocating proper nodes for the offline tasks according to the existing resources, scheduling the tasks to the nodes, and starting the tasks.
Specifically, in the embodiment of the present application, the scheduler allocates a suitable node to the offline task according to the existing resource, and schedules the task to the node.
Further, the above method for allocating resources based on dynamic resource views further includes: further comprising: the dynamic adjustment of p2 level and p3 level resources is performed through the Cgroup mechanism.
Specifically, in the embodiment of the present application, the resources for adjusting the tasks at the p2 and p3 levels are dynamically adjusted by modifying the underlying Cgroup working mode.
The following are exemplary: tasks at the level of P1, whose quotas are controlled by the native cluster, are based on static resources, such as when node A also has 30 cores cpu, the user applies for P1 task a, requesting 29 cores, that task a can be successfully scheduled to node A; the highest quota of the task at the level of P2 is also controlled by the cluster static resource, but the request amount of the task is allocated according to the dynamic resource view, for example, if the node A still has 5-core static CPUs and 8-core dynamic CPUs, the quota of the task a at the level of P2 is 7, that a can be successfully scheduled to the node A, the actual usage amount of the task a is 7-core CPUs, and if the utilization rate of the online task CPU of the node rises, only 5 dynamic CPUs remain, and the actual maximum request of the task a is reduced to 5-core CPUs; tasks at the level of P3, whose quotas and requests are only affected by dynamic resources, can be run at the level of P3 as long as there are remaining dynamic resources in a node, for example, node a has 20 cores, task a at the level of P1 occupies 15 cores, and task b at the level of P2 has a request amount of 5 but a maximum quota of 10, so when the cluster still has remaining dynamic resources at the level of P3, the task request of task c at the level of P3 can be scheduled to node a, but if task a and task b are fully loaded, the resources of c will be recovered and the task will be killed.
Fig. 2 is a schematic view of a resource allocation apparatus based on a dynamic resource view according to an embodiment of the present disclosure.
In a second aspect, an embodiment of the present application further provides a resource allocation apparatus based on a dynamic resource view, which is applied to an off-line hybrid deployment task, and with reference to fig. 2, includes:
the definition module 201: for defining the level of the task.
Specifically, in the embodiment of the present application, the definition module 201 defines the levels of the tasks to be classified into a p1 level, a p2 level and a p3 level according to the priorities of the tasks.
The custom module 202: and the method is used for customizing the dynamic actually available resources of the cluster.
Specifically, in this embodiment of the present application, the customization module 202 performs hierarchical customization on the dynamic actual available resources by using a resource extension manner, and registers the dynamic actual available resources to the API-Server of the cluster, so that the cluster can sense the change of the dynamic actual available resources of the cluster.
The calculation module 203: the method is used for calculating the real-time resource utilization rate of each service to predict the utilization rate of the host, and calculating the dynamic actual available resources of the cluster according to the utilization rate of the host.
Specifically, in this embodiment of the present application, dynamic-resource-plugin is deployed at each node in the cluster, the plugin calculates the utilization rate of the predicted host of the real-time resource utilization rate of each service, and the calculating module 203 calculates the number of resources that can be dynamically allocated according to the utilization rate of the host to calculate the dynamic actually available resources of the cluster.
The determination module 204: and the dynamic resource view is used for determining the dynamic resource view of the cluster according to the dynamic actual available resources of the cluster.
Specifically, in the embodiment of the present application, after the number of dynamically allocatable resources is calculated, the plug-in of each node periodically sends the resources to the controller, and the controller collects the resources and arranges the resources into the dynamic resource view of the whole cluster.
The assignment module 205: and the system is used for allocating resources according to the dynamic resource view and the level of the task.
Specifically, in the embodiment of the present application, the allocation module 203 of the scheduler in the control center allocates resources according to the dynamic resource view and the level of the task, thereby implementing a mixed part of offline and online tasks, and effectively improving the utilization rate of the resources.
In a third aspect, an embodiment of the present application further provides an electronic device, including: a processor and a memory;
the processor is used for executing the resource allocation method based on the dynamic resource view by calling the program or the instruction stored in the memory.
In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores a program or instructions, and the program or instructions cause a computer to perform the above method for resource allocation based on a dynamic resource view.
Fig. 3 is a schematic block diagram of an electronic device provided by an embodiment of the present disclosure.
As shown in fig. 3, the electronic apparatus includes: at least one processor 301, at least one memory 302, and at least one communication interface 303. The various components in the electronic device are coupled together by a bus system 304. A communication interface 303 for information transmission with an external device. It will be appreciated that the bus system 304 is used to enable communications among the components. The bus system 304 includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, the various buses are labeled as bus system 304 in fig. 3.
It will be appreciated that the memory 302 in this embodiment can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory.
In some embodiments, memory 302 stores the following elements, executable units or data structures, or a subset thereof, or an expanded set thereof: an operating system and an application program.
The operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, and the like, and is used for implementing various basic services and processing hardware-based tasks. The application programs, including various application programs such as a Media Player (Media Player), a Browser (Browser), etc., are used to implement various application services. The program for implementing any method of the dynamic resource view-based resource allocation method provided by the embodiment of the present application may be included in an application program.
In this embodiment of the present application, the processor 301 is configured to execute the steps of the embodiments of the resource allocation method based on a dynamic resource view provided by the embodiment of the present application by calling a program or an instruction stored in the memory 302, which may be specifically a program or an instruction stored in an application program.
Defining the level of the task;
customizing the dynamic actually available resources of the cluster;
calculating the real-time resource utilization rate of each service to predict the utilization rate of the host, and calculating the dynamic actual available resources of the cluster according to the utilization rate of the host;
determining a dynamic resource view of the cluster according to the dynamic actual available resources of the cluster;
and allocating resources according to the dynamic resource view and the level of the task.
Any method of the resource allocation method based on the dynamic resource view provided by the embodiment of the present application may be applied to the processor 301, or implemented by the processor 301. The processor 301 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 301. The Processor 301 may be a general-purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, or discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The steps of any one of the methods for resource allocation based on the dynamic resource view provided by the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software units in the decoding processor. The software elements may be located in ram, flash, rom, prom, or eprom, registers, among other storage media that are well known in the art. The storage medium is located in the memory 302, and the processor 301 reads the information in the memory 302 and completes the steps of a resource allocation method based on dynamic resource view in combination with the hardware.
Those skilled in the art will appreciate that although some embodiments described herein include some features included in other embodiments instead of others, combinations of features of different embodiments are meant to be within the scope of the application and form different embodiments.
Those skilled in the art will appreciate that the description of each embodiment has a respective emphasis, and reference may be made to the related description of other embodiments for those parts of an embodiment that are not described in detail.
While the invention has been described with reference to specific embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A resource allocation method based on a dynamic resource view is applied to an off-line hybrid deployment task, and comprises the following steps:
defining the level of the task;
customizing the dynamic actually available resources of the cluster;
calculating the real-time resource utilization rate of each service to predict the utilization rate of a host, and calculating the dynamic actual available resources of the cluster according to the utilization rate of the host;
determining a dynamic resource view of the cluster according to the dynamic actual available resources of the cluster;
and allocating resources according to the dynamic resource view and the level of the task.
2. The method according to claim 1, wherein the defining the level of the task comprises:
defining the grades of the tasks as three grades, namely p1, p2 and p3 according to the priorities of the tasks;
where the p1 rating is highest for online traffic; p2 is used for offline tasks, and p3 is the lowest level used for offline tasks.
3. The method of claim 1, wherein the self-defined cluster of dynamic and actual available resources comprises:
the dynamic actual available resources are graded and registered to the API-Server of the cluster; including the four p 2-cpu-allocates, p 2-mem-allocates, p 3-cpu-allocates and p 3-mem-allocates.
4. The method according to claim 1, wherein calculating the real-time resource utilization rate of each service predicts the utilization rate of the host, and calculates the cluster dynamic actual available resource according to the utilization rate of the host, comprises:
deploying a dynamic resource detection plug-in at each node in the cluster, wherein the plug-in is used for calculating the utilization rate of a real-time resource utilization rate prediction host of each service;
calculating the dynamic actual available resources of the cluster according to the utilization rate of the host according to the following formula:
p2-CPU-allocate is the node total CPU resource-p 1 level task CPU real-time utilization rate-safety threshold;
p3-CPU-allocate ═ node total CPU resources-task CPU real-time utilization at the p1 level-task real-time utilization at the p2 level-security threshold.
5. The method according to claim 1, wherein the allocating resources according to the dynamic resource view and the task level comprises:
if the level of the task is the level p1, calling a native static resource map of the cluster to allocate resources;
if the task level is a p2 level, judging whether the task request is a dynamic resource request, if so, calling a dynamic resource map for scheduling;
if the task level is the p3 level, judging whether the task request is a dynamic resource request, if so, calling a dynamic resource map for scheduling.
6. The method of claim 5, wherein the method further comprises: and allocating proper nodes for the offline tasks according to the existing resources, scheduling the tasks to the nodes, and starting the tasks.
7. The method of claim 5, wherein the method further comprises: the dynamic adjustment of p2 level and p3 level resources is performed through the Cgroup mechanism.
8. A resource allocation device based on a dynamic resource view is applied to an off-line hybrid deployment task, and comprises the following components:
a definition module: for defining a level of a task;
a self-defining module: the method is used for customizing the dynamic actually available resources of the cluster;
a calculation module: the system comprises a host, a server and a cluster, wherein the host is used for calculating the real-time resource utilization rate of each service and predicting the utilization rate of the host, and the cluster dynamic actual available resources are calculated according to the utilization rate of the host;
a determination module: a dynamic resource view used for determining the cluster according to the dynamic actual available resource of the cluster;
a distribution module: and the resource is distributed according to the dynamic resource view and the level of the task.
9. An electronic device, comprising: a processor and a memory;
the processor is configured to execute a method for resource allocation based on dynamic resource view according to any one of claims 1 to 7 by calling a program or instructions stored in the memory.
10. A computer-readable storage medium storing a program or instructions for causing a computer to perform a method for dynamic resource view based resource allocation according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111504156.XA CN114090267A (en) | 2021-12-09 | 2021-12-09 | Resource allocation method, device, equipment and medium based on dynamic resource view |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111504156.XA CN114090267A (en) | 2021-12-09 | 2021-12-09 | Resource allocation method, device, equipment and medium based on dynamic resource view |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114090267A true CN114090267A (en) | 2022-02-25 |
Family
ID=80307460
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111504156.XA Pending CN114090267A (en) | 2021-12-09 | 2021-12-09 | Resource allocation method, device, equipment and medium based on dynamic resource view |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114090267A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102722413A (en) * | 2012-05-16 | 2012-10-10 | 上海兆民云计算科技有限公司 | Distributed resource scheduling method for desktop cloud cluster |
| CN112783607A (en) * | 2021-01-29 | 2021-05-11 | 上海哔哩哔哩科技有限公司 | Task deployment method and device in container cluster |
| CN112948113A (en) * | 2021-03-01 | 2021-06-11 | 上海微盟企业发展有限公司 | Cluster resource management scheduling method, device, equipment and readable storage medium |
| CN113377541A (en) * | 2021-06-22 | 2021-09-10 | 云知声智能科技股份有限公司 | Cluster resource allocation and recovery method and device, electronic equipment and storage medium |
-
2021
- 2021-12-09 CN CN202111504156.XA patent/CN114090267A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102722413A (en) * | 2012-05-16 | 2012-10-10 | 上海兆民云计算科技有限公司 | Distributed resource scheduling method for desktop cloud cluster |
| CN112783607A (en) * | 2021-01-29 | 2021-05-11 | 上海哔哩哔哩科技有限公司 | Task deployment method and device in container cluster |
| CN112948113A (en) * | 2021-03-01 | 2021-06-11 | 上海微盟企业发展有限公司 | Cluster resource management scheduling method, device, equipment and readable storage medium |
| CN113377541A (en) * | 2021-06-22 | 2021-09-10 | 云知声智能科技股份有限公司 | Cluster resource allocation and recovery method and device, electronic equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 百度开发者中心: "百度混部实践系列 | 如何提高 K8S 集群资源利用率？", pages 2 - 7, Retrieved from the Internet <URL:https://segmentfault.com/a/1190000040657778> * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111344688B (en) | Method and system for providing resources in cloud computing | |
| US8656404B2 (en) | Statistical packing of resource requirements in data centers | |
| CN113835865B (en) | Task deployment method and device, electronic equipment and storage medium | |
| CN104239150B (en) | A kind of method and device of hardware resource adjustment | |
| Lee et al. | Resource scheduling in dependable integrated modular avionics | |
| CN114625500B (en) | Topology-aware microservice application scheduling method and application in cloud environment | |
| CN114327843A (en) | Task scheduling method and device | |
| WO2016101996A1 (en) | Allocating cloud computing resources in a cloud computing environment | |
| Maiti et al. | Internet of Things applications placement to minimize latency in multi-tier fog computing framework | |
| CN112905342A (en) | Resource scheduling method, device, equipment and computer readable storage medium | |
| Ghattas et al. | Preemption threshold scheduling: Stack optimality, enhancements and analysis | |
| CN111796933A (en) | Resource scheduling method, device, storage medium and electronic equipment | |
| CN112948113A (en) | Cluster resource management scheduling method, device, equipment and readable storage medium | |
| Rolia et al. | Resource access management for a utility hosting enterprise applications | |
| CN107203256B (en) | Energy-saving distribution method and device under network function virtualization scene | |
| CN114629960A (en) | Resource scheduling method, device, system, device, medium, and program product | |
| CN111240824A (en) | CPU resource scheduling method and electronic equipment | |
| Kong et al. | Guaranteed scheduling for (m, k)-firm deadline-constrained real-time tasks on multiprocessors | |
| CN114090267A (en) | Resource allocation method, device, equipment and medium based on dynamic resource view | |
| JP6191361B2 (en) | Information processing system, information processing system control method, and control program | |
| CN113468442A (en) | Resource bit flow distribution method, computing device and computer storage medium | |
| CN117178257A (en) | Task scheduling architecture and method | |
| CN113626162A (en) | Data center task hybrid deployment method and system based on dynamic resource sharing | |
| Awan et al. | Memory bandwidth regulation for multiframe task sets | |
| Ali et al. | Virtual gang based scheduling of real-time tasks on multicore platforms |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |