CN111506414B - Resource scheduling method, device, equipment, system and readable storage medium - Google Patents

Resource scheduling method, device, equipment, system and readable storage medium Download PDF

Info

Publication number
CN111506414B
CN111506414B CN201910091106.XA CN201910091106A CN111506414B CN 111506414 B CN111506414 B CN 111506414B CN 201910091106 A CN201910091106 A CN 201910091106A CN 111506414 B CN111506414 B CN 111506414B
Authority
CN
China
Prior art keywords
resource
target
application object
information
online
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910091106.XA
Other languages
Chinese (zh)
Other versions
CN111506414A (en
Inventor
叶良
蒋明江
陈龙刚
黄城
杨国东
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Group Holding Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to CN201910091106.XA priority Critical patent/CN111506414B/en
Publication of CN111506414A publication Critical patent/CN111506414A/en
Application granted granted Critical
Publication of CN111506414B publication Critical patent/CN111506414B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061Partitioning or combining of resources
    • G06F9/5066Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs

Abstract

The invention discloses a resource scheduling method, a device, equipment, a system and a readable storage medium, wherein the method comprises the following steps: acquiring resource demand information of a target application object, wherein the target application object is one of an online class application, an offline class application or a platform class application; distributing target equipment for providing resources for the target application object in a plurality of candidate equipment supporting the provision of resources according to the resource demand information of the target application object, and generating corresponding resource distribution information; and sending the resource allocation information to the target equipment so as to trigger the target equipment to provide resources for the target application object according to the resource allocation information.

Description

Resource scheduling method, device, equipment, system and readable storage medium
Technical Field
The present invention relates to the field of resource scheduling technologies, and in particular, to a method, an apparatus, a device, a system, and a readable storage medium for resource scheduling.
Background
An online service cluster includes a plurality of devices (e.g., servers) that are joined together and that have data processing capabilities that collectively respond to or process various service requests. With the rapid development of the internet and computer technology, the number of scales of applications providing user-specific application service functions has exploded, and the processing efficiency can be improved by providing resources required by the applications by an online service cluster in response to resource requests of a large number of applications.
However, the average utilization rate of processor resources is generally low due to factors such as difficulty in predicting peak-to-valley characteristics of traffic and preventive disaster backup of the online server cluster.
Disclosure of Invention
It is an object of the present invention to provide a new solution for resource scheduling.
According to a first aspect of the present invention, there is provided a resource scheduling method, comprising:
acquiring resource demand information of a target application object, wherein the target application object is one of an online class application, an offline class application or a platform class application;
distributing target equipment for providing resources for the target application object in a plurality of candidate equipment supporting the provision of resources according to the resource demand information of the target application object, and generating corresponding resource distribution information;
and sending the resource allocation information to the target equipment so as to trigger the target equipment to provide resources for the target application object according to the resource allocation information.
Preferably, the resource demand information at least includes resource request information, resource estimation information and demand priority of the target application object; the demand priority comprises an online class priority, an offline class priority and a platform class priority;
The step of obtaining the resource requirement information of the target application object comprises the following steps:
when the target application object is the online class application or the offline class application, setting the resource request information and the demand priority according to the target application object, and acquiring the resource estimation information according to the resource request information;
when the target application object is the platform application, setting the resource estimation information and the demand priority according to the target application object, and acquiring the resource request information according to the resource estimation information.
Preferably, when the target application object is the online class application or the offline class application, the step of obtaining the resource estimation information according to the resource request information includes:
acquiring the resource estimation information according to the resource request information, the minimum guaranteed load and the predicted load of the target application object;
the predicted load is obtained according to the historical load information of the target application object.
Preferably, when the target application object is the platform class application, the step of obtaining the resource request information according to the resource estimation information includes:
And acquiring the resource request information of the target application object according to the resource estimation information of the target application object, the inherent resources of the equipment supporting the resource supply and the resource estimation information of other application objects which can be acquired.
Preferably, the step of allocating a target device for providing resources to the target application object among a plurality of candidate devices supporting the provision of resources includes:
acquiring a resource score of each alternative device according to the resource demand information of the target application object;
and allocating the target devices according to the descending sort order of the resource scores of each candidate device.
Preferably, the resource demand information at least includes resource request information, resource estimation information and demand priority of the target application object; the demand priority comprises an online class priority, an offline class priority and a platform class priority;
the step of obtaining the resource score of each candidate device according to the resource demand information of the target application object comprises the following steps:
when the demand priority is an online class priority, acquiring a resource score of the alternative equipment according to the resource request information of the target application object, the resource request information corresponding to other applications on the alternative equipment and the first resource state parameter of the alternative equipment;
The first resource state parameters of the alternative equipment at least comprise an equilibrium index, a resource scarcity index, a preemption cost index, an application diversity index and an application preference index of the alternative equipment;
when the demand priority is an offline class priority or a platform class priority, acquiring a resource score of the alternative equipment according to the resource request information of the target application object, the resource request information corresponding to other applications on the alternative equipment and the second resource state parameter of the alternative equipment;
the second resource state parameter of the alternative device at least comprises an equilibrium index and a preemption cost index of the alternative device.
Preferably, the resource allocation information at least includes a resource allocation indication of the target application object on each of the target devices and a quality of service configuration parameter;
the step of triggering the target device to provide resources for the target application object according to the resource allocation information comprises the following steps:
triggering the target equipment to acquire the service quality parameters of the target equipment according to the resource allocation indication and the service quality configuration parameters so as to set the target equipment to provide resources for the target application object;
The service quality parameter at least comprises one of a processor service quality parameter, a memory service quality parameter, a network service quality parameter, a read-write service quality parameter and a control service quality parameter.
Preferably, the processor service quality parameter at least comprises an indication of processing resource limitation of an online service group and an indication of processing resource limitation of an offline service group; the online service group comprises tasks related to the online class application, and the offline service group comprises tasks related to the offline class application and the platform class application;
the QoS configuration parameters at least comprise processing load control instructions, minimum load limit and processing configuration weights;
the step of obtaining the quality of service parameter of the target device includes:
indicating the processing resource limit of the online service group as unrestricted;
acquiring a processing resource limit instruction according to the processor resource of the target equipment, the processing load control instruction, the minimum load limit and the online processing load of the online service group of the target equipment;
the online processing load is obtained according to the resource prediction information of the application object, the application processing load and the processing configuration weight, which are included in the online service group.
Preferably, the storage service quality parameter at least comprises a storage resource limitation indication of an online service group and a storage resource limitation indication of an offline service group; the online service group comprises tasks related to the online class application, and the offline service group comprises tasks related to the offline class application and the platform class application;
the QoS configuration parameters at least comprise a storage load control instruction;
the step of obtaining the quality of service parameter of the target device includes:
indicating the storage resource limit of the online service group as unlimited;
and acquiring the storage resource limit indication according to the storage resource of the target equipment, the storage load control indication and the online storage load of the online service group of the target equipment.
Preferably, the network quality of service parameter includes at least a network bandwidth limitation indication and a transmission quality of service indication;
the step of obtaining the quality of service parameter of the target device includes:
distinguishing a control group, an online service group and an offline service group on the target equipment, and respectively setting the network service quality parameters;
wherein the control group includes tasks related to control; the online service group includes tasks related to the online class application, and the offline service group includes tasks related to the offline class application and the platform class application.
Preferably, the read-write service quality parameter at least comprises a read-write limitation indication related to storage;
the step of obtaining the quality of service parameter of the target device includes:
distinguishing an online service group and an offline service group on the target equipment, and respectively setting the read-write service quality parameters;
wherein the online service group comprises tasks related to the online class application, and the offline service group comprises tasks related to the offline class application and the platform class application.
Preferably, the control quality of service parameter includes at least a processor link control indication and a memory bandwidth ratio;
the step of obtaining the quality of service parameter of the target device includes:
distinguishing an online service group and an offline service group on the target equipment, and respectively setting the control service quality parameters;
wherein the online service group comprises tasks related to the online class application, and the offline service group comprises tasks related to the offline class application and the platform class application.
Preferably, the resource allocation indication at least includes resource request information of application objects allocated on the target device and a demand priority;
The quality of service parameters further include an object score for each of the application objects on the target device; the object score of the application object is used for selecting the application object of the preempted resource when the target device is triggered to implement the resource preemption;
the step of obtaining the quality of service parameter of the target device includes:
and obtaining the object score of the application object according to the resource request information, the application storage load, the operation time length, the demand priority and the physical storage resource of the target device of each application object on the target device.
According to a second aspect of the present invention, there is provided a resource scheduling apparatus, comprising:
the demand acquisition unit is used for acquiring resource demand information of a target application object, wherein the target application object is one of an online class application, an offline class application or a platform class application;
the resource allocation unit is used for allocating target equipment for providing resources for the target application object in a plurality of candidate equipment supporting the provision of resources according to the resource demand information of the target application object, and generating corresponding resource allocation information;
and the resource providing unit is used for sending the resource allocation information to the target equipment so as to trigger the target equipment to provide resources for the target application object according to the resource allocation information.
According to a third aspect of the present invention, there is also provided a resource scheduling apparatus, including:
a memory for storing executable instructions;
and the processor is used for running the resource scheduling device to execute the resource scheduling method according to the first aspect of the invention according to the control of the executable instructions.
According to a fourth aspect of the present invention, there is also provided a readable storage medium storing a computer program readable and executable by a computer for performing the resource scheduling method according to the first aspect of the present invention when executed by the computer.
According to a fifth aspect of the present invention, there is provided a resource scheduling system, comprising:
a plurality of devices for providing resources;
a resource scheduling device as described in the second aspect of the invention or a resource scheduling apparatus as described in the third aspect of the invention.
According to an embodiment of the present disclosure, a new technical solution for resource scheduling is provided, by acquiring resource requirement information of a target application object, and according to the resource requirement information of the target application object, allocating target devices for providing resources to the target application object among a plurality of candidate devices supporting provision of resources, generating corresponding resource allocation information, and sending the resource allocation information to the target devices, so as to trigger the target devices to provide resources to the target application object according to the resource allocation information.
Other features of the present invention and its advantages will become apparent from the following detailed description of exemplary embodiments of the invention, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description, serve to explain the principles of the invention.
Fig. 1 is a block diagram showing an example of a hardware configuration of a computer cluster that may be used to implement an embodiment of the invention.
FIG. 2 is a schematic flow chart of a resource scheduling method according to an embodiment of the invention;
FIG. 3 is a schematic flow chart of a resource scheduling method according to another embodiment of the invention;
FIG. 4 is a functional block diagram of a resource scheduling apparatus according to an embodiment of the present invention;
FIG. 5 is a hardware configuration block diagram of a resource scheduling device according to an embodiment of the present invention;
FIG. 6 is a block diagram of a resource scheduling system according to an embodiment of the present invention;
fig. 7 is a block diagram of a resource scheduling system according to another embodiment of the present invention.
Detailed Description
Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless it is specifically stated otherwise.
The following description of at least one exemplary embodiment is merely exemplary in nature and is in no way intended to limit the invention, its application, or uses.
Techniques, methods, and apparatus known to one of ordinary skill in the relevant art may not be discussed in detail, but are intended to be part of the specification where appropriate.
In all examples shown and discussed herein, any specific values should be construed as merely illustrative, and not a limitation. Thus, other examples of exemplary embodiments may have different values.
It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further discussion thereof is necessary in subsequent figures.
< hardware configuration >
Fig. 1 is a block diagram showing a hardware configuration of a server cluster in which the present embodiment can be implemented.
As shown in fig. 1, the server cluster of the present embodiment includes a plurality of servers 1000, and 4 servers 1000 are shown in fig. 1, which are server 1000A, server 1000B, server 1000C, and server 1000D, respectively.
In the present embodiment, the number of servers 1000 included in the server cluster may be determined according to the actual scenario, and is not limited in any way.
In this embodiment, the servers 1000 form a server cluster, and each server 1000 may be used as a device for providing resources, where a Master component is set in a small number of servers 1000 in the server cluster, and a Slave component is set in each server 1000 in the server cluster.
In this embodiment, the server 1000 with the Master component in any one of the server clusters may be used as a Master server, and other servers with Slave components in the server clusters are used as Slave servers, where the Master server and the Slave servers cooperate with each other to implement the resource scheduling method of the present invention, for example, the Master server obtains resource requirement information of a target application object, allocates target Slave servers for providing resources to the target application object in multiple Slave servers according to the resource requirement information of the target application object, generates corresponding resource allocation information, and sends the resource allocation information to the target Slave servers, where the target Slave servers provide resources to the target application object according to the resource allocation information.
The server 1000 provides the service points for processing, database, communication facilities. The server 1000 may be a unitary server or a distributed server across multiple computers or computer data centers. The server may be of various types such as, but not limited to, a web server, news server, mail server, message server, advertisement server, file server, application server, interaction server, database server, or proxy server. In some embodiments, each server may include hardware, software, or embedded logic components or a combination of two or more such components for performing the appropriate functions supported by or implemented by the server. For example, a server, such as a blade server, cloud server, etc., or may be a server group consisting of multiple servers, may include one or more of the types of servers described above, etc.
In one embodiment, the server 1000 may include a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, and an input device 1600, as shown in fig. 1.
In this embodiment, the server 1000 may further include a speaker, a microphone, etc., which are not limited herein.
The processor 1100 may be a dedicated server processor, or may be a desktop processor, a mobile processor, or the like that meets performance requirements, and is not limited herein. The memory 1200 includes, for example, ROM (read only memory), RAM (random access memory), nonvolatile memory such as a hard disk, and the like. The interface device 1300 includes, for example, various bus interfaces such as a serial bus interface (including a USB interface), a parallel bus interface, and the like. The communication device 1400 is capable of wired or wireless communication. The display device 1500 is, for example, a liquid crystal display, an LED display, a touch display, or the like. The input device 1600 may include, for example, a touch screen, keyboard, etc.
Although a plurality of devices of the server 1000 are shown in fig. 1, the present invention may relate to only some of the devices, for example, the server 1000 may relate to only the memory 1200, the communication device 1400, and the processor 1100.
The communication network 2000 may be a wireless communication network or a wired communication network, and may be a local area network or a wide area network. In the server cluster shown in fig. 1, a plurality of servers 1000 may communicate with each other via a network 2000. The network 2000 on which the plurality of servers 1000 communicate may be the same or different.
The server clusters shown in fig. 1 are only illustrative, and are not intended to limit the present invention, its application or use in any way, and in practical applications, other numbers of server clusters may be included, for example, 2 server clusters, 3 server clusters, 5 server clusters, or even more, and no limitation is made herein. In an embodiment of the present invention, the memory 1200 of the server 1000 is configured to store instructions for controlling the processor 1100 to operate to perform any one of the resource scheduling methods provided in the embodiment of the present invention. The skilled person can design instructions according to the disclosed solution. How the instructions control the processor to operate is well known in the art and will not be described in detail here.
< first embodiment >
< method >
Fig. 2 is a schematic flow diagram of a resource scheduling method according to one embodiment.
Referring to fig. 2, the resource scheduling method of the present embodiment may be implemented by a resource scheduling device or a resource scheduling apparatus, and the resource scheduling device or the resource scheduling apparatus may be specifically distributed on a device providing a resource, and the resource scheduling method of the present embodiment may include steps S2100 to S2300 as follows:
in step S2100, resource requirement information of the target application object is acquired.
The target application object may be an application object requiring resource scheduling, and in general, the target application object may be divided into an online class application, an offline class application, and a platform class application according to different service types, that is, in this embodiment, the target application object may be one of the online class application, the offline class application, or the platform class application.
The resource requirement information of the target application object at least can represent requirement information of the target application object on required resources, the resource requirement information can be an elastic resource group, and the elastic resource group is used for describing resource request information, resource estimation information and requirement priority required by the target application object, and correspondingly, the resource requirement information can at least comprise the resource request information, the resource estimation information and the requirement priority of the target application object.
The above resource request information may at least represent the type of the resource required by the target application object, and the above resource estimation information may at least represent the estimated value of the resource required by the target application object, in this embodiment, the required resource may be multiple for the target application object, where, in order to distinguish different types of resources, the type of the required resource, the specific number corresponding to each type of resource, etc. may be carried in the resource request information, the type of the required resource, the estimated value corresponding to each type of resource, etc. may also be carried in the resource estimation information, and of course, the resource request information and the resource estimation information may also include other information of the required resource by the target application object, which is not limited herein.
The demand priority is used for representing the priority among different target application objects, the demand priority comprises an online class priority, an offline class priority and a platform class priority, and when the target application objects are online class applications, the online class priority is corresponding to the target application objects; when the target application object is an offline class application, the offline class priority is corresponding to the target application object; when the target application object is a platform class application, the platform class priority corresponds to the target application object. The demand priority may be set according to the resource estimation information of the resource attribute platform of the target application object, the resource request information of the non-platform, and the type of the target application object, for example, the online class priority may be higher than the offline class priority, and the offline class priority may be higher than the platform class priority, which is not limited herein.
In one example, the acquiring the resource requirement information of the target application object in the step S2100 may include the following steps S2110 to S2120:
in step S2110, when the target application object is an online class application or an offline class application, the resource request information and the demand priority are set according to the target application object, and the resource estimation information is obtained according to the resource request information.
In this example, when the target application object is an online class application, the resource request information and the demand priority corresponding to the online class application may be directly set, and the resource estimation information may be calculated according to the resource request information.
In this example, when the target application object is an offline application, the resource request information and the demand priority corresponding to the offline application may be directly set, and the resource estimation information may be calculated according to the resource request information.
In this example, in the case where the target application object is an online class application or an offline class application, the step of obtaining the resource prediction information according to the resource request information may include:
and obtaining resource estimation information according to the resource request information, the minimum guaranteed load and the predicted load of the target application object.
In this example, the resource prediction information prediction of the target application object may be calculated according to the following formula:
predict=request*max{g_load,p_load} (1)
Wherein max { g_load, p_load } represents the maximum value between taking g_load, p_load; the request represents resource request information of a target application object; g_load represents the minimum guaranteed load of the target application object, and g_load can be a constant value set according to engineering experience or experimental simulation results; the p_load represents a predicted load of the target application object, the p_load may be obtained according to historical load information of the target application object, specifically, the p_load may be a value obtained by weighted averaging based on the historical load information according to a preset time unit, when the preset time unit is one hour, and the current hour is T1, the historical load information l_load may include load information load1 of the previous hour T2, load information load2 of the current hour T1, load information load3 of the next hour T3 of the previous day, and load information load4 of the next hour T4 of the previous day, and the historical load information p_load may be calculated according to the following formula:
Figure BDA0001963285490000111
wherein a is 1 、a 2 、a 3 、a 4 The weight values corresponding to load1, load2, load3 and load4 respectively can be set as a according to the specific application scenario 1 、a 2 、a 3 、a 4
In step S2120, when the target application object is a platform application, the resource estimation information and the demand priority are set according to the target application object, and the resource request information is obtained according to the resource estimation information.
In this example, when the target application object is a platform application, the resource estimation information and the demand priority of the platform application may be directly set, and the resource request information may be calculated according to the resource estimation information.
In this example, when the target application object is a platform class application, the step of obtaining the resource request information according to the resource estimation information may include:
and acquiring the resource request information of the target application object according to the resource estimation information of the target application object, the inherent resources of the equipment supporting the resource supply and the resource estimation information of other application objects which can be acquired.
The above other application objects may be one or more of other online class applications, other offline class applications, and other platform class applications including in addition to the target application object.
In this example, the resource request information request of the target application object may be calculated according to the following formula:
Figure BDA0001963285490000121
wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure BDA0001963285490000122
in n represents the number of online class applications already running on the current device, prod_project k The resource estimation information of the kth online class application which is already running on the current equipment is represented, namely, the resource estimation information of all the online class applications which are already running on the current equipment is calculated; / >
Figure BDA0001963285490000123
In m denotes the number of offline class applications already running on the current device, non_prod_project k The resource estimation information of the kth offline class application already running on the current equipment is represented, namely the resource estimation information of all the offline class applications already running on the current equipment is calculated; />
Figure BDA0001963285490000124
Where l represents the number of platform class applications already running on the current device, platform_project k The method comprises the steps that resource prediction information of a kth platform class application already running on current equipment is represented, namely, the resource prediction information of all the platform class applications already running on the current equipment and the resource prediction information prediction of a target application object to be allocated are calculated; host_resource represents an inherent resource of a device that supports providing a resource.
In this example, after each allocation of the target application object is finished, the resource request information request of the platform class application may be calculated again based on the above formula, so that the overall resource score is more accurate.
After the resource requirement information of the target application object is obtained through step S2100, a target device for providing resources to the target application object is allocated in a plurality of candidate devices supporting to provide resources according to the resource requirement information of the target application object in combination with the subsequent steps, and corresponding resource allocation information is generated, so that the resource allocation information is sent to the target device, so that the target device provides resources to the target application object according to the resource allocation information, and the resource utilization rate is improved.
After obtaining the resource requirement information of the target application object, entering:
step S2200, distributing target equipment for providing resources for the target application object in a plurality of candidate equipment supporting the provision of resources according to the resource demand information of the target application object, and generating corresponding resource distribution information.
The alternative device supporting provision of the resource is a device capable of providing the resource, and the alternative device may be all devices or part of devices in the computer cluster, which is not limited herein.
The target device is a device that ultimately allocates resources to the target application object, and the target device may be any one or more of alternative devices, which is not limited herein.
The resource allocation information includes at least an indication of the resource allocation of the target application object on each target device and a quality of service configuration parameter.
In one example, referring to fig. 3, the step of allocating a target device for providing resources to a target application object among a plurality of candidate devices supporting the provision of resources in step S2200 may include the steps of S2210 to S2220:
step S2210, obtaining the resource score of each candidate device according to the resource demand information of the target application object.
In an example, the resource score of each candidate device may be obtained according to the resource requirement information of the target application object, so as to allocate one or more target devices from high to low according to the actual required resources of the target application object.
In one example, the step of obtaining the resource score of each candidate device according to the resource requirement information of the target application object in the step S2210 may include steps S2211 to S2212:
in step S2211, when the demand priority is the online class priority, the resource score of the candidate device is obtained according to the resource request information of the target application object, the resource request information of other application objects on the candidate device, and the first resource status parameter of the candidate device.
The first resource status parameter of the above alternative device may at least include a balance index, a resource scarcity index, a preemption cost index, an application diversity index, and an application preference index of the alternative device.
In this example, the resource score for the candidate device may be calculated according to the following method:
Figure BDA0001963285490000131
wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure BDA0001963285490000132
in n, represents the number of online class applications already running on the current device, prod_request k The method comprises the steps that resource request information of a kth online class application which is already running on current equipment is represented, namely, resource request information of all the online class applications which are already running on the current equipment and resource request information of a target application object to be allocated are calculated; the balance indicates an equilibrium index of the candidate device, for example, the better the equilibrium of the CPU resource of the candidate device and/or the storage resource of the candidate device is, the higher the equilibrium index balance of the candidate device is; the scarcity indicates a resource scarcity index of the candidate device, for example, as long as a small number of candidate devices in the computer cluster have an image processor (GPU) or an FPGA, the higher the resource scarcity index of the small number of candidate devices is; the preemption_cost represents a preemption cost index of the alternative device, and specifically can be cost after the preempted resource is applied to the alternative device, for example, how long after the preempted resource is applied, the application can be restarted, etc.; priority_diversity represents an application diversity index of the candidate device For example, the more balanced the duty ratio among the applications with different priorities on the alternative device, the better the application diversity index priority_diversity of the alternative device; the reference represents an application preference index of the candidate device, and may be, for example, an association between an application already running on the candidate device and an application to be run, where the association may be, for example, a complementary application, a call to each other, etc.; />
Figure BDA0001963285490000141
The function is used to calculate a weighted average of all input parameters as a resource score for the candidate device, which may be calculated specifically according to the following formula:
Figure BDA0001963285490000142
wherein b in the above formula 1 、b 2 、b 3 、b 4 、b 5 、b 6 Respectively correspond to
Figure BDA0001963285490000143
The weight value of balance, scarcity, preemption _ cost, priority _ diversity, preference can be set b according to the specific application scene 1 、b 2 、b 3 、b 4 、b 5 、b 6
In step S2212, when the demand priority is the offline class priority or the platform class priority, the resource score of the candidate device is obtained according to the resource request information of the target application object, the resource request information corresponding to other applications on the candidate device, and the second resource status parameter of the candidate device.
The second resource status parameter of the alternative device at least comprises an equilibrium index of the alternative device and a preemption cost index.
In this example, the resource score for the candidate device may be calculated according to the following:
Figure BDA0001963285490000144
wherein, the liquid crystal display device comprises a liquid crystal display device,
Figure BDA0001963285490000145
m represents the number of offline class applications or the number of platform class applications which are already running on the current device, namely the resource request information comprising the offline class applications and the platform class applications. The balance indicates an equilibrium index of the candidate device, for example, the better the equilibrium of the CPU resource of the candidate device and/or the storage resource of the candidate device is, the higher the equilibrium index balance of the candidate device is; the preemption_cost represents a preemption cost index of the alternative device, and specifically can be cost after the preempted resource is applied to the alternative device, for example, how long after the preempted resource is applied, the application can be restarted, etc.; />
Figure BDA0001963285490000151
The function is used to calculate a weighted average of all input parameters as a resource score for the candidate device, which may be calculated specifically according to the following formula:
Figure BDA0001963285490000152
wherein c 1 、c 2 、c 3 Respectively correspond to
Figure BDA0001963285490000153
The weight value of balance, preemption _cost can be set to c according to specific application scene 1 、c 2 、c 3
Step S2220 assigns the target device according to the descending sort order of the resource scores of each candidate device.
After the resource score of each candidate device is obtained according to the above step S2210, the resource score of each candidate device may be sorted in a descending order according to the step S2220 to obtain a descending order, and one or more candidate devices are sequentially selected from high to low as the target device according to the type of the resource required by the target application object, so as to send the resource allocation information to the target device, so that the target device provides the resource for the target application object according to the resource allocation information, and the resource utilization rate is improved.
In this embodiment, according to the above step S2200, according to the resource requirement information of the target application object, the target device for providing the resource to the target application object is allocated among the multiple candidate devices supporting the provision of the resource, and the corresponding resource allocation information is generated, and then the resource allocation information is sent to the target device in combination with the subsequent step, so that the target device provides the resource to the target application object according to the resource allocation information, and the resource utilization rate is improved.
After generating the corresponding resource allocation information, entering:
step S2300, the resource allocation information is sent to the target device, so as to trigger the target device to provide the resource for the target application object according to the resource allocation information.
The resource allocation information includes at least an indication of the resource allocation of the target application object on each target device and a quality of service configuration parameter.
In this embodiment, the resource allocation information may be sent to the target device to control the target device to provide the resource to the target application object according to the resource allocation information, or the target device may provide the resource to the target application object according to the resource allocation information by itself after receiving the resource allocation information.
In one example, the triggering the target device to provide the resource to the target application object according to the resource allocation information in step S2300 may include:
and triggering the target equipment to acquire the service quality parameter of the target equipment according to the resource allocation indication and the service quality configuration parameter so as to set the target equipment to provide resources for the target application object.
The quality of service parameter includes at least one of a processor quality of service parameter, a memory quality of service parameter, a network quality of service parameter, a read-write quality of service parameter, and a control quality of service parameter.
In this example, after obtaining the qos parameters of the target device, the target device may set the qos parameters of the target device according to Cgroup (Control Groups), where the Cgroup is a mechanism provided by the Linux kernel and capable of limiting, recording and isolating the physical resources used by the process group, which provides a unified framework for resource management, and in this example, the hierarchy corresponding to the Cgroup may be set to provide resources to the target application object.
In this example, a calculation period may be set, and the quality of service parameter of the target device is calculated every one calculation period, and the current resource status information of the target device is collected to update the resource allocation status.
In one example, the processor quality of service parameters include at least an indication of a processing resource limitation of the online service group and an indication of a processing resource limitation of the offline service group.
The above online service group includes tasks related to online class applications, and the above offline service group includes tasks related to offline class applications and platform class applications.
The qos configuration parameters include at least a processing load control indication, a minimum load limit, and a processing configuration weight.
In this example, the processor qos parameters may further include cpu. Shares, which may be a fixed value set according to a specific application scenario or specific application requirement, for example, the cpu. Shares may be 1024, and for example, the cpu. Shares may also be 2, which is not limited herein.
In this example, the acquisition of the qos parameters of the target device may include the following steps S2311 to S2312:
in step S2311, the processing resource limitation of the online service group is indicated as unlimited.
In step S2312, a processing resource limitation instruction is obtained according to the processor resource of the target device, the processing load control instruction, the minimum load limitation, and the online processing load of the online service group of the target device.
In this step S2312, the processing resource limitation indication may be calculated according to the following formula:
Figure BDA0001963285490000171
wherein non_prod_cpu_limit represents a processing resource limitation indication of the offline service group; host_cpu represents the processor resources of the target device; cpu_ctrl_load represents a processing load control instruction; m_limit represents a minimum load limit; the prod_cpu_load represents an online processing load of the online service group of the target device, and may be specifically obtained according to resource prediction information of an application object included in the online service group, an application processing load and a processing configuration weight, for example, a weighted average of the resource prediction information of the application object included in the online service group, the application processing load fed back by Cgoup and the processing configuration weight may be calculated to obtain the prod_cpu_load of each application;
Figure BDA0001963285490000172
in n, represents the number of online class applications already running on the current device, prodcpu load k Representing the online processing load of the kth online class application on the current device, namely calculating the online processing load of all the online class applications already running on the current device.
In one example, the storage quality of service parameters include at least an indication of storage resource limitations of the online service group and an indication of storage resource limitations of the offline service group.
The above online service group includes tasks related to online class applications, and the offline service group includes tasks related to offline class applications and platform class applications.
The qos configuration parameter includes at least a storage load control indication.
In this example, obtaining the qos parameters of the target device may further include the following steps S2321 to S2322:
step S2321 indicates that the storage resource limitation of the online service group is not limited.
Step S2322, a storage resource limitation instruction is obtained according to the storage resource of the target device, the storage load control instruction and the online storage load of the online service group of the target device.
In this step S2322, the storage resource limitation indication may be calculated according to the following formula:
Figure BDA0001963285490000181
wherein non_prod_mem_limit represents a storage resource limitation indication of the offline service group; host_mem represents the memory resource of the target device; mem_ctrl_load represents a storage load control indication, which is used for avoiding unstable traffic caused by too high load of the target device; prod_mem_load represents an online storage load of an online service group of the target device;
Figure BDA0001963285490000182
In n represents the number of online class applications already running on the current device, prodmem load k Representing the online storage load of the kth online class application on the current device, namely calculating the online storage load of all the online class applications already running on the current device.
In one example, the network quality of service parameters include at least a network bandwidth limitation indication and a transmission quality of service indication.
In this example, acquiring the quality of service parameter of the target device may further include the following step S2330:
step S2330, the control group, the online service group and the offline service group on the target device are distinguished, and the network service quality parameters are set respectively.
The control group includes tasks related to control.
The online service group includes tasks related to online class applications, and the offline service group includes tasks related to offline class applications and platform class applications.
In this example, the control group, the online service group and the offline service group on the target device may be distinguished, and the network bandwidth limitation indication and the transmission service quality indication of the control group, the network bandwidth limitation indication and the transmission service quality indication of the online service group, and the network bandwidth limitation indication and the transmission service quality indication of the offline service group are respectively set.
The above network bandwidth limitation indication may include at least one of a maximum constraint of a network bandwidth or a minimum constraint of a network bandwidth, where the maximum constraint of the network bandwidth and the minimum constraint of the network bandwidth select a value with a better effect according to engineering experience or experimental simulation results in a specific application scenario.
The above transmission quality of service indication may include labeling gold, silver, copper on the network link for differentiating transmission quality of service for the control group, the online service group, and the offline service group. For example, "gold" may be marked on the network link of the control group as a transmission service quality indicator, "silver" may be marked on the network link of the online service group as a transmission service quality indicator, and "copper" may be marked on the network link of the offline service group as a transmission service quality indicator, so long as the transmission service quality of the control group, the online service group, and the offline service group can be distinguished, which is not limited herein.
In one example, the read-write quality of service parameter includes at least a read-write limitation indication associated with the storage.
In this example, acquiring the quality of service parameter of the target device may further include the following step S2340:
Step S2340, the online service group and the offline service group on the target device are distinguished, and the read-write service quality parameters are set respectively.
The online service group includes tasks related to online class applications, and the offline service group includes tasks related to offline class applications and platform class applications.
In this example, the online service group and the offline service group on the target device may be distinguished, and respective corresponding storage-related read-write restriction indications are set, where the storage-related read-write restriction indications include at least one of a read-write bit rate (bps) and an IOPS (Input/Output Operations Per Second) restriction, and the read-write bit rate and the IOPS restriction may be set according to a storage medium of a specific target device, and are not limited herein.
In one example, the control quality of service parameters include at least a processor link control indication and a memory bandwidth ratio.
The processor link control indication is used to establish a connection between the target device and the target application object.
The memory bandwidth ratio is the ratio of the amount of information accessed by the memory per unit time.
In this example, acquiring the quality of service parameter of the target device may include the following step S2350:
In step S2350, the online service group and the offline service group on the target device are distinguished, and the quality of service control parameters are set respectively.
The online service group includes tasks related to online class applications, and the offline service group includes tasks related to offline class applications and platform class applications.
In this example, the online service group and the offline service group on the target device may be distinguished, and the respective corresponding processor link control instruction (CPU LLC) and memory bandwidth limitation ratio may be set. The processor link control indication and the memory bandwidth limit ratio may be set according to model parameters of a specific target device, and are not limited herein.
In one example, the resource allocation indication includes at least resource request information and a demand priority of an application object allocated on the target device.
The quality of service parameter also includes an object score for each application object on the target device for selecting an application object for the preempted resource when the target device is triggered to conduct the resource preemption.
In this example, acquiring the quality of service parameter of the target device may further include the following step S2360:
in step S2360, the object score of the application object is obtained according to the resource request information, the application storage load, the running time, the demand priority and the physical storage resource of the target device of each application object on the target device.
In this example, the object score for an application object may be calculated according to the following:
memory_badness=g(mem_load,mem_limit,host_mem,run_time,priority)(10)
wherein memory_badness represents an object score of the application object; mem_limit represents resource request information of an application object; the mem_load represents an application storage load of the application object, and may be obtained by dividing an actual load of the application object by resource request information of the application object; host_mem represents the physical storage resource of the target device, run_time represents the running time of the application object, and priority represents the demand priority of the application object; the g (mem_load, mem_limit, host_mem, run_time, priority) function is used to calculate a weighted average of all input parameters as an object score of an application object, specifically, the object score of the application object may be calculated according to the following formula:
Figure BDA0001963285490000201
/>
wherein d in the above formula 1 、d 2 、d 3 、d 4 、d 5 The weight values corresponding to mem_load, mem_limit, host_mem, run_ time, priority, respectively, may be set d according to the specific application scenario 1 、d 2 、d 3 、d 4 、d 5
In an example, in the case that the application object preempts the resource, the application object that needs to leave the resource is selected according to the sorting order of the object scores of the application object, so that the selected application object releases the resource for other application objects to preempt.
< resource scheduling device >
In this embodiment, as shown in fig. 4, the resource scheduling apparatus 4000 further includes a requirement obtaining unit 4100, a resource allocation unit 4200, and a resource providing unit 4300, which are used to implement the resource scheduling method provided in this embodiment, and are not described herein.
The requirement obtaining unit 4100 is configured to obtain resource requirement information of a target application object, where the target application object is one of an online class application, an offline class application, and a platform class application.
A resource allocation unit 4200, configured to allocate, according to the resource requirement information of the target application object, a target device for providing resources to the target application object among a plurality of candidate devices supporting providing resources, and generate corresponding resource allocation information.
The resource providing unit 4300 is configured to send the resource allocation information to the target device, so as to trigger the target device to provide resources to the target application object according to the resource allocation information.
In one embodiment, the resource requirement information at least includes resource request information, resource estimation information and requirement priority of the target application object; the demand priorities include an online class priority, an offline class priority, and a platform class priority.
The requirement obtaining unit 4100 is further configured to set the resource request information and the requirement priority according to the target application object when the target application object is the online class application or the offline class application, and obtain the resource estimation information according to the resource request information; and when the target application object is the platform class application, setting the resource estimation information and the demand priority according to the target application object, and acquiring the resource request information according to the resource estimation information.
In one embodiment, the requirement obtaining unit 4100 is further configured to obtain the resource estimation information according to the resource request information and the minimum guaranteed load and the predicted load of the target application object.
The predicted load is obtained according to the historical load information of the target application object.
In one embodiment, the requirement obtaining unit 4100 is further configured to obtain the resource request information of the target application object according to the resource estimation information of the target application object, the inherent resources of the device supporting providing the resource, and the available resource estimation information of other application objects.
In one embodiment, the resource allocation unit 4200 is further configured to obtain a resource score of each candidate device according to the resource requirement information of the target application object; and allocating the target devices according to the descending sort order of the resource scores of each candidate device.
In one embodiment, the resource requirement information at least includes resource request information, resource estimation information and requirement priority of the target application object; the demand priorities include an online class priority, an offline class priority, and a platform class priority.
The resource allocation unit 4200 is further configured to obtain, when the demand priority is an online class priority, a resource score of the candidate device according to the resource request information of the target application object, the resource request information corresponding to other applications on the candidate device, and the first resource status parameter of the candidate device.
The first resource status parameter of the alternative device at least comprises an equilibrium index, a resource scarcity index, a preemption cost index, an application diversity index and an application preference index of the alternative device.
The resource allocation unit 4200 is further configured to obtain, when the demand priority is an offline class priority or a platform class priority, a resource score of the candidate device according to the resource request information of the target application object, the resource request information corresponding to other applications on the candidate device, and a second resource status parameter of the candidate device.
The second resource status parameter of the alternative device at least comprises a balance index and a preemption cost index of the alternative device.
In one embodiment, the resource allocation information includes at least an indication of the resource allocation of the target application object on each of the target devices and a quality of service configuration parameter.
The resource providing unit 4300 is further configured to trigger the target device to obtain a quality of service parameter of the target device according to the resource allocation indication and the quality of service configuration parameter, so as to set the target device to provide resources to the target application object.
The service quality parameter at least comprises one of a processor service quality parameter, a memory service quality parameter, a network service quality parameter, a read-write service quality parameter and a control service quality parameter.
In one embodiment, the processor quality of service parameter includes at least an indication of a processing resource limitation of an online service group and an indication of a processing resource limitation of an offline service group; the online service group includes tasks related to the online class application, and the offline service group includes tasks related to the offline class application and the platform class application.
The qos configuration parameters at least include a processing load control indication, a minimum load limit, and a processing configuration weight.
A resource providing unit 4300 further configured to indicate the processing resource limit of the online service group as unlimited; and obtaining the processing resource limit instruction according to the processor resource of the target device, the processing load control instruction, the minimum load limit and the online processing load of the online service group of the target device.
The online processing load is obtained according to the resource prediction information of the application object, the application processing load and the processing configuration weight, which are included in the online service group.
In one embodiment, the storage quality of service parameter includes at least an indication of storage resource limitation of an online service group and an indication of storage resource limitation of an offline service group; the online service group includes tasks related to the online class application, and the offline service group includes tasks related to the offline class application and the platform class application.
The quality of service configuration parameter at least comprises a storage load control indication.
A resource providing unit 4300 further configured to indicate the storage resource limit of the online service group as unlimited; and acquiring the storage resource limit indication according to the storage resource of the target equipment, the storage load control indication and the online storage load of the online service group of the target equipment.
In one embodiment, the network quality of service parameter includes at least a network bandwidth limitation indication and a transmission quality of service indication.
The resource providing unit 4300 is further configured to distinguish a control group, an online service group, and an offline service group on the target device, and set the network quality of service parameters respectively.
The control group includes tasks related to control; the online service group includes tasks related to the online class application, and the offline service group includes tasks related to the offline class application and the platform class application.
In one embodiment, the read-write quality of service parameter includes at least a read-write limitation indication associated with the storage.
The resource providing unit 4300 is further configured to distinguish an online service group and an offline service group on the target device, and set the read-write quality of service parameters respectively.
The online service group includes tasks related to the online class application, and the offline service group includes tasks related to the offline class application and the platform class application.
In one embodiment, the control quality of service parameters include at least a processor link control indication and a memory bandwidth ratio.
The resource providing unit 4300 is further configured to distinguish an online service group and an offline service group on the target device, and set the control quality of service parameters respectively.
The online service group includes tasks related to the online class application, and the offline service group includes tasks related to the offline class application and the platform class application.
In one embodiment, the resource allocation indication includes at least resource request information of application objects allocated on the target device and a demand priority.
The quality of service parameters further include an object score for each of the application objects on the target device; the object score of the application object is used for selecting the application object of the preempted resource when the target device is triggered to implement the resource preemption.
The resource providing unit 4300 is further configured to obtain an object score of the application object according to the resource request information, the application storage load, the running duration, the demand priority, and the physical storage resource of the target device of each application object on the target device.
< resource scheduling device >
In this embodiment, there is also provided a resource scheduling device 5000, as shown in fig. 5, including:
A memory 5100 for storing executable instructions;
the processor 5200 is configured to execute the resource scheduling device to perform the resource scheduling method as provided in the present embodiment according to the control of the executable instruction.
In this embodiment, the resource scheduling device 5000 may be a server. For example, the resource scheduling device 5000 may be a server 1000 as shown in fig. 1.
The resource scheduling device 5000 may further include other devices, for example, a server 1000 as shown in fig. 1, and may further include an input device, a communication device, an interface device, a display device, and the like.
< readable storage Medium >
In the present embodiment, there is also provided a readable storage medium storing a computer program readable and executable by a computer for executing the resource scheduling method described in the present embodiment when executed by the computer.
In this embodiment, the readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: portable computer disks, hard disks, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static Random Access Memory (SRAM), portable compact disk read-only memory (CD-ROM), digital Versatile Disks (DVD), memory sticks, floppy disks, mechanical coding devices, punch cards or in-groove structures such as punch cards or grooves having instructions stored thereon, and any suitable combination of the foregoing. A readable storage medium as used herein is not to be construed as a transitory signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a pulse of light through a fiber optic cable), or an electrical signal transmitted through an electrical wire.
< second embodiment >
In this embodiment, there is also provided a resource scheduling system 6000, as shown in fig. 6, including:
a plurality of devices 6100 for providing resources;
the resource scheduling apparatus 4000 or the resource scheduling device 5000 provided in the first embodiment.
The above devices 6100 may be, for example, a device 6100A for providing a resource and a device 6100B for providing a resource, where the multiple devices 6100 for providing a resource may be devices within one server cluster, or may be devices within different server clusters, respectively, where the multiple devices for providing a resource 6100 are each provided with a Slave component, and at least one of the multiple devices 6100 for providing a resource is provided with a Master component.
In this embodiment, the number of the devices 6100 for providing the resource may be determined according to the actual scene, and is not limited in any way.
The resource scheduling apparatus 4000 or the resource scheduling device 5000 may be distributed on the device 6100 that provides the resource.
< example >
The resource scheduling method implemented by the resource scheduling system 6000 provided in the present embodiment will be further exemplified below.
In this example, the resource scheduling system 6000 includes a plurality of devices 6100 for providing resources, which devices 6100 may be servers having Master (Master) components and Slave (Slave) components, and resource scheduling means 4000, which may be distributed over a plurality of such devices 6100. For example, as shown in fig. 7, a device 6100 including a Master component may be selected as a Master device, and a plurality of devices 6100 including Slave components may be selected as Slave devices, where functions corresponding to a demand acquisition unit in the resource scheduling apparatus 4000 are implemented by a resource estimation module and a quality of service configuration (Qos) module in the Master device, and functions corresponding to a resource allocation unit in the resource scheduling apparatus 4000 are implemented by a resource matching calculation module and a resource allocation overseal module in the Master device, and functions corresponding to a resource providing unit in the resource scheduling apparatus 4000 are implemented by a quality of service (Quality of Service, qos) scheduling module, a hierarchical control (Cgroup) module, and a Metric (Metric) acquisition module in the Slave device. Based on the master device and the slave device shown in fig. 7, the corresponding resource scheduling method includes: steps S201 to S208.
In step S201, the resource estimation module obtains resource requirement information of a target application object on the application control device.
The resource demand information may be an elastic resource group, where the resource demand information at least includes resource request information, resource estimation information, and a demand priority; the demand priorities include an online class priority, an offline class priority, and a platform class priority.
The target application object is one of an online class application, an offline class application, or a platform class application.
In this step S201, the application control device may describe the target application object through the elastic resource group using the service AM (Application Master).
In step S202, the resource estimation module determines whether the target application object is an online class application or an offline class application, if so, proceeds to step S203, otherwise, proceeds to step S204.
In step S203, the resource estimation module sets resource request information and demand priority according to the target application object, and obtains resource estimation information according to the resource request information, the minimum guaranteed load sent by the service quality configuration module, and the historical load information sent by the communication module a.
The historical load information sent by the communication module a above is used to determine the predicted load.
In step S204, the resource estimation module sets resource estimation information and demand priority according to the target application object, and obtains resource request information according to the resource estimation information.
In step S205, the resource matching calculation module obtains the resource score of each slave device according to the resource requirement information of the target application object.
In step S206, the resource matching super-selling module allocates the target slave devices according to the descending order of the resource scores of each slave device, and generates a resource allocation indication and a quality of service configuration parameter corresponding to each target slave device.
In step S207, the communication module a communicates with the communication module B, and sends the resource allocation indication and the qos configuration parameter corresponding to each target slave device to the qos scheduling module of the target slave device.
Step S208, the service quality scheduling module obtains the service quality parameters according to the resource allocation indication and the service quality configuration parameters.
In step S209, the hierarchical control module sets the quality of service parameter by setting the corresponding hierarchical Cgroup parameter, and provides the resource to the target application object.
In step S210, the measurement collection module collects new index information and transmits the new index information back to the master device through the communication module B.
The present invention may be a system, method, and/or computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions embodied thereon for causing a processor to implement aspects of the present invention.
The computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: portable computer disks, hard disks, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static Random Access Memory (SRAM), portable compact disk read-only memory (CD-ROM), digital Versatile Disks (DVD), memory sticks, floppy disks, mechanical coding devices, punch cards or in-groove structures such as punch cards or grooves having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media, as used herein, are not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses through fiber optic cables), or electrical signals transmitted through wires.
The computer readable program instructions described herein may be downloaded from a computer readable storage medium to a respective computing/processing device or to an external computer or external storage device over a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. The network interface card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in the respective computing/processing device.
Computer program instructions for carrying out operations of the present invention may be assembly instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, c++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present invention are implemented by personalizing electronic circuitry, such as programmable logic circuitry, field Programmable Gate Arrays (FPGAs), or Programmable Logic Arrays (PLAs), with state information for computer readable program instructions, which can execute the computer readable program instructions.
Various aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, implementation by software, and implementation by a combination of software and hardware are all equivalent.
The foregoing description of embodiments of the invention has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the technical improvements in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims (16)

1. A method of resource scheduling, comprising:
acquiring resource demand information of a target application object, wherein the target application object is one of an online class application, an offline class application or a platform class application; the resource demand information at least comprises resource request information, resource estimation information and demand priority of the target application object; the demand priority comprises an online class priority, an offline class priority and a platform class priority;
distributing target equipment for providing resources for the target application object in a plurality of candidate equipment supporting the provision of resources according to the resource demand information of the target application object, and generating corresponding resource distribution information;
The resource allocation information is sent to the target equipment so as to trigger the target equipment to provide resources for the target application object according to the resource allocation information;
the step of obtaining the resource requirement information of the target application object includes:
when the target application object is the online class application or the offline class application, setting the resource request information and the demand priority according to the target application object, and acquiring the resource estimation information according to the resource request information;
when the target application object is the platform application, setting the resource estimation information and the demand priority according to the target application object, and acquiring the resource request information according to the resource estimation information.
2. The method of claim 1, wherein when the target application object is the online class application or the offline class application, the step of obtaining the resource forecast information from the resource request information comprises:
acquiring the resource estimation information according to the resource request information, the minimum guaranteed load and the predicted load of the target application object;
The predicted load is obtained according to the historical load information of the target application object.
3. The method of claim 1, wherein when the target application object is the platform class application, the step of obtaining the resource request information according to the resource forecast information comprises:
and acquiring the resource request information of the target application object according to the resource estimation information of the target application object, the inherent resources of the equipment supporting the resource supply and the resource estimation information of other application objects which can be acquired.
4. The method of claim 1, wherein the step of allocating, among the plurality of candidate devices supporting the provision of resources, a target device for providing resources to the target application object comprises:
acquiring a resource score of each alternative device according to the resource demand information of the target application object;
and allocating the target devices according to the descending sort order of the resource scores of each candidate device.
5. The method of claim 4, wherein,
the resource demand information at least comprises resource request information, resource estimation information and demand priority of the target application object; the demand priority comprises an online class priority, an offline class priority and a platform class priority;
The step of obtaining the resource score of each candidate device according to the resource demand information of the target application object comprises the following steps:
when the demand priority is an online class priority, acquiring a resource score of the alternative equipment according to the resource request information of the target application object, the resource request information corresponding to other applications on the alternative equipment and the first resource state parameter of the alternative equipment;
the first resource state parameters of the alternative equipment at least comprise an equilibrium index, a resource scarcity index, a preemption cost index, an application diversity index and an application preference index of the alternative equipment;
when the demand priority is an offline class priority or a platform class priority, acquiring a resource score of the alternative equipment according to the resource request information of the target application object, the resource request information corresponding to other applications on the alternative equipment and the second resource state parameter of the alternative equipment;
the second resource state parameter of the alternative device at least comprises an equilibrium index and a preemption cost index of the alternative device.
6. The method of claim 1, wherein,
The resource allocation information at least comprises a resource allocation indication and a service quality configuration parameter of the target application object on each target device;
the step of triggering the target device to provide resources for the target application object according to the resource allocation information comprises the following steps:
triggering the target equipment to acquire the service quality parameters of the target equipment according to the resource allocation indication and the service quality configuration parameters so as to set the target equipment to provide resources for the target application object;
the service quality parameter at least comprises one of a processor service quality parameter, a memory service quality parameter, a network service quality parameter, a read-write service quality parameter and a control service quality parameter.
7. The method of claim 6, wherein,
the processor service quality parameter at least comprises a processing resource limit indication of an online service group and a processing resource limit indication of an offline service group; the online service group comprises tasks related to the online class application, and the offline service group comprises tasks related to the offline class application and the platform class application;
the QoS configuration parameters at least comprise processing load control instructions, minimum load limit and processing configuration weights;
The step of obtaining the quality of service parameter of the target device includes:
indicating the processing resource limit of the online service group as unrestricted;
acquiring a processing resource limit instruction according to the processor resource of the target equipment, the processing load control instruction, the minimum load limit and the online processing load of the online service group of the target equipment;
the online processing load is obtained according to the resource prediction information of the application object, the application processing load and the processing configuration weight, which are included in the online service group.
8. The method of claim 6, wherein,
the storage service quality parameter at least comprises a storage resource limit indication of an online service group and a storage resource limit indication of an offline service group; the online service group comprises tasks related to the online class application, and the offline service group comprises tasks related to the offline class application and the platform class application;
the QoS configuration parameters at least comprise a storage load control instruction;
the step of obtaining the quality of service parameter of the target device includes:
indicating the storage resource limit of the online service group as unlimited;
And acquiring the storage resource limit indication according to the storage resource of the target equipment, the storage load control indication and the online storage load of the online service group of the target equipment.
9. The method of claim 6, wherein,
the network service quality parameter at least comprises a network bandwidth limit indication and a transmission service quality indication;
the step of obtaining the quality of service parameter of the target device includes:
distinguishing a control group, an online service group and an offline service group on the target equipment, and respectively setting the network service quality parameters;
wherein the control group includes tasks related to control; the online service group includes tasks related to the online class application, and the offline service group includes tasks related to the offline class application and the platform class application.
10. The method of claim 6, wherein,
the read-write service quality parameter at least comprises a read-write limit indication related to storage;
the step of obtaining the quality of service parameter of the target device includes:
distinguishing an online service group and an offline service group on the target equipment, and respectively setting the read-write service quality parameters;
Wherein the online service group comprises tasks related to the online class application, and the offline service group comprises tasks related to the offline class application and the platform class application.
11. The method of claim 6, wherein,
the control quality of service parameter includes at least a processor link control indication and a memory bandwidth ratio;
the step of obtaining the quality of service parameter of the target device includes:
distinguishing an online service group and an offline service group on the target equipment, and respectively setting the control service quality parameters;
wherein the online service group comprises tasks related to the online class application, and the offline service group comprises tasks related to the offline class application and the platform class application.
12. The method of claim 6, wherein,
the resource allocation indication at least comprises resource request information of application objects allocated on the target equipment and a demand priority;
the quality of service parameters further include an object score for each of the application objects on the target device; the object score of the application object is used for selecting the application object of the preempted resource when the target device is triggered to implement the resource preemption;
The step of obtaining the quality of service parameter of the target device includes:
and obtaining the object score of the application object according to the resource request information, the application storage load, the operation time length, the demand priority and the physical storage resource of the target device of each application object on the target device.
13. A resource scheduling apparatus, comprising:
the demand acquisition unit is used for acquiring resource demand information of a target application object, wherein the target application object is one of an online class application, an offline class application or a platform class application; the resource demand information at least comprises resource request information, resource estimation information and demand priority of the target application object; the demand priority comprises an online class priority, an offline class priority and a platform class priority;
the resource allocation unit is used for allocating target equipment for providing resources for the target application object in a plurality of candidate equipment supporting the provision of resources according to the resource demand information of the target application object, and generating corresponding resource allocation information;
the resource providing unit is used for sending the resource allocation information to the target equipment so as to trigger the target equipment to provide resources for the target application object according to the resource allocation information;
The step of obtaining the resource requirement information of the target application object includes:
when the target application object is the online class application or the offline class application, setting the resource request information and the demand priority according to the target application object, and acquiring the resource estimation information according to the resource request information;
when the target application object is the platform application, setting the resource estimation information and the demand priority according to the target application object, and acquiring the resource request information according to the resource estimation information.
14. A resource scheduling apparatus, comprising:
a memory for storing executable instructions;
a processor for executing the resource scheduling device to perform the resource scheduling method according to claims 1-12, according to the control of the executable instructions.
15. A readable storage medium, wherein the readable storage medium stores a computer program readable and executable by a computer for performing the resource scheduling method of claims 1-12 when executed by the computer.
16. A resource scheduling system, comprising:
A plurality of devices for providing resources;
the resource scheduling apparatus of claim 13 or the resource scheduling device of claim 14.
CN201910091106.XA 2019-01-30 2019-01-30 Resource scheduling method, device, equipment, system and readable storage medium Active CN111506414B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910091106.XA CN111506414B (en) 2019-01-30 2019-01-30 Resource scheduling method, device, equipment, system and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910091106.XA CN111506414B (en) 2019-01-30 2019-01-30 Resource scheduling method, device, equipment, system and readable storage medium

Publications (2)

Publication Number Publication Date
CN111506414A CN111506414A (en) 2020-08-07
CN111506414B true CN111506414B (en) 2023-04-25

Family

ID=71863794

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910091106.XA Active CN111506414B (en) 2019-01-30 2019-01-30 Resource scheduling method, device, equipment, system and readable storage medium

Country Status (1)

Country Link
CN (1) CN111506414B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113018871A (en) * 2021-04-19 2021-06-25 腾讯科技(深圳)有限公司 Service processing method, device and storage medium
CN113703962B (en) * 2021-07-22 2023-08-22 北京华胜天成科技股份有限公司 Cloud resource allocation method and device, electronic equipment and storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102958166A (en) * 2011-08-29 2013-03-06 华为技术有限公司 Resource allocation method and resource management platform
CN103577268A (en) * 2012-08-07 2014-02-12 复旦大学 Self-adaptive resource supply method based on application load
CN107968810A (en) * 2016-10-20 2018-04-27 阿里巴巴集团控股有限公司 A kind of resource regulating method of server cluster, device and system

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2443229B (en) * 2006-08-23 2009-10-14 Cramer Systems Ltd Capacity management for data networks

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102958166A (en) * 2011-08-29 2013-03-06 华为技术有限公司 Resource allocation method and resource management platform
CN103577268A (en) * 2012-08-07 2014-02-12 复旦大学 Self-adaptive resource supply method based on application load
CN107968810A (en) * 2016-10-20 2018-04-27 阿里巴巴集团控股有限公司 A kind of resource regulating method of server cluster, device and system

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Parminder Singh等.A study on resource provisioning of multi-tier web applications in cloud computing.2015 2nd International Conference on Computing for Sustainable Global Development (INDIACom).2015,全文. *
杨忙忙.Spark数据处理平台中资源动态分配技术研究.中国优秀硕士学位论文全文数据库 信息科技辑.2017,(第03期),全文. *
邓晓衡 ; 关培源 ; 万志文 ; 刘恩陆 ; 罗杰 ; 赵智慧 ; 刘亚军 ; 张洪刚 ; .基于综合信任的边缘计算资源协同研究.计算机研究与发展.2018,第55卷(第03期),全文. *

Also Published As

Publication number Publication date
CN111506414A (en) 2020-08-07

Similar Documents

Publication Publication Date Title
WO2021036936A1 (en) Method and apparatus for allocating resources and tasks in distributed system, and system
US10389800B2 (en) Minimizing execution time of a compute workload based on adaptive complexity estimation
US10691647B2 (en) Distributed file system metering and hardware resource usage
US11088961B2 (en) Monitoring data streams and scaling computing resources based on the data streams
US9213573B2 (en) Management of prioritizing virtual machines in an operating environment
US10678589B2 (en) Leveraging directed acyclic graph (DAG) information to group tasks for execution
WO2020258290A1 (en) Log data collection method, log data collection apparatus, storage medium and log data collection system
US10462067B2 (en) Proactive channel agent
US20210029204A1 (en) Methods and apparatus to generate migration recommendations to migrate services between geographic regions
CN111694646A (en) Resource scheduling method and device, electronic equipment and computer readable storage medium
CN112148468B (en) Resource scheduling method and device, electronic equipment and storage medium
CN110399213A (en) Determine method, apparatus, electronic equipment and the medium of the resource requirement of application program
CN110782122B (en) Data processing method and device and electronic equipment
US20180046476A1 (en) Workload placement in a hybrid cloud environment
CN111506414B (en) Resource scheduling method, device, equipment, system and readable storage medium
CN111143039A (en) Virtual machine scheduling method and device and computer storage medium
US10031781B2 (en) Estimating job start times on workload management systems
US20230222004A1 (en) Data locality for big data on kubernetes
US10979531B2 (en) Pessimistic scheduling for topology optimized workload placement
US10254725B2 (en) Utilizing automated lighting system to determine occupancy
US10594620B1 (en) Bit vector analysis for resource placement in a distributed system
WO2022078060A1 (en) Tag-driven scheduling of computing resources for function execution
US11556387B2 (en) Scheduling jobs
JP2022067642A (en) Computer system, computer program and method for identifying and prioritizing re-factoring to improve micro-service identification (method and system for identifying and prioritizing re-factoring to improve micro-service identification)
US11487750B2 (en) Dynamically optimizing flows in a distributed transaction processing environment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant