WO2017166803A1 - Procédé et dispositif de planification de ressources - Google Patents

Procédé et dispositif de planification de ressources Download PDF

Info

Publication number
WO2017166803A1
WO2017166803A1 PCT/CN2016/103591 CN2016103591W WO2017166803A1 WO 2017166803 A1 WO2017166803 A1 WO 2017166803A1 CN 2016103591 W CN2016103591 W CN 2016103591W WO 2017166803 A1 WO2017166803 A1 WO 2017166803A1
Authority
WO
WIPO (PCT)
Prior art keywords
resource
historical
application
scheduling
applications
Prior art date
Application number
PCT/CN2016/103591
Other languages
English (en)
Chinese (zh)
Inventor
赵波
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2017166803A1 publication Critical patent/WO2017166803A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/503—Resource availability

Definitions

  • the present invention relates to the field of resource management technologies, and in particular, to a resource scheduling method and apparatus.
  • the cluster resource unified management and scheduling system is proposed as a cluster resource sharing platform, the system includes a cluster, the cluster includes a resource manager and a plurality of nodes, the cluster resource is a resource of multiple nodes, and the resource manager is deployed in the cluster. On the node, the system needs to support multiple different computing frameworks at the same time, as shown in Figure 1.
  • the system supports MR (MapReduce), Storm, S4 (Simple Scalable Streaming System), and MPI (Message Passing Interface).
  • MR MapReduce
  • Storm Storm
  • S4 Simple Scalable Streaming System
  • MPI Message Passing Interface
  • the computing framework the resource manager performs unified scheduling on applications of different computing framework types sent by the client device, so as to improve resource utilization and reduce operation and maintenance costs.
  • the resource scheduling strategies frequently used by the cluster resource unified management and scheduling system include a fair scheduling policy, a capacity scheduling policy, and a first-in first-out scheduling policy.
  • the fair scheduling policy is a memory resource used by an application submitted by a client device, and a central resource.
  • the ratio of the ratio of resources such as a central processing unit (CPU) resource or a disk I/O resource is used for resource scheduling;
  • the capacity scheduling policy divides resources by application, and each application can Set the minimum resource guarantee and the upper limit of use, and schedule resources according to the set minimum resource guarantee and use limit;
  • the first-in-first-out scheduling policy is scheduled according to the order of the applications submitted by the client device.
  • the unified management and scheduling system of the cluster resources cannot be used for each application according to the actual operation of the application.
  • the program allocates resources, which makes the resource allocation unreasonable and the resource utilization needs to be improved.
  • Embodiments of the present invention provide a resource scheduling method and apparatus for improving utilization of cluster resources.
  • a resource scheduling method is provided, which is applied to a resource manager, where a resource manager is used to manage and schedule cluster resources, the cluster includes multiple nodes, and R applications that are adapted to the computing framework supported by the cluster run in multiple On the node, R is a positive integer, the method includes: the resource manager receives the resource request of the R applications; the resource manager obtains the historical feature information, where the historical feature information includes at least one of the first historical feature information and the second historical feature information.
  • the first historical feature information is a historical feature of N applications in the R applications
  • the historical features of the application include information indicating a history resource usage of the application and a representation of the application.
  • the second historical feature information is a historical feature of M nodes having idle resources among the plurality of nodes, and the historical characteristics of the node include at least one of a failure rate of the node and a task failure rate One, 1 ⁇ N ⁇ R, M ⁇ 1, N, M are integers; the resource manager according to the first historical feature information and the first tune The degree policy schedules resources for the N applications; or the resource manager schedules resources for at least one of the R applications according to the second historical feature information and the second scheduling policy.
  • the method provided by the embodiment of the present invention acquires historical features of a node and/or an application after receiving a resource request sent by multiple applications, and allocates resources to the application according to historical features of the node and/or the application, because
  • the historical characteristics of nodes can indirectly characterize nodes
  • Pre- or future features the historical features of the application can indirectly characterize the current or future runtime characteristics of the application.
  • the amount of resources requested in the resource request of the application is not the application. Allocating resources, but allocating resources to the application through the historical characteristics of the nodes and/or applications, can prevent the resource manager from wasting resources due to the resources allocated by the application according to the amount of resources requested by the application, making the resource scheduling more reasonable. .
  • the method further includes: the resource manager adjusting the first scheduling parameter according to the first historical feature information, where the first scheduling parameter includes each of the N applications Corresponding scheduling parameters, the first scheduling parameter is used to determine an order of scheduling resources for the N applications; the resource manager schedules resources for the N applications according to the first historical feature information and the first scheduling policy, including: the resource manager according to The adjusted first scheduling parameter and the first scheduling policy schedule resources for N applications.
  • the method further includes: the resource manager adjusting the first scheduling parameter according to the first historical feature information and the second historical feature information, where the first scheduling parameter includes N applications a scheduling parameter corresponding to each application in the medium and a scheduling parameter corresponding to each of the M nodes, the first scheduling parameter is used to determine an order of scheduling resources for the N applications and a sequence of scheduling resources of the M nodes;
  • the resource manager schedules resources for the N applications according to the first historical feature information and the first scheduling policy, including: the resource manager scheduling resources for the N applications according to the adjusted first scheduling parameter and the first scheduling policy.
  • the method further includes: the resource manager adjusting the second scheduling parameter according to the second historical feature information, where the second scheduling parameter includes a corresponding one of each of the M nodes a scheduling parameter, where the second scheduling parameter is used to determine an order of scheduling resources of the M nodes; the resource manager scheduling resources for at least one of the R applications according to the second historical feature information and the second scheduling policy, including: resources The manager applies at least one of the R applications according to the adjusted second scheduling parameter and the second scheduling policy. Order resources.
  • the historical features of the N applications include a comprehensive historical running time of each of the N applications; M nodes
  • the historical feature includes a failure rate and a task failure rate of each of the M nodes, and the resource manager adjusts the first scheduling parameter according to the first historical feature information and the second historical feature information, including: the resource manager determines the N applications.
  • the node with the higher health rate of the node is allocated to the node with the longer comprehensive running time, which can improve the length.
  • the method further includes: the resource manager calculating the task success rate of all the nodes in the multiple nodes multiple times, and each time After the calculation is completed, the value of ⁇ is adjusted according to the task success rate until the determined value of ⁇ is such that the task success rate is the highest.
  • the optimal value of ⁇ can be determined, so that the task success rate of the system is the highest.
  • the historical features of the N applications include comprehensive historical resource utilization and comprehensive history of each of the N applications.
  • the resource usage amount and the comprehensive historical resource application quantity the resource manager adjusts the first scheduling parameter according to the first historical feature information, including: when the comprehensive historical resource utilization of the application is less than a preset threshold, the resource manager adopts a preset algorithm Initially calculating the comprehensive historical resource usage of the application to obtain a first threshold corresponding to the application, first The threshold is greater than the comprehensive historical resource usage of the application; the resource manager determines the application's comprehensive historical resource application amount as the second threshold corresponding to the application, and the first threshold is less than the second threshold; the resource manager is adjusted according to the threshold
  • the first scheduling parameter and the first scheduling policy are used to schedule resources for the application, including: the resource manager scheduling resources according to the first threshold, and real-time monitoring whether the amount of resources actually used by the application exceeds a first threshold.
  • the amount of resources actually used by the application is calculated by using a preset algorithm to obtain a first threshold corresponding to the application, and the difference between the first threshold obtained according to the current calculation and the first threshold obtained by the previous calculation is The application adds resources until the resource actually used by the application is at a second threshold.
  • the first threshold and the second threshold are determined according to the comprehensive historical resource usage of the application, and the resource is first scheduled according to the first threshold, and then the application is allocated according to the amount of resources actually used by the application. The amount of resources, thereby avoiding waste of resources and improving resource utilization.
  • the historical feature of an application further includes the application At least one of a history priority of the program and a history tag of the application
  • the historical feature of the node further includes at least one of a historical resource occupancy of the node and a historical resource usage of the node.
  • the method further includes: when the resource management When the historical feature information is not successfully obtained, the resource manager schedules resources for the R applications according to a preset scheduling policy.
  • a resource scheduling apparatus for managing and scheduling cluster resources, where the cluster includes multiple nodes, and R applications adapted to the computing framework supported by the cluster run on multiple nodes, and R is a positive integer.
  • the device includes: a receiving unit, configured to receive resource requests of R applications; an acquiring unit, configured to acquire historical feature information, where the historical feature information includes the first calendar At least one of history feature information and second history feature information, the first historical feature information is a historical feature of N applications in the R applications, and the historical feature of the application includes a historical resource used to represent the application At least one of information of usage status and information for indicating historical running time of the application; the second historical feature information is a historical feature of M nodes having idle resources among the plurality of nodes, and historical characteristics of the node include At least one of a failure rate of the node and a failure rate of the node, 1 ⁇ N ⁇ R, M ⁇ 1, N, M are integers, and a scheduling unit is configured to use the first historical feature information and the first scheduling policy
  • the apparatus further includes: an adjusting unit, configured to adjust a first scheduling parameter according to the first historical feature information, where the first scheduling parameter includes each of the N applications The scheduling parameter corresponding to the application, the first scheduling parameter is used to determine the order of scheduling resources for the N applications; the scheduling unit is specifically configured to: schedule the N applications according to the adjusted first scheduling parameter and the first scheduling policy Resources.
  • the apparatus further includes: an adjusting unit, configured to adjust the first scheduling parameter according to the first historical feature information and the second historical feature information, where the first scheduling parameter includes a scheduling parameter corresponding to each of the N applications and a scheduling parameter corresponding to each of the M nodes, the first scheduling parameter is used to determine the order of scheduling resources for the N applications and scheduling the M nodes The order of the resources; the scheduling unit is specifically configured to: schedule resources for the N applications according to the adjusted first scheduling parameter and the first scheduling policy.
  • the apparatus further includes: an adjusting unit, configured to adjust a second scheduling parameter according to the second historical feature information, where the second scheduling parameter includes each of the M nodes Corresponding scheduling parameters, the second scheduling parameter is used to determine scheduling M The order of the resources of the node; the scheduling unit is specifically configured to: schedule resources for at least one of the R applications according to the adjusted second scheduling parameter and the second scheduling policy.
  • the historical features of the N applications include a comprehensive historical running time of each of the N applications; M nodes
  • the historical feature includes a failure rate and a task failure rate of each of the M nodes;
  • the adjustment unit is specifically configured to: determine a comprehensive historical running time of each of the N applications; according to the M nodes
  • the scheduling unit is specifically configured to: allocate resources of a node with a higher health rate of the node to an application whose synthesis history is longer.
  • the node with the higher health rate of the node is allocated to the node with the longer comprehensive running time, which can improve the length.
  • the apparatus further includes: a calculating unit, configured to calculate a task success rate of all nodes of the multiple nodes multiple times, and After each calculation is completed, the value of ⁇ is adjusted according to the task success rate until the determined value of ⁇ is such that the task success rate is the highest.
  • the optimal value of ⁇ can be determined, so that the task success rate of the system is the highest.
  • historical features of the N applications include comprehensive historical resource utilization and comprehensive history of each of the N applications Resource usage and comprehensive historical resource application amount; the adjustment unit is specifically configured to: when the comprehensive historical resource utilization of the application is less than a preset threshold, the initial calculation of the comprehensive historical resource usage of the application by using a preset algorithm is obtained.
  • the first threshold corresponding to the application is greater than the comprehensive historical resource usage of the application; the application for the comprehensive historical resource of the application is determined as the second threshold corresponding to the application, The threshold is smaller than the second threshold; the scheduling unit is configured to: schedule the resource according to the first threshold, and monitor in real time whether the resource actually used by the application exceeds the first threshold, and if yes, use a preset algorithm to The amount of resources actually used by the application is calculated again to obtain a first threshold corresponding to the application, and the difference between the first threshold obtained in the current calculation and the first threshold obtained in the previous calculation is used to add resources to the application until the The resource actually used by the application is the second threshold.
  • the first threshold and the second threshold are determined according to the comprehensive historical resource usage of the application, and the resource is first scheduled according to the first threshold, and then the application is allocated according to the amount of resources actually used by the application. The amount of resources, thereby avoiding waste of resources and improving resource utilization.
  • the historical feature of an application further includes the application At least one of a history priority of the program and a history tag of the application
  • the historical feature of the node further includes at least one of a historical resource occupancy of the node and a historical resource usage of the node.
  • the scheduling unit is further configured to: When the acquiring unit fails to acquire the historical feature information, the resource is scheduled for the R applications according to a preset scheduling policy.
  • a resource scheduling apparatus for managing and scheduling cluster resources, where the cluster includes multiple nodes, and R applications adapted to the computing framework supported by the cluster run on multiple nodes, where R is a positive integer.
  • the device comprises: a receiver, a memory and a processor; a receiver for receiving resource requests of R applications; a memory for storing a set of codes, the processor performing the following actions according to the set of codes: acquiring historical feature information, history The feature information includes at least one of the first historical feature information and the second historical feature information, wherein the first historical feature information is a historical feature of the N applications in the R applications, and the historical features of the application include And at least one of information indicating a history resource usage of the application and information indicating a historical running time of the application; the second historical feature information is M nodes of the plurality of nodes having idle resources Historical feature, the historical feature of the node includes at least one of a failure rate of the node and a failure rate of the task, 1 ⁇ N ⁇ R, M ⁇ 1, N, M
  • the processor is further configured to: adjust the first scheduling parameter according to the first historical feature information, where the first scheduling parameter includes a corresponding one of each of the N applications a scheduling parameter, where the first scheduling parameter is used to determine an order of scheduling resources for the N applications; the processor is specifically configured to: schedule resources for the N applications according to the adjusted first scheduling parameter and the first scheduling policy.
  • the processor is further configured to: adjust the first scheduling parameter according to the first historical feature information and the second historical feature information, where the first scheduling parameter includes N applications a scheduling parameter corresponding to each application in the medium and a scheduling parameter corresponding to each of the M nodes, the first scheduling parameter is used to determine an order of scheduling resources for the N applications and a sequence of scheduling resources of the M nodes;
  • the processor is specifically configured to: schedule resources for the N applications according to the adjusted first scheduling parameter and the first scheduling policy.
  • the processor is further configured to: adjust the second scheduling parameter according to the second historical feature information, where the second scheduling parameter includes a scheduling parameter corresponding to each of the M nodes
  • the second scheduling parameter is used to determine a sequence of scheduling resources of the M nodes; the processor is specifically configured to: schedule resources for at least one of the R applications according to the adjusted second scheduling parameter and the second scheduling policy.
  • the historical characteristics of the N applications include the comprehensive historical running time of each of the N applications; the historical features of the M nodes include the failure rate and the task failure rate of each of the M nodes;
  • the device is specifically configured to: determine a comprehensive historical running time of each of the N applications; calculate a health rate of each of the M nodes according to a failure rate and a task failure rate of each of the M nodes
  • the node with the higher health rate of the node is allocated to the node with the longer comprehensive running time, which can improve the length.
  • the processing tool is further configured to: calculate a task success rate of all nodes in the multiple nodes multiple times, and complete each calculation The value of ⁇ is then adjusted according to the task success rate until the determined value of ⁇ is such that the task success rate is highest.
  • the optimal value of ⁇ can be determined, so that the task success rate of the system is the highest.
  • the historical features of the N applications include comprehensive historical resource utilization and comprehensive history of each of the N applications.
  • the resource usage amount and the comprehensive historical resource application amount; the processor is specifically configured to: when the comprehensive historical resource utilization of the application is less than a preset threshold, use a preset algorithm to calculate the comprehensive historical resource usage of the application for the first time.
  • the first threshold corresponding to the application is greater than the comprehensive historical resource usage of the application; the comprehensive historical resource application amount of the application is determined as a second threshold corresponding to the application, and the first threshold is less than the second threshold
  • the processor is specifically configured to: schedule the resource according to the first threshold, and monitor in real time whether the amount of resources actually used by the application exceeds a first threshold, and if so, use a preset algorithm to actually use the resource for the application. Calculate the application corresponding again The first threshold value, and the difference between the first threshold value obtained by the current calculation and the previous calculated first threshold value is added to the application until the resource actually used by the application is the second threshold.
  • the first threshold and the second threshold are determined according to the comprehensive historical resource usage of the application, and the resource is first scheduled according to the first threshold, and then the application is allocated according to the amount of resources actually used by the application. The amount of resources, thereby avoiding waste of resources and improving resource utilization.
  • the historical feature of an application further includes the application At least one of a history priority of the program and a history tag of the application
  • the historical feature of the node further includes at least one of a historical resource occupancy of the node and a historical resource usage of the node.
  • the processor is further configured to: When the processor does not successfully acquire the historical feature information, the resource manager schedules resources for the R applications according to a preset scheduling policy.
  • FIG. 1 is a schematic diagram of a composition of a cluster resource unified management and scheduling system in the prior art
  • FIG. 2 is a schematic diagram of a cluster resource unified management and scheduling system according to an embodiment of the present invention.
  • FIG. 3 is a flowchart of a resource scheduling method according to an embodiment of the present invention.
  • FIG. 4 is a flowchart of still another resource scheduling method according to an embodiment of the present invention.
  • FIG. 5 is a flowchart of still another resource scheduling method according to an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of a resource scheduling apparatus according to an embodiment of the present disclosure.
  • FIG. 7 is a schematic structural diagram of still another resource scheduling apparatus according to an embodiment of the present disclosure.
  • FIG. 8 is a schematic structural diagram of still another resource scheduling apparatus according to an embodiment of the present invention.
  • FIG. 2 it is a schematic diagram of a cluster resource unified management and scheduling system, including a cluster and a client device.
  • the cluster includes a resource manager and multiple nodes (three nodes in FIG. 2 are taken as an example), and resource management is performed. Deployed on a node in the cluster, each node can communicate with the resource manager, the client device submits the resource request of the application to the resource manager, and the resource manager can allocate the resource of the node to the application, Causes the application to run on that node based on the allocated node resources.
  • the node in the embodiment of the present invention has both a computing function and a storage function.
  • An embodiment of the present invention provides a resource scheduling method, which is applied to a resource manager, where the resource manager is configured to manage and schedule cluster resources, where the cluster includes multiple nodes, and R devices that are matched with the computing framework supported by the cluster.
  • the application runs on the plurality of nodes, and R is a positive integer.
  • the method includes:
  • the resource manager receives a resource request of the R applications.
  • the resource manager is deployed on any one of the plurality of nodes, and the cluster resource is the resource of the plurality of nodes, and the resource manager belongs to the cluster resource unified management and scheduling system (hereinafter referred to as the system), and the system may be Yarn or mesos. , coraca, Torca or Omega.
  • the cluster supports at least one computing framework, and an application adapted to any one of the computing frameworks can run on the node.
  • a resource request of multiple applications received by the resource manager may be a resource request of an application submitted by multiple client devices, and an application is a task belonging to a certain computing framework type, and the application runs once once.
  • the mission may be a resource request of an application submitted by multiple client devices, and an application is a task belonging to a certain computing framework type, and the application runs once once. The mission.
  • the resource request of the multiple applications received by the resource manager may be a resource request of all applications received in one heartbeat cycle, and the resource manager processes the resource request of the application received within one heartbeat cycle at a time.
  • the heartbeat period can be a preset value.
  • the resource manager acquires historical feature information, where the historical feature information includes at least one of first historical feature information and second historical feature information.
  • the first historical feature information is a historical feature of the N applications of the R applications; the second historical feature information is a historical feature of the M nodes of the plurality of nodes; M nodes are M nodes having idle resources among the plurality of nodes, and historical features of the application include information indicating a history resource usage of the application and a historical running time for indicating the application.
  • the historical characteristic of the node includes at least one of a failure rate of the node and a failure rate of the task, 1 ⁇ N ⁇ R, M ⁇ 1, and N and M are integers.
  • the application when the client device submits an application, the application may carry a unique identifier for the resource manager to distinguish different applications.
  • An application may only run once or multiple times. For applications that run multiple times, if the application has already run i (i ⁇ 1, i is an integer), then The historical features of the program can be obtained from the i-history running process and stored in the database.
  • application 1 ie, the application identified as 1
  • application 2 ie, identified as 2
  • the application is the first application running on November 10, 2015, and the application runs once a week
  • application 3 the application identified as 3) is first run on November 20, 2015.
  • application 4 (that is, the application identified as 4) is the application that was first run on December 1, 2015, and the application only runs once, if it is monthly
  • Table 1 the historical running times of the four applications before December 1, 2015 are as shown in Table 1, wherein the historical characteristics of the application 1, the application 2 or the application 3 can be based on at least the application
  • a historical running process acquisition eg, the historical characteristics of application 1 can be obtained according to the 2 historical running processes of the application
  • application 4 has no historical features.
  • the multiple applications received by the resource manager may include an application that runs only once and/or an application that runs multiple times.
  • the resource manager receives the When a resource request of multiple applications is requested, the historical feature of the application may be queried in the database according to the identifier of the application.
  • the resource manager may Query only in the database Historical characteristics of some applications in multiple applications.
  • N applications may be in N′ applications. Part or all of the application; in another application scenario, the resource manager can query the historical features of all the applications in the multiple applications in the database, then the N applications can be all of the N′ applications.
  • the application, N' is an integer greater than or equal to N.
  • the resource manager can The historical features of Application 1, Application 2, and Application 3 are queried in the database.
  • the M nodes may be nodes that have idle resources in all of the plurality of nodes, or may be partial nodes among nodes of the plurality of nodes that have idle resources.
  • the historical running time information of the application may include a historical running time of the one or more historical running times of the application, a historical starting time and a historical stop time, a comprehensive historical running time of the application, etc.;
  • the information of the historical resource usage may include the historical resource request quantity, the historical resource usage amount, and the historical resource utilization rate of the application one or more historical running times, the comprehensive historical resource request amount of the application, and the comprehensive historical resource usage amount. And comprehensive historical resource utilization.
  • a comprehensive historical feature of an application refers to a historical feature that is comprehensively determined according to the historical feature of one or more historical operations of the application, for example, “a comprehensive historical operation of an application”
  • “Time” refers to the historical running time determined by the historical running time of one or more historical running times of the application.
  • the integrated historical running time of the application may be an average of multiple historical running times. Can be the highest value of multiple historical run times.
  • the number of times involved in the above calculation is the number of times in the preset time period, and the preset time period may be half year or one year, which is not limited by the embodiment of the present invention.
  • running an application on a node runs a task. If the operation is successful, the task succeeds. If the operation fails, the task fails.
  • the historical feature of an application further includes at least one of a history priority of the application and a history tag of the application, and a historical feature of the node further includes a historical resource occupancy of the node.
  • the historical priority of the application includes a priority of one or more historical runtimes of the application; the history tag of the application includes a label carried by the application during one or more historical runs, and the label may be
  • the indication information included in the application for example, information indicating a resource using a certain node, the label may also be a time parameter indicating that the application requires completion of the operation before the time indicated by the time parameter, such as time parameter 21: 10, which means that the application requires to run before 21:10.
  • the historical feature of the node may also include the historical resource usage of the node and the historical resource occupancy of the node.
  • the historical resource usage of the node may be the resource usage of the test at multiple test time points in a period of time. The average value or the highest value, the historical resource occupancy rate of the node can be calculated according to the historical resource usage of the node and the total resource amount of the node.
  • the above resources include resources such as CPU resources, memory resources, disk space resources, disk I/O resources, network I/O resources, and file handles.
  • the node reports the resource usage information and the task running information of the node to the resource manager in real time, so that the resource manager determines the historical characteristics of the application, and the manager for collecting the fault information of each node is real-time.
  • the fault information of each node is reported to the resource manager, so that the resource manager determines the historical characteristics of the node.
  • the resource manager schedules resources for the N applications according to the first historical feature information and the first scheduling policy; or the resource manager is configured according to the second historical feature information and the second scheduling policy. Tuning for at least one of the R applications Degree resources.
  • the first historical feature information is used in the process of scheduling resources for the N applications by using the first scheduling policy, and the first historical feature information includes historical features of the N applications, and therefore, according to the first historical feature.
  • the information and the first scheduling policy may only allocate resources for the N applications in the foregoing R applications, and the resources used by the applications other than the N applications in the R applications may be scheduled by using a preset scheduling policy. Scheduling, specifically, a description of the preset scheduling strategy is provided below.
  • the historical feature of the application is not needed in the process of scheduling the resource by using the second scheduling policy.
  • the resource manager may use the second historical feature information and the second scheduling policy as multiple applications. One or more or all of the applications in the scheduling resource.
  • the first scheduling policy may be: scheduling resources for N applications according to the comprehensive historical resource usage of each of the N applications, or according to each of the N applications.
  • the (1+x) times of the comprehensive historical resource usage is N application scheduling resources.
  • x can take 0.1 or 0.2.
  • the second scheduling policy may be: allocate resources of the nodes with lower failure rates among the M nodes earlier. In this case, if the number of applications running on the M nodes is small, the stability of the running application can be made high.
  • the specific content of the first scheduling policy and the second scheduling policy are not limited.
  • the first scheduling policy and the second scheduling policy are scheduling destination services. When the scheduling purpose changes, the first scheduling policy And the second scheduling strategy can also be adjusted accordingly.
  • the method further includes: when the resource manager does not successfully acquire the historical feature information, the resource manager schedules resources for the R applications according to a preset scheduling policy.
  • the preset scheduling policy may be any one of a scheduling policy such as a fair scheduling policy, a capacity scheduling policy, and a first-in-first-out scheduling policy.
  • the method further includes:
  • the resource manager adjusts a first scheduling parameter according to the first historical feature information, where the first scheduling parameter includes a scheduling parameter corresponding to each application of the N applications, where the first scheduling parameter is used by Determining an order of scheduling resources for the N applications; the resource manager scheduling resources for the N applications according to the first historical feature information and the first scheduling policy, including: the resource manager according to The adjusted first scheduling parameter and the first scheduling policy schedule resources for the N applications.
  • the method further includes:
  • the resource manager adjusts a first scheduling parameter according to the first historical feature information and the second historical feature information, where the first scheduling parameter includes a corresponding one of each of the N applications Scheduling parameters and scheduling parameters corresponding to each of the M nodes, the first scheduling parameter is used to determine an order of scheduling resources for the N applications and an order of scheduling resources of the M nodes; And the resource manager, according to the first historical feature information and the first scheduling policy, scheduling resources for the N applications, including: the resource manager according to the adjusted first scheduling parameter and the first scheduling policy Scheduling resources for the N applications.
  • the historical features of the N applications include a comprehensive historical running time of each of the N applications;
  • the historical features of the M nodes include each of the M nodes The failure rate of the node and the failure rate of the task, the resource manager adjusting the first scheduling parameter according to the first historical feature information and the second historical feature information, including:
  • the resource manager determines a comprehensive historical runtime of each of the N applications
  • the resource manager determines a failure rate and a task loss according to each of the M nodes
  • the resource manager allocates resources for the N applications according to the adjusted first scheduling parameter and the first scheduling policy, including: the resource manager allocates resources of a node with a higher node health rate to the resource manager The longer the integrated history runs, the longer the application.
  • the first scheduling policy that is, the resource manager allocates the resources of the node with the higher health rate to the application whose synthesis history is longer;
  • the first scheduling parameter is the health rate of each of the M nodes.
  • the initial value of ⁇ can be preset, for example, ⁇ can be 0.5.
  • the higher the health rate of the node the higher the stability of the node.
  • the optional method is because the resource of the node with higher health rate of the node is allocated to the comprehensive historical running time. Nodes that improve the operational stability of long-lived applications (that is, applications that take longer to run an application).
  • the method further includes: the resource manager calculating a task success rate of all the nodes in the multiple nodes multiple times, and adjusting the ⁇ according to the task success rate after each calculation is completed.
  • the value until the determined value of the alpha is such that the task has the highest success rate.
  • the task success rate of all nodes the number of successful running tasks of all nodes ⁇ the total number of tasks of all nodes running.
  • This optional method is able to determine the optimal alpha value, thus making the system's task success rate the highest.
  • the historical features of the N applications include comprehensive historical resource utilization, comprehensive historical resource usage, and comprehensive historical resource application amount of each of the N applications, and the resource manager Adjusting the first scheduling parameter according to the first historical feature information, including:
  • the resource manager uses a preset algorithm to calculate a comprehensive threshold of the application for the first time, and the first threshold corresponding to the application is obtained.
  • the resource manager determines the total historical resource application amount of the application as a second threshold corresponding to the application, and the first threshold is smaller than the second threshold;
  • the resource manager schedules resources for the application according to the adjusted first scheduling parameter and the first scheduling policy, including: the resource manager scheduling resources according to the first threshold, and monitoring in real time Whether the amount of resources actually used by the application exceeds the first threshold, and if so, the first threshold corresponding to the application is calculated by using a preset algorithm to calculate the amount of resources actually used by the application, and the first calculation is performed according to the current calculation.
  • the difference between the threshold and the previously calculated first threshold is to add resources to the application until the resource actually used by the application is the second threshold.
  • the first scheduling parameter is a first threshold and a second threshold.
  • the resource in the optional method may be any one of a node's CPU resource, memory resource, disk space resource, disk I/O resource, network I/O resource, and file handle number, and the optional method. Determining the first threshold and the second threshold according to the comprehensive historical resource usage of the application, first scheduling the resource according to the first threshold, and then according to the actual application The amount of resources used is adjusted to the amount of resources allocated by the application, thereby avoiding waste of resources and improving resource utilization.
  • the method further includes:
  • the resource manager adjusts a second scheduling parameter according to the second historical feature information, where the second scheduling parameter includes a scheduling parameter corresponding to each of the M nodes, and the second scheduling parameter is used to determine Scheduling the order of the resources of the M nodes; the resource manager scheduling resources for at least one of the R applications according to the second historical feature information and the second scheduling policy, including: the resources The manager schedules resources for at least one of the R applications according to the adjusted second scheduling parameter and the second scheduling policy.
  • the resource manager after receiving the resource request sent by multiple applications, acquires the historical features of the node and/or the application, and allocates the application according to the historical characteristics of the node and/or the application.
  • Resources because the historical features of the nodes can indirectly characterize the current or future characteristics of the nodes, the historical features of the application can indirectly characterize the current or future runtime characteristics of the application, in the embodiment of the present invention, not according to the application.
  • the amount of resources requested in the resource request of the program allocates resources to the application, but allocates resources to the application through the historical characteristics of the node and/or application, which prevents the resource manager from allocating the application according to the amount of resources requested by the application. The waste of resources caused by resources makes resource scheduling more reasonable.
  • the embodiment of the present invention further provides a resource scheduling method.
  • the foregoing resource scheduling method is exemplarily described based on the example in Table 1.
  • the first scheduling policy is: a node with a higher node health rate.
  • the resource is allocated to the application whose synthetic history is longer, as shown in FIG. 4, the method specifically includes:
  • the resource manager receives resource requests of four applications.
  • 4 applications are the 4 applications shown in Table 1.
  • the resource requests of the four applications may be resource requests of all the applications received by the resource manager in one heartbeat cycle, and the resource requests of the four applications may be submitted by the application of the four client devices. Resource request.
  • the resource manager determines a comprehensive historical running time of each of the four applications, and determines a failure rate and a task failure rate of each of the M nodes.
  • the resource manager receives the resource requests of the four applications on December 1, 2015, the historical running times of each of the four applications can be seen in Table 1, and the comprehensive history of each application runs.
  • the duration can be the average or maximum of the application's multiple historical run times, and application 4 has no historical run time.
  • the method for the resource manager to calculate the failure rate and the task failure rate of each of the M nodes can be referred to above, and details are not described herein again.
  • the resource manager calculates the health rate of the M nodes.
  • the resource manager is based on the application 1, the integrated historical running time of the application 2 and the application 3, the health rate of each of the M nodes, and the first scheduling policy for the application 1, the application 2, and the application. 3 scheduling resources.
  • the resource manager allocates the resources of the node with the higher health rate of the M nodes to the application program in the application 1, the application 2, and the application 3, which has a longer integrated history running time.
  • the resource manager may arrange the comprehensive historical running durations of the application 1, the application 2, and the application 3 in descending order, and the health rate of the M nodes is followed. The order is arranged from high to low, and then scheduled according to the arrangement result.
  • the resource manager schedules resources for the application 4 by using a preset scheduling policy.
  • the preset scheduling policy is used to schedule resources for the application 4, and the preset scheduling policy may be fair scheduling. Any one of the scheduling policies, such as the policy, the capacity scheduling policy, and the first-in-first-out scheduling policy.
  • the resource manager determines the comprehensive historical running time of the multiple applications and the health rate of the M nodes
  • the resource of the node with the higher health rate of the node may be allocated to the comprehensive historical running time. For long applications, the higher the health of the node, the higher the stability of the node. Therefore, this method can improve the running stability of long-term applications.
  • the embodiment of the present invention further provides a resource scheduling method, which is used to describe the resource scheduling method provided by the foregoing embodiment.
  • the first scheduling policy is: scheduling resources according to the first threshold value, And real-time monitoring whether the resource actually used by the application exceeds the first threshold. If yes, the first threshold corresponding to the application is calculated by using a preset algorithm to calculate the resource actually used by the application, and the first calculation is performed according to the current calculation. The difference between a threshold and a previously calculated first threshold is that the application adds resources until the resource actually used by the application is the second threshold.
  • the method specifically includes:
  • the resource manager receives a resource request of an application submitted by the client device.
  • the resource manager determines a comprehensive historical resource utilization rate, a comprehensive historical resource usage amount, and a comprehensive historical resource application amount of the application.
  • the “resource” here may be any one of a node's CPU resource, memory resource, disk space resource, disk I/O resource, network I/O resource, and file handle number.
  • step 503 When the resource manager successfully determines the comprehensive historical resource utilization, the comprehensive historical resource usage, and the comprehensive historical resource application amount of the application, step 503 is performed; otherwise, step 507 is performed.
  • the resource manager determines whether the comprehensive historical resource utilization of the application is less than a preset threshold.
  • the preset threshold may be 80%, 90%, or other values.
  • the resource manager uses a preset algorithm to calculate a first threshold value corresponding to the application for the initial history resource usage of the application, and determine the application amount of the comprehensive historical resource of the application as the second corresponding to the application. Threshold.
  • the first threshold is greater than the comprehensive historical resource usage of the application, and the first threshold is less than the second threshold.
  • the resource manager schedules resources for the application according to the first threshold, the second threshold, and the first scheduling policy.
  • the resource manager schedules the resource according to the first threshold, and monitors whether the resource actually used by the application exceeds the first threshold in real time, and if yes, calculates the resource actually used by the application by using a preset algorithm.
  • the first threshold corresponding to the application, and the difference between the first threshold obtained in the current calculation and the first threshold calculated in the previous time is added to the application, until the resource actually used by the application is the second threshold. .
  • the resource manager schedules resources for the application according to the comprehensive historical resource application amount of the application.
  • the comprehensive historical resource utilization of the application is greater than or equal to a preset threshold, if the comprehensive historical resource utilization of the application is considered to be high, the application for the comprehensive historical resource according to the application is the application.
  • Program scheduling resources are the application.
  • the resource manager uses a preset scheduling policy to schedule resources for the application.
  • the first scheduling policy cannot be used to schedule resources for the application, and therefore, the preset is adopted.
  • the scheduling policy allocates resources for the application, and the preset scheduling policy may be any one of a scheduling policy such as a fair scheduling policy, a capacity scheduling policy, and a first-in first-out scheduling policy.
  • the first threshold and the second threshold may be determined, where the first If the threshold is greater than the comprehensive historical resource usage, and the first threshold is less than the second threshold (that is, the comprehensive historical resource application amount), the resource manager may first schedule the resource according to the first threshold, and if the resource manager actually uses the resource. If the resource is always smaller than the first threshold, the resource is no longer added to the application. If the amount of resources actually used by the resource manager is greater than the first threshold, resources are added to the application, thereby preventing the application from occupying redundant resources and improving. Resource utilization.
  • the embodiment of the present invention further provides a resource scheduling apparatus 60, configured to manage and schedule cluster resources, where the cluster includes multiple nodes, and R applications that are adapted to the computing framework supported by the cluster run in the multiple On the node, R is a positive integer.
  • the device 60 includes:
  • the receiving unit 601 is configured to receive a resource request of the R applications.
  • the obtaining unit 602 is configured to acquire historical feature information, where the historical feature information includes at least one of first historical feature information and second historical feature information, where the first historical feature information is in the R applications a historical feature of the N applications; the second historical feature information is a historical feature of the M nodes of the plurality of nodes; the M nodes are M nodes of the plurality of nodes having idle resources , where the historical features of the application include At least one of information indicating a history resource usage of the application and information indicating a historical running time of the application, the historical characteristic of the node includes at least one of a failure rate of the node and a failure rate of the task. , 1 ⁇ N ⁇ R, M ⁇ 1, N, M are integers;
  • the scheduling unit 603 is configured to schedule resources for the N applications according to the first historical feature information and the first scheduling policy; or, the R applications according to the second historical feature information and the second scheduling policy At least one application in the program dispatches resources.
  • the device 60 further includes:
  • the adjusting unit 604 is configured to adjust the first scheduling parameter according to the first historical feature information, where the first scheduling parameter includes a scheduling parameter corresponding to each of the N applications, and the first scheduling parameter Used to determine an order in which resources are scheduled for the N applications;
  • the scheduling unit 603 is specifically configured to: schedule resources for the N applications according to the adjusted first scheduling parameter and the first scheduling policy.
  • the device 60 further includes:
  • the adjusting unit 604 is configured to adjust the first scheduling parameter according to the first historical feature information and the second historical feature information, where the first scheduling parameter includes each of the N applications a scheduling parameter and a scheduling parameter corresponding to each of the M nodes, the first scheduling parameter is used to determine an order of scheduling resources for the N applications and a sequence of scheduling resources of the M nodes ;
  • the scheduling unit 603 is specifically configured to: schedule resources for the N applications according to the adjusted first scheduling parameter and the first scheduling policy.
  • the device 60 further includes:
  • the adjusting unit 604 is configured to adjust, according to the second historical feature information, a second scheduling parameter, where the second scheduling parameter includes a scheduling parameter corresponding to each of the M nodes, where the second scheduling parameter is used Determining an order in which resources of the M nodes are scheduled;
  • the scheduling unit 603 is specifically configured to: schedule resources for at least one of the R applications according to the adjusted second scheduling parameter and the second scheduling policy.
  • the historical features of the N applications include a comprehensive historical running time of each of the N applications;
  • the historical features of the M nodes include each of the M nodes Node failure rate and task failure rate;
  • the adjusting unit 604 is specifically configured to: determine a comprehensive historical running time of each of the N applications; calculate the failure rate and the task failure rate according to each of the M nodes.
  • the scheduling unit 603 is specifically configured to: allocate resources of a node whose node has a higher health rate to an application whose synthesis history is longer.
  • the device 60 further includes:
  • the calculating unit 605 is configured to calculate a task success rate of all the nodes in the multiple times, and adjust the value of the ⁇ according to the task success rate after each calculation is completed, until the determined ⁇ is determined.
  • the value of the task is such that the task has the highest success rate.
  • the historical features of the N applications include comprehensive historical resource utilization, comprehensive historical resource usage, and comprehensive historical resource application amount of each of the N applications;
  • the adjusting unit 604 is specifically configured to: when the comprehensive historical resource utilization of the application is less than a preset threshold, use a preset algorithm to calculate the comprehensive historical resource usage of the application for the first time to obtain the first corresponding to the application.
  • a threshold the first threshold is greater than a comprehensive historical resource usage of the application;
  • the comprehensive historical resource application amount of the application is determined as a second threshold corresponding to the application, and the first threshold is smaller than the second threshold ;
  • the scheduling unit 603 is specifically configured to: schedule the application according to the first threshold Resource, and real-time monitoring whether the amount of resources actually used by the application exceeds a first threshold, and if so, using a preset algorithm, the amount of resources actually used by the application is again calculated to obtain a first threshold corresponding to the application, and according to this time
  • the difference between the calculated first threshold and the previous calculated first threshold is that the application adds resources until the resource actually used by the application is the second threshold.
  • the historical resource includes one or more resources of a CPU, a memory resource, a disk space resource, a disk I/O resource, a network I/O resource, and a file handle resource of the node. .
  • the embodiment of the present invention further provides a resource scheduling apparatus 80, configured to manage and schedule cluster resources, where the cluster includes multiple nodes, and R applications that are adapted to the computing framework supported by the cluster run in the multiple On the node, R is a positive integer, the device 80 includes: a receiver 801, a memory 802, a processor 803, and a bus system 804;
  • the receiver 801, the memory 802 and the processor 803 are coupled together by a bus system 804, wherein the memory 802 may include random access memory, and may also include non-volatile memory, such as at least one disk storage.
  • the bus system 804 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus.
  • ISA Industry Standard Architecture
  • PCI Peripheral Component
  • EISA Extended Industry Standard Architecture
  • the bus system 804 can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is shown in Figure 8, but it does not mean that there is only one bus or one type of bus.
  • the receiver 801, the memory 802, and the processor 803 are configured to perform the method described in the foregoing embodiment.
  • the receiver 801 is configured to perform the method according to FIG. Step 301
  • the processor is configured to perform the method according to FIG. Step 302 and step 303
  • the receiver 801 is further configured to perform step 401 in the method according to FIG. 4
  • the processor is further configured to perform step 402 to step 405 in the method according to FIG. 4
  • the receiver 801 further For performing step 501 in the method based on FIG. 5, the processor is further configured to perform steps 502 to 507 in the method based on FIG. 5.
  • the receiving unit 601 can be a receiver 801, and the obtaining unit 602, the scheduling unit 603, the adjusting unit 604, and the calculating unit 605 can be the processor 803.
  • the receiving unit in device 60 and the receiver in device 80 may be radio frequency (RF) circuits, and other units in device 60 may be embedded in hardware or independent of the device.
  • the processor may be stored in the memory of the device in a software format, so that the processor may perform the operations corresponding to the above units, and the processor may be a CPU, an Application Specific Integrated Circuit (ASIC), or configured.
  • ASIC Application Specific Integrated Circuit
  • the disclosed apparatus and method may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the modules is only a logical function division.
  • there may be another division manner for example, multiple modules or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the modules described as separate components may or may not be physically separated.
  • the components displayed as modules may or may not be physical modules, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional module in various embodiments of the present invention may be integrated into one processing module.
  • two or more modules can also be integrated in one module.
  • the above integrated modules can be implemented in the form of hardware or in the form of hardware plus software function modules.
  • the above-described integrated modules implemented in the form of software function modules can be stored in a computer readable storage medium.
  • the software functional modules described above are stored in a storage medium and include instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform some of the steps of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, and the program code can be stored. Medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)
  • Stored Programmes (AREA)

Abstract

La présente invention concerne un procédé et un dispositif de planification de ressources, se rapportant au domaine technique de la gestion de ressources et utilisés pour augmenter l'utilisation de ressources de grappe. Le procédé est utilisé pour un gestionnaire de ressources ; le gestionnaire de ressources est utilisé pour gérer et planifier des ressources de groupe ; une grappe comprend une pluralité de nœuds et R applications adaptées au cadre de calcul pris en charge par la grappe sont exécutées sur la pluralité de nœuds, R étant un nombre entier positif ; le procédé comprend les étapes suivantes : un gestionnaire de ressources reçoit des demandes de ressources en provenance des R applications (301) ; le gestionnaire de ressources obtient des informations de caractéristique d'historique, lesdites informations de caractéristique d'historique comportant des premières informations de caractéristique d'historique et/ou des secondes informations de caractéristique d'historique (302) ; en fonction des premières informations de caractéristiques d'historique et d'une première politique de planification, le gestionnaire de ressources planifie des ressources pour les N applications ; ou, en fonction des secondes informations de caractéristiques d'historique et d'une seconde politique de planification, le gestionnaire de ressources planifie des ressources pour au moins l'une des R applications (303).
PCT/CN2016/103591 2016-03-30 2016-10-27 Procédé et dispositif de planification de ressources WO2017166803A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610193081.0 2016-03-30
CN201610193081.0A CN107291546B (zh) 2016-03-30 2016-03-30 一种资源调度方法及装置

Publications (1)

Publication Number Publication Date
WO2017166803A1 true WO2017166803A1 (fr) 2017-10-05

Family

ID=59963388

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/103591 WO2017166803A1 (fr) 2016-03-30 2016-10-27 Procédé et dispositif de planification de ressources

Country Status (2)

Country Link
CN (1) CN107291546B (fr)
WO (1) WO2017166803A1 (fr)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111813548A (zh) * 2020-06-30 2020-10-23 北京金山云网络技术有限公司 一种资源调度方法、装置、电子设备及存储介质
CN112286673A (zh) * 2019-07-22 2021-01-29 北京车和家信息技术有限公司 一种节点资源调配方法及装置
CN112789599A (zh) * 2018-11-30 2021-05-11 北京比特大陆科技有限公司 信息推荐方法、装置、设备和可读存储介质
CN113377540A (zh) * 2021-06-15 2021-09-10 上海商汤科技开发有限公司 集群资源调度方法及装置、电子设备和存储介质
CN114095567A (zh) * 2021-10-12 2022-02-25 鸬鹚科技(深圳)有限公司 数据访问请求的处理方法、装置、计算机设备及介质
CN114650292A (zh) * 2020-12-17 2022-06-21 中移(苏州)软件技术有限公司 一种跨域数据传输方法、装置及存储介质
CN115794337A (zh) * 2022-11-14 2023-03-14 北京百度网讯科技有限公司 资源调度方法、装置、云平台、设备及存储介质
CN116074262A (zh) * 2023-01-07 2023-05-05 廊坊奎达信息技术有限公司 一种基于大数据平台的资源优化分配方法

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107909207A (zh) * 2017-11-16 2018-04-13 北京卫星信息工程研究所 遥感卫星地面站多目标优化调度方法及系统
CN108733484B (zh) * 2018-04-13 2021-12-03 华为技术有限公司 管理应用程序的方法与装置
CN110597614B (zh) * 2018-06-12 2023-11-21 阿里巴巴集团控股有限公司 一种资源调整方法及装置
CN109032793B (zh) * 2018-07-11 2021-03-16 Oppo广东移动通信有限公司 资源配置的方法、装置、终端及存储介质
CN108932165A (zh) * 2018-07-19 2018-12-04 中山大学 一种计算机集群资源分配与调度系统
CN110012054B (zh) 2018-12-28 2020-06-26 阿里巴巴集团控股有限公司 一种基于联盟链网络的业务处理方法及系统
CN109857550A (zh) * 2019-01-07 2019-06-07 平安科技（深圳）有限公司 基于机器学习的资源配置方法、装置、设备及存储介质
CN111221629B (zh) * 2020-01-09 2023-09-05 上海沄界信息科技有限公司 计算资源使用量量化方法及装置
CN111625331B (zh) * 2020-05-20 2023-06-16 拉扎斯网络科技（上海）有限公司 任务调度方法、装置、平台、服务器及存储介质
CN112099952A (zh) * 2020-09-16 2020-12-18 亚信科技（中国）有限公司 资源调度方法、装置、电子设备及存储介质
CN113225388B (zh) * 2021-04-22 2023-05-16 深信服科技股份有限公司 一种分布式调度方法及装置、设备、存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050132379A1 (en) * 2003-12-11 2005-06-16 Dell Products L.P. Method, system and software for allocating information handling system resources in response to high availability cluster fail-over events
US20100205137A1 (en) * 2009-02-10 2010-08-12 International Business Machines Corporation Optimizing Power Consumption and Performance in a Hybrid Computer Evironment
CN103164283A (zh) * 2012-05-10 2013-06-19 上海兆民云计算科技有限公司 一种虚拟桌面系统中虚拟化资源动态调度管理方法及系统
CN103685563A (zh) * 2013-12-31 2014-03-26 湖南师范大学 一种基于移动通信网络的云计算系统及其资源管理方法
CN104102548A (zh) * 2014-08-04 2014-10-15 北京京东尚科信息技术有限公司 任务资源调度处理方法和系统

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104391749B (zh) * 2014-11-26 2020-06-05 北京奇艺世纪科技有限公司 一种资源分配方法及装置
CN104915407B (zh) * 2015-06-03 2018-06-12 华中科技大学 一种基于Hadoop多作业环境下的资源调度方法
CN105404549B (zh) * 2015-12-06 2019-04-26 北京天云融创软件技术有限公司 基于yarn架构的虚拟机调度系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050132379A1 (en) * 2003-12-11 2005-06-16 Dell Products L.P. Method, system and software for allocating information handling system resources in response to high availability cluster fail-over events
US20100205137A1 (en) * 2009-02-10 2010-08-12 International Business Machines Corporation Optimizing Power Consumption and Performance in a Hybrid Computer Evironment
CN103164283A (zh) * 2012-05-10 2013-06-19 上海兆民云计算科技有限公司 一种虚拟桌面系统中虚拟化资源动态调度管理方法及系统
CN103685563A (zh) * 2013-12-31 2014-03-26 湖南师范大学 一种基于移动通信网络的云计算系统及其资源管理方法
CN104102548A (zh) * 2014-08-04 2014-10-15 北京京东尚科信息技术有限公司 任务资源调度处理方法和系统

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112789599A (zh) * 2018-11-30 2021-05-11 北京比特大陆科技有限公司 信息推荐方法、装置、设备和可读存储介质
CN112286673A (zh) * 2019-07-22 2021-01-29 北京车和家信息技术有限公司 一种节点资源调配方法及装置
CN112286673B (zh) * 2019-07-22 2024-05-24 北京车和家信息技术有限公司 一种节点资源调配方法及装置
CN111813548A (zh) * 2020-06-30 2020-10-23 北京金山云网络技术有限公司 一种资源调度方法、装置、电子设备及存储介质
CN114650292A (zh) * 2020-12-17 2022-06-21 中移(苏州)软件技术有限公司 一种跨域数据传输方法、装置及存储介质
CN114650292B (zh) * 2020-12-17 2024-01-30 中移(苏州)软件技术有限公司 一种跨域数据传输方法、装置及存储介质
CN113377540A (zh) * 2021-06-15 2021-09-10 上海商汤科技开发有限公司 集群资源调度方法及装置、电子设备和存储介质
CN114095567B (zh) * 2021-10-12 2023-12-29 鸬鹚科技(深圳)有限公司 数据访问请求的处理方法、装置、计算机设备及介质
CN114095567A (zh) * 2021-10-12 2022-02-25 鸬鹚科技(深圳)有限公司 数据访问请求的处理方法、装置、计算机设备及介质
CN115794337A (zh) * 2022-11-14 2023-03-14 北京百度网讯科技有限公司 资源调度方法、装置、云平台、设备及存储介质
CN115794337B (zh) * 2022-11-14 2023-09-26 北京百度网讯科技有限公司 资源调度方法、装置、云平台、设备及存储介质
CN116074262B (zh) * 2023-01-07 2023-10-31 廊坊奎达信息技术有限公司 一种基于大数据平台的资源优化分配方法
CN116074262A (zh) * 2023-01-07 2023-05-05 廊坊奎达信息技术有限公司 一种基于大数据平台的资源优化分配方法

Also Published As

Publication number Publication date
CN107291546B (zh) 2020-07-14
CN107291546A (zh) 2017-10-24

Similar Documents

Publication Publication Date Title
WO2017166803A1 (fr) Procédé et dispositif de planification de ressources
CN107580023B (zh) 一种动态调整任务分配的流处理作业调度方法及系统
CN112162865B (zh) 服务器的调度方法、装置和服务器
WO2017045553A1 (fr) Procédé et système d'attribution de tâches
CN110308983B (zh) 资源负载均衡方法及系统、服务节点和客户端
CN107832153B (zh) 一种Hadoop集群资源自适应分配方法
Chard et al. Cost-aware cloud provisioning
US9870269B1 (en) Job allocation in a clustered environment
CN109564528B (zh) 分布式计算中计算资源分配的系统和方法
Sun et al. Rose: Cluster resource scheduling via speculative over-subscription
US10356167B1 (en) Workload profiling
CN109614227B (zh) 任务资源调配方法、装置、电子设备及计算机可读介质
WO2017088393A1 (fr) Procédé et système d'attribution de largeur de bande
CN103761146B (zh) 一种MapReduce动态设定slots数量的方法
CN110599148B (zh) 集群数据处理方法、装置、计算机集群及可读存储介质
WO2016041446A1 (fr) Procédé, appareil et dispositif d'attribution de ressources
CN112130966A (zh) 任务调度方法和系统
CN111625331A (zh) 任务调度方法、装置、平台、服务器及存储介质
WO2017107456A1 (fr) Procédé et appareil permettant de déterminer les ressources consommées par une tâche
US10643193B2 (en) Dynamic workload capping
Choi et al. pHPA: A proactive autoscaling framework for microservice chain
US20120324466A1 (en) Scheduling Execution Requests to Allow Partial Results
KR20180072295A (ko) 분산 인 메모리 환경에서 실시간 스트림 데이터 처리를 위한 동적 잡 스케쥴링 시스템 및 방법
US9680657B2 (en) Cost optimization in dynamic workload capping
CN114116173A (zh) 动态调整任务分配的方法、装置和系统

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16896572

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 16896572

Country of ref document: EP

Kind code of ref document: A1