CN115981865B - Service resource scheduling method and device, electronic equipment and storage medium - Google Patents
Service resource scheduling method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115981865B CN115981865B CN202211743243.5A CN202211743243A CN115981865B CN 115981865 B CN115981865 B CN 115981865B CN 202211743243 A CN202211743243 A CN 202211743243A CN 115981865 B CN115981865 B CN 115981865B
- Authority
- CN
- China
- Prior art keywords
- task
- node
- time
- scheduling
- aggregation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000003860 storage Methods 0.000 title claims abstract description 12
- 230000002776 aggregation Effects 0.000 claims abstract description 122
- 238000004220 aggregation Methods 0.000 claims abstract description 122
- 238000010586 diagram Methods 0.000 claims abstract description 82
- 238000009826 distribution Methods 0.000 claims abstract description 13
- 238000004590 computer program Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000035484 reaction time Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000005304 joining Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005293 physical law Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a service resource scheduling method, a device, electronic equipment and a storage medium, which are used for solving the technical problems of low resource utilization rate and low service scheduling efficiency of the conventional service resource scheduling method. The invention comprises the following steps: acquiring scheduling arrangement information of each task; generating a service time sequence diagram by adopting the scheduling arrangement information; performing affinity aggregation on each service based on the service timing diagram to obtain aggregation clusters; acquiring the minimum resource demand of the aggregation group and the execution start-stop time; determining machine node leasing requirements and node task allocation information by adopting the minimum resource demand, the execution start-stop time and a preset node resource upper limit; and acquiring a machine node based on the node lease requirement, and distributing each task to the machine node according to the node task distribution information to carry out resource scheduling.
Description
Technical Field
The present invention relates to the field of resource scheduling technologies, and in particular, to a service resource scheduling method, device, electronic device, and storage medium.
Background
With the rise of cloud computing, dynamic expansion of resources as needed has become normal. How to more efficiently realize the automatic expansion service, thereby avoiding excessive waste of resources becomes a research focus. Currently the open source community provides three solutions for automatic telescoping based on the Kubernetes scheme: pod horizontal auto-scaling (Horizontal Pod Autoscaler, HPA), pod vertical auto-scaling (Vertical Pod Autoscaler, VPA) and cluster auto-scaling (Cluster Autoscaler, CA). On the basis of the open source community scheme, a task queue mechanism is introduced to improve the scheme, and based on the task queue, the clusters can be logically grouped, so that project management is convenient for users. On one hand, computing resources with different limits can be configured for different projects, and on the other hand, different types of services can be divided into different task queues for management. More refined management of traffic and computing resources is performed.
However, even though cloud computing has good elastic capability, the elastic capability is still limited by physical laws of reality, meanwhile, elastic expansion and contraction need to rely on data collection and analysis, and by default, an open source community can only passively grasp CPU and memory usage of Pod every 10 seconds. The summary analysis is performed once per minute (resources are wasted due to the excessive frequency of grabbing and summary analysis). If the new nodes are excessively created, resource waste can be caused. While the time required to create a new Pod on a new node has four main factors to determine: the reaction time of HPA, the reaction time of CA, the reaction time of Node, and the time of Pod creation. Creating new nodes during service execution can lead to time delay and affect service efficiency.
Disclosure of Invention
The invention provides a service resource scheduling method, a device, electronic equipment and a storage medium, which are used for solving the technical problems of low resource utilization rate and low service scheduling efficiency of the conventional service resource scheduling method.
The invention provides a service resource scheduling method, wherein the service comprises a plurality of tasks; the method comprises the following steps:
Acquiring scheduling arrangement information of each task;
Generating a service time sequence diagram by adopting the scheduling arrangement information;
performing affinity aggregation on each service based on the service timing diagram to obtain aggregation clusters;
acquiring the minimum resource demand of the aggregation group and the execution start-stop time;
Determining machine node leasing requirements and node task allocation information by adopting the minimum resource demand, the execution start-stop time and a preset node resource upper limit;
And acquiring a machine node based on the node lease requirement, and distributing each task to the machine node according to the node task distribution information to carry out resource scheduling.
Optionally, the step of generating a traffic timing diagram using the scheduling information includes:
generating an execution resource demand graph by adopting the scheduling arrangement information;
Acquiring the task starting time of each execution resource demand graph;
And generating a service time sequence diagram according to the task starting time and each execution resource demand diagram.
Optionally, the step of generating an execution resource requirement graph using the scheduling information includes:
acquiring resource demand information of each task from the scheduling information;
Acquiring the dependency relationship among the tasks;
dividing all the tasks into a plurality of task groups according to the dependency relationship;
Acquiring the scheduling time length of each task in the task group;
And generating an execution resource demand graph by adopting the resource demand information and the scheduling time length of each task in the task group.
Optionally, the step of generating a service timing diagram according to the task start time and each execution resource requirement diagram includes:
Calculating the scheduling time of each task according to the task starting time of the execution resource demand graph, the scheduling time of each task and the dependency relationship;
And generating a service time sequence diagram of the service according to the scheduling time of all tasks of the service.
Optionally, the execution start-stop time includes an execution start time and an execution end time; the aggregation group comprises a plurality of aggregation tasks; the step of obtaining the minimum resource requirement of the aggregation group and executing start-stop time comprises the following steps:
calculating the execution ending time of each aggregation task according to the scheduling time length and the scheduling time of each aggregation task;
taking the scheduling time with the forefront time in the aggregation task as the execution starting time of the aggregation grouping;
taking the last execution ending time of the time in the aggregation task as the execution ending time of the aggregation grouping;
Acquiring a resource requirement lower limit value of each aggregation task;
And calculating the sum of the lower limit values of the resource requirements of all the aggregation tasks of the aggregation cluster to obtain the minimum resource requirement of the aggregation cluster.
Optionally, the machine node leasing requirement includes the number of machine nodes and the machine node leasing duration; the step of determining the machine node lease requirement and the node task allocation information by adopting the minimum resource requirement, the execution start-stop time and the preset node resource upper limit comprises the following steps:
determining the number of the participation nodes required by each aggregation grouping by adopting the minimum resource demand of each aggregation grouping and the upper limit of the preset node resources;
Splicing the participation nodes of each aggregation group according to the number of the participation nodes of each aggregation group and the execution start-stop time, generating a plurality of participation node combinations, and determining the duration of each participation node combination;
Determining the number of the participating node combinations as the number of the machine nodes, and determining the duration of the participating node combinations as the duration of machine node leasing;
And determining node task allocation information according to the tasks in each participating node combination.
The invention also provides a service resource scheduling device, wherein the service comprises a plurality of tasks; the device comprises:
the scheduling arrangement information acquisition module is used for acquiring scheduling arrangement information of each task;
A service timing diagram generating module for generating a service timing diagram by using the scheduling information;
the aggregation module is used for carrying out affinity aggregation on each service based on the service time sequence diagram to obtain aggregation groups;
the minimum resource demand and execution start-stop time acquisition module is used for acquiring the minimum resource demand and execution start-stop time of the aggregation grouping;
The machine node leasing requirement and node task allocation information determining module is used for determining machine node leasing requirement and node task allocation information by adopting the minimum resource requirement amount, the execution start-stop time and a preset node resource upper limit;
And the scheduling module is used for acquiring the machine node based on the node lease requirement, and distributing each task to the machine node according to the node task distribution information to perform resource scheduling.
Optionally, the service timing diagram generating module includes:
An execution resource demand graph generation sub-module for generating an execution resource demand graph by adopting the scheduling arrangement information;
The task starting time acquisition sub-module is used for acquiring the task starting time of each execution resource demand graph;
and the service time sequence diagram generating sub-module is used for generating a service time sequence diagram according to the task starting time and each execution resource demand diagram.
The invention also provides an electronic device comprising a processor and a memory:
The memory is used for storing program codes and transmitting the program codes to the processor;
the processor is configured to execute the service resource scheduling method according to any one of the above claims according to instructions in the program code.
The present invention also provides a computer readable storage medium for storing program code for performing the method of traffic resource scheduling as defined in any one of the above.
From the above technical scheme, the invention has the following advantages: the invention provides a service resource scheduling method, which comprises the following steps: acquiring scheduling arrangement information of each task; generating a service time sequence diagram by adopting scheduling information; carrying out affinity aggregation on each service based on a service time sequence diagram to obtain aggregation groups; acquiring the minimum resource demand of the aggregation group and the execution start-stop time; determining the machine node leasing requirement and node task allocation information by adopting the minimum resource demand, the execution start-stop time and the preset node resource upper limit; and acquiring the machine node based on the node leasing requirement, and distributing each task to the machine node according to the node task distribution information to carry out resource scheduling. The invention determines the relation between the execution time of each task by generating the service time sequence diagram of the task in the service, thereby carrying out affinity aggregation on each task according to the relation, dividing the tasks in the same time interval together from the execution time sequence angle for machine node distribution, and further improving the resource utilization rate of the machine node as much as possible; meanwhile, the machine node can be created in advance by pre-calculating the demand of the machine node, so that the machine node can be directly called when the resource scheduling is carried out, the waiting time is reduced, and the service scheduling efficiency is improved.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the description below are only some embodiments of the invention, and that other drawings can be obtained from these drawings without inventive faculty for a person skilled in the art.
Fig. 1 is a flowchart of steps of a service resource scheduling method according to an embodiment of the present invention;
Fig. 2 is a flowchart of steps of a service resource scheduling method according to another embodiment of the present invention;
FIG. 3 is a schematic diagram of an initial execution resource requirement diagram according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of an execution resource requirement diagram according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a service timing diagram according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of machine node allocation with an execution resource requirement graph as an allocation principle;
FIG. 7 is a schematic diagram of an embodiment of an aggregation grouping;
fig. 8 is a schematic diagram of a participating node allocation according to an embodiment of the present invention;
Fig. 9 is a schematic view of joining participating nodes according to an embodiment of the present invention;
Fig. 10 is a block diagram of a service resource scheduling device according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a service resource scheduling method, a device, electronic equipment and a storage medium, which are used for solving the technical problems of low resource utilization rate and low service scheduling efficiency of the conventional service resource scheduling method.
In order to make the objects, features and advantages of the present invention more comprehensible, the technical solutions in the embodiments of the present invention are described in detail below with reference to the accompanying drawings, and it is apparent that the embodiments described below are only some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, fig. 1 is a flowchart illustrating steps of a service resource scheduling method according to an embodiment of the present invention.
The service resource scheduling method provided by the invention comprises a plurality of tasks, and the method specifically comprises the following steps:
Step 101, scheduling arrangement information of each task is obtained;
In the embodiment of the invention, the scheduling information can comprise the name and type of the task, the scheduling time of the task, the computing resources and the memory resources required by the task.
Step 102, generating a service time sequence diagram by adopting scheduling arrangement information;
The service timing diagram is a timing diagram of task execution in service, and the execution sequence of each task can be known through the service timing diagram.
After the scheduling information of the tasks is acquired, a traffic timing diagram may be generated based on the scheduling information of all the tasks of the traffic.
Step 103, carrying out affinity aggregation on each service based on the service time sequence diagram to obtain aggregation grouping;
In the embodiment of the present invention, affinity refers to the time overlapping degree of execution time of each task, and the higher the time overlapping degree is, the stronger the affinity is.
The services can be aggregated according to the time overlapping degree of each service in the service time sequence diagram, so as to obtain different aggregation groups. The execution times of tasks in the same aggregate cluster are similar.
104, Acquiring the minimum resource demand of the aggregation grouping and the execution start-stop time;
the minimum resource requirement of the aggregate cluster is the sum of the computing resources and the memory resources that are the minimum requirements of each task within the aggregate cluster.
The execution start-stop time is the start time of the task which starts to be executed first and the end time of the task which completes last in the aggregation group.
According to the minimum resource demand of the aggregation grouping, the upper limit of node resources required by the aggregation grouping can be known; the time of the node needed by the aggregation grouping can be known according to the execution start-stop time of the aggregation grouping.
Step 105, determining machine node leasing requirements and node task allocation information by adopting the minimum resource demand, execution start-stop time and preset node resource upper limit;
The upper limit of node resources refers to the upper limit of resources which can be provided by a node in unit time, and the resources comprise computing resources and memory resources.
The machine node lease requirement refers to the lease number and lease duration of the machine nodes requiring the known resource upper limit of lease.
In cloud computing, the calling of the machine nodes is required to pay the cost, so that the renting number and renting duration of the machine nodes are reduced, and the renting cost can be effectively saved. Meanwhile, the renting number and the renting duration of the machine nodes are controlled, so that the utilization rate of resources can be improved under the condition that the task execution requirements are met.
The node task allocation information is information in which tasks allocated to machine nodes to run are recorded, and different machine nodes may allocate different tasks.
In the embodiment of the invention, after the minimum resource demand, the execution start-stop time and the preset node resource upper limit are obtained, the machine node lease demand and the node task allocation information can be determined according to the execution start-stop time and the minimum resource demand.
And step 106, obtaining the machine node based on the node leasing requirement, and distributing each task to the machine node according to the node task distribution information to carry out resource scheduling.
In the embodiment of the invention, the machine nodes meeting the leasing requirement can be obtained, and each task is distributed to the corresponding machine node according to the node task distribution information to carry out resource scheduling so as to realize the normal operation of the task.
The invention determines the relation between the execution time of each task by generating the service time sequence diagram of the task in the service, thereby carrying out affinity aggregation on each task according to the relation, dividing the tasks in the same time interval together from the execution time sequence angle for machine node distribution, and further improving the resource utilization rate of the machine node as much as possible; meanwhile, the machine node can be created in advance by pre-calculating the demand of the machine node, so that the machine node can be directly called when the resource scheduling is carried out, the waiting time is reduced, and the service scheduling efficiency is improved.
Referring to fig. 2, fig. 2 is a flowchart illustrating steps of a service resource scheduling method according to another embodiment of the present invention. The method specifically comprises the following steps:
Step 201, obtaining scheduling information of each task;
In the embodiment of the invention, the scheduling information can comprise the name and type of the task, the scheduling time of the task, the computing resources and the memory resources required by the task.
Wherein example codes for scheduling the scheduling information are as follows:
step 202, generating an execution resource demand graph by adopting scheduling arrangement information;
And the resource demand graph is used for representing resources required by a series of tasks with dependency relationships when the tasks are executed. And recording the dependency relationship among the tasks, the names, the types, the scheduling time and the required resources of each task in the resource demand graph.
In the embodiment of the invention, the execution resource demand graph for executing the series of tasks can be generated according to the scheduling arrangement information of each task.
In one example, the step of generating the execution resource requirement map using scheduling information may comprise the sub-steps of:
s21, acquiring resource demand information of each task from scheduling information;
s22, acquiring the dependency relationship among the tasks;
s23, dividing all tasks into a plurality of task groups according to the dependency relationship;
S24, acquiring the scheduling time length of each task in the task group;
s25, generating an execution resource demand graph by adopting resource demand information and scheduling time length of each task in the task group.
In a specific implementation, a service may include a plurality of task groups having a dependency relationship, and each task group may generate an execution resource requirement graph. A dependency refers to the need for completion of one task to be based on the results of execution of another task.
Therefore, by acquiring the dependency relationship among the tasks, a plurality of task groups can be obtained, each task group is arranged according to the dependency relationship, and an initial execution resource demand graph as shown in fig. 3 can be obtained. In fig. 3, A, B, C, D is four tasks, where the execution basis of task B, C is the execution result of task a, and the execution basis of task D is the execution result of task B and task C. Each task is recorded with a corresponding Type (e.g. alpine: 3.7), resource requirement information (CPU: 100m, memory:32 Mi).
Considering that the scheduling time length of each task is different, the execution time of the task caused by the dependency relationship is influenced by the execution time of the previous task and the size of the scheduling time length, and for accurate scheduling of each task, the scheduling time length of each task can be added in the initial execution resource demand graph so as to calculate the execution time of each task. Accordingly, the generated execution resource requirement diagram carrying the scheduling Duration (Duration) is shown in fig. 4.
Step 203, acquiring the task starting time of each execution resource demand graph;
step 204, generating a service time sequence diagram according to the task starting time and each execution resource demand diagram;
after the execution resource demand graph of each task group is obtained, each task group can be arranged according to the task starting time of each execution resource demand graph, and a service time sequence graph of the service is obtained.
In one example, the step of generating a traffic timing diagram from a task start time and execution resource demand diagrams may include the sub-steps of:
s41, calculating the scheduling time of each task according to the task starting time of the execution resource demand graph, the scheduling time of each task and the dependency relationship;
s42, generating a service timing diagram of the service according to the scheduling time of all tasks of the service.
The task start time of executing the resource demand graph refers to the execution time of executing the task that starts to be executed first in the resource demand graph.
The scheduling time is the execution time when each task starts to execute.
In a specific implementation, the scheduling time of each task can be calculated according to the task starting time of the execution resource demand graph, the scheduling time of each task and the dependency relationship.
Taking fig. 4 as an example, the task a is the first task to start executing in the execution resource demand graph, the scheduling time is 0 minutes and 0 seconds, the scheduling time is 32 seconds, and the scheduling time of the task B and the task C depending on the execution result is 0 minutes and 32 seconds; the scheduling time length of the task B is 35 seconds, and the scheduling time length of the task C is 55 seconds; since task D needs to rely on the common result of task B and task C to start execution, the scheduling time of task D is based on the end time of task C that ends at the latest, and thus, the scheduling time of task D is 1 minute and 27 seconds.
And the same can calculate the scheduling time of each task in each execution resource demand graph.
Fig. 5 is a schematic structural diagram of a service timing diagram according to an embodiment of the present invention, as shown in fig. 5. Assuming that the service includes 26 tasks A-Z, a total of 6 task groups A-D, E-G, H-I, J-O, P-V, W-Z can be formed according to the dependency relationship. In the process of executing the task, the task group H-I needs to be executed twice. So that 7 execution resource demand maps can be generated. Based on the scheduling time of the tasks in each execution resource demand graph, a corresponding traffic timing graph can be generated.
Step 205, carrying out affinity aggregation on each service based on the service timing diagram to obtain aggregation clusters;
Affinity refers to the degree of temporal overlap in execution time between tasks, and the higher the degree of temporal overlap, the stronger the affinity.
In a practical scenario, if machine nodes are allocated according to the execution resource demand graph, a machine node allocation result as shown in fig. 6 will be formed.
Whereas in cloud computing, the invocation of a machine node is a fee. According to the machine node allocation method of fig. 6, the task is divided into three machine nodes (machine nodes 1,2, and 3), and the utilization rate of the machine nodes is low. In order to reduce the payment cost, the number of machine nodes and the lease time are required to be reduced.
In the embodiment of the invention, the tasks can be aggregated through affinity, so that the tasks in the same time interval can be intensively processed, and the participation number of the machine nodes can be reduced by intensively processing the tasks which are originally scattered and processed by different machine nodes.
In a specific implementation, the services can be aggregated according to the time overlapping degree of each service in the service time sequence diagram, so as to obtain different aggregation groups. The execution times of tasks in the same aggregate cluster are similar.
It should be noted that, the embodiment of the present invention does not restrict the aggregation algorithm, and those skilled in the art may adopt any feasible aggregation algorithm to aggregate under the condition of affinity.
In one example, based on the traffic timing diagram of FIG. 5, aggregate clusters, including aggregate clusters 1-4, as shown in FIG. 7, may be aggregated.
Step 206, obtaining the minimum resource demand of the aggregation grouping and the execution start-stop time;
After the aggregate cluster is acquired, a minimum resource requirement and execution start-stop time of the aggregate cluster may be calculated to schedule the node for the aggregate cluster.
In one example, the execution start-stop time includes an execution start time and an execution end time; the aggregation group comprises a plurality of aggregation tasks; the step of obtaining the minimum resource requirement of the aggregate clusters and performing start-stop times may comprise the sub-steps of:
s61, calculating the execution ending time of each aggregation task according to the scheduling time length and the scheduling time of each aggregation task;
S62, taking the scheduling time with the forefront time in the aggregation task as the execution starting time of the aggregation grouping;
s63, taking the last execution ending time of the time in the aggregation task as the execution ending time of the aggregation grouping;
S64, obtaining a resource requirement lower limit value of each aggregation task;
S65, calculating the sum of the lower limit values of the resource requirements of all the aggregation tasks of the aggregation group, and obtaining the minimum resource requirement of the aggregation group.
In a practical scenario, the loading capacity of a single machine node is limited, the demand computation force of each task is a range, the demand memory of the task is a range, and the task is also subject to the constraint of scheduling time. Thus, the number of tasks that each machine node can accommodate can be calculated based on the resource requirements of each task, the scheduling time, and the loading capabilities of the machine node. Similarly, according to the resource demand and the scheduling time of the aggregation tasks in the aggregation group, the number of the participation nodes (pre-allocated machine nodes) and the lease time required by the aggregation group can be calculated.
In a specific implementation, each aggregation group may include a plurality of aggregation tasks, the execution end time of each aggregation task may be calculated according to the scheduling time and the scheduling time of each aggregation task, and the lease time of the required participating node may be calculated according to the execution end time and the scheduling time of each task. And according to the lower limit value of the resource requirement of each aggregation task, the total minimum resource requirement of the aggregation group can be calculated.
Step 207, determining machine node leasing requirements and node task allocation information by adopting the minimum resource requirement amount, execution start-stop time and preset node resource upper limit;
in the embodiment of the invention, the minimum resource demand, the execution start-stop time and the preset node resource upper limit can be adopted to determine the machine node leasing demand and the node task allocation information.
In one example, the machine node lease requirements include a number of machine nodes and a machine node lease duration; the method comprises the steps of determining machine node leasing requirements and node task allocation information by adopting the minimum resource demand, execution start-stop time and preset node resource upper limit, and comprises the following steps:
S71, determining the number of the participation nodes required by each aggregation grouping by adopting the minimum resource demand of each aggregation grouping and the upper limit of the preset node resources;
s72, splicing the participation nodes of each aggregation group according to the number of the participation nodes of each aggregation group and the start-stop execution time, generating a plurality of participation node combinations, and determining the duration of each participation node combination;
s73, determining the number of the participating node combinations as the number of the machine nodes, and determining the duration of the participating node combinations as the duration of machine node leases;
and S74, determining node task allocation information according to the tasks in each participating node combination.
The preset node resource upper limit refers to the resource upper limit of the participating node (pre-allocated machine node), and comprises a computing resource upper limit and a memory resource upper limit.
In a specific implementation, the number of participating nodes required by each aggregation group can be calculated according to the minimum resource requirement of each aggregation group and the preset node resource upper limit. As shown in fig. 8, a, b, c, d, e are five participating nodes.
And splicing the participating nodes according to the execution time of the participating nodes, so that a plurality of participating node combinations can be obtained, wherein the number of the participating node combinations can be the number of machine nodes, the total duration of the participating node combinations can be the duration of machine node leasing, and the tasks in each participating node combination are node tasks (recorded in node task allocation information) allocated to the corresponding machine nodes. The splice results are shown in fig. 9.
And step 208, acquiring the machine node based on the node leasing requirement, and distributing each task to the machine node according to the node task distribution information to perform resource scheduling.
In the embodiment of the invention, the machine nodes meeting the leasing requirement can be obtained, and each task is distributed to the corresponding machine node according to the node task distribution information to carry out resource scheduling so as to realize the normal operation of the task.
The invention determines the relation between the execution time of each task by generating the service time sequence diagram of the task in the service, thereby carrying out affinity aggregation on each task according to the relation, dividing the tasks in the same time interval together from the execution time sequence angle for machine node distribution, and further improving the resource utilization rate of the machine node as much as possible; meanwhile, the machine node can be created in advance by pre-calculating the demand of the machine node, so that the machine node can be directly called when the resource scheduling is carried out, the waiting time is reduced, and the service scheduling efficiency is improved.
Referring to fig. 10, fig. 10 is a block diagram illustrating a service resource scheduling apparatus according to an embodiment of the present invention.
The embodiment of the invention provides a service resource scheduling device, wherein a service comprises a plurality of tasks; the device comprises:
A scheduling information acquiring module 1001, configured to acquire scheduling information of each task;
a service timing diagram generating module 1002, configured to generate a service timing diagram using scheduling information;
An aggregation module 1003, configured to perform affinity aggregation on each service based on the service timing diagram, to obtain an aggregate group;
A minimum resource demand and execution start-stop time acquisition module 1004, configured to acquire a minimum resource demand and execution start-stop time of an aggregate group;
A machine node lease requirement and node task allocation information determining module 1005 configured to determine machine node lease requirement and node task allocation information using a minimum resource requirement amount, an execution start-stop time, and a preset node resource upper limit;
The scheduling module 1006 is configured to obtain a machine node based on the node lease requirement, and allocate each task to the machine node according to the node task allocation information to perform resource scheduling.
In an embodiment of the present invention, the service timing diagram generating module 1002 includes:
An execution resource demand graph generation sub-module for generating an execution resource demand graph by adopting scheduling arrangement information;
The task starting time acquisition sub-module is used for acquiring the task starting time of each execution resource demand graph;
and the service time sequence diagram generating sub-module is used for generating a service time sequence diagram according to the task starting time and each execution resource demand diagram.
In an embodiment of the present invention, executing a resource demand graph generation sub-module includes:
the resource demand information acquisition unit is used for acquiring the resource demand information of each task from the scheduling information;
the dependency relationship acquisition unit is used for acquiring the dependency relationship among the tasks;
The task group dividing unit is used for dividing all tasks into a plurality of task groups according to the dependency relationship;
the scheduling time length acquisition unit is used for acquiring the scheduling time length of each task in the task group;
and the execution resource demand graph generating unit is used for generating an execution resource demand graph by adopting the resource demand information and the scheduling time length of each task in the task group.
In an embodiment of the present invention, a service timing diagram generating sub-module includes:
The scheduling time calculation unit is used for calculating the scheduling time of each task according to the task starting time of the execution resource demand graph, the scheduling time of each task and the dependency relationship;
And the service timing diagram generating unit is used for generating a service timing diagram of the service according to the scheduling time of all the tasks of the service.
In the embodiment of the invention, the execution start-stop time comprises an execution start time and an execution end time; the aggregation group comprises a plurality of aggregation tasks; the minimum resource demand and execution start-stop time acquisition module 1004 includes:
The execution end time calculation sub-module is used for calculating the execution end time of each aggregation task according to the scheduling time length and the scheduling time of each aggregation task;
the execution starting time determining submodule is used for taking the scheduling time with the forefront time in the aggregation task as the execution starting time of the aggregation grouping;
the execution termination time determining submodule is used for taking the last execution termination time of the time in the aggregation task as the execution termination time of the aggregation group;
the resource demand lower limit value acquisition sub-module is used for acquiring the resource demand lower limit value of each aggregation task;
and the minimum resource demand determining submodule is used for calculating the sum of the lower limit values of the resource demands of all the aggregation tasks of the aggregation group to obtain the minimum resource demand of the aggregation group.
In the embodiment of the invention, the machine node leasing requirement comprises the number of machine nodes and the machine node leasing duration; machine node lease requirement and node task allocation information determination module 1005 includes:
The number determination submodule is used for determining the number of the participation nodes required by each aggregation grouping by adopting the minimum resource demand of each aggregation grouping and the upper limit of the preset node resources;
The time length determining submodule is used for splicing the participation nodes of each aggregation group according to the number of the participation nodes of each aggregation group and the start-stop execution time, generating a plurality of participation node combinations and determining the time length of each participation node combination;
The machine node number and machine node lease time determination submodule is used for determining the number of the participating node combinations as the number of the machine nodes and determining the time of the participating node combinations as the machine node lease time;
The node task allocation information determination submodule is used for determining node task allocation information according to the tasks in each participating node combination.
The embodiment of the invention also provides electronic equipment, which comprises a processor and a memory:
the memory is used for storing the program codes and transmitting the program codes to the processor;
the processor is configured to execute the service resource scheduling method according to the embodiment of the present invention according to the instructions in the program code.
The embodiment of the invention also provides a computer readable storage medium, which is used for storing program codes, and the program codes are used for executing the service resource scheduling method of the embodiment of the invention.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described by differences from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other.
It will be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the invention may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or terminal device that comprises the element.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (9)
1. A service resource scheduling method is characterized in that the service comprises a plurality of tasks; the method comprises the following steps:
Acquiring scheduling arrangement information of each task;
Generating a service time sequence diagram by adopting the scheduling arrangement information;
performing affinity aggregation on each service based on the service timing diagram to obtain aggregation clusters;
acquiring the minimum resource demand of the aggregation group and the execution start-stop time;
Determining machine node leasing requirements and node task allocation information by adopting the minimum resource demand, the execution start-stop time and a preset node resource upper limit;
Acquiring a machine node based on the node lease requirement, and distributing each task to the machine node according to the node task distribution information to perform resource scheduling;
the machine node leasing requirements comprise the number of machine nodes and the machine node leasing duration; the step of determining the machine node lease requirement and the node task allocation information by adopting the minimum resource requirement, the execution start-stop time and the preset node resource upper limit comprises the following steps:
determining the number of the participation nodes required by each aggregation grouping by adopting the minimum resource demand of each aggregation grouping and the upper limit of the preset node resources;
Splicing the participation nodes of each aggregation group according to the number of the participation nodes of each aggregation group and the execution start-stop time, generating a plurality of participation node combinations, and determining the duration of each participation node combination;
Determining the number of the participating node combinations as the number of the machine nodes, and determining the duration of the participating node combinations as the duration of machine node leasing;
And determining node task allocation information according to the tasks in each participating node combination.
2. The method of claim 1, wherein the step of generating a traffic timing diagram using the scheduling information comprises:
generating an execution resource demand graph by adopting the scheduling arrangement information;
Acquiring the task starting time of each execution resource demand graph;
And generating a service time sequence diagram according to the task starting time and each execution resource demand diagram.
3. The method of claim 1, wherein the step of generating an execution resource requirement map using the scheduling information comprises:
acquiring resource demand information of each task from the scheduling information;
Acquiring the dependency relationship among the tasks;
dividing all the tasks into a plurality of task groups according to the dependency relationship;
Acquiring the scheduling time length of each task in the task group;
And generating an execution resource demand graph by adopting the resource demand information and the scheduling time length of each task in the task group.
4. A method according to claim 3, wherein said step of generating a traffic timing diagram from said task start time and each of said execution resource demand diagrams comprises:
Calculating the scheduling time of each task according to the task starting time of the execution resource demand graph, the scheduling time of each task and the dependency relationship;
And generating a service time sequence diagram of the service according to the scheduling time of all tasks of the service.
5. The method of claim 4, wherein the execution start-stop time comprises an execution start time and an execution end time; the aggregation group comprises a plurality of aggregation tasks; the step of obtaining the minimum resource requirement of the aggregation group and executing start-stop time comprises the following steps:
calculating the execution ending time of each aggregation task according to the scheduling time length and the scheduling time of each aggregation task;
taking the scheduling time with the forefront time in the aggregation task as the execution starting time of the aggregation grouping;
taking the last execution ending time of the time in the aggregation task as the execution ending time of the aggregation grouping;
Acquiring a resource requirement lower limit value of each aggregation task;
And calculating the sum of the lower limit values of the resource requirements of all the aggregation tasks of the aggregation cluster to obtain the minimum resource requirement of the aggregation cluster.
6. A service resource scheduling device, wherein the service comprises a plurality of tasks; the device comprises:
the scheduling arrangement information acquisition module is used for acquiring scheduling arrangement information of each task;
A service timing diagram generating module for generating a service timing diagram by using the scheduling information;
the aggregation module is used for carrying out affinity aggregation on each service based on the service time sequence diagram to obtain aggregation groups;
the minimum resource demand and execution start-stop time acquisition module is used for acquiring the minimum resource demand and execution start-stop time of the aggregation grouping;
The machine node leasing requirement and node task allocation information determining module is used for determining machine node leasing requirement and node task allocation information by adopting the minimum resource requirement amount, the execution start-stop time and a preset node resource upper limit;
The scheduling module is used for acquiring machine nodes based on the node leasing requirements, and distributing the tasks to the machine nodes according to the node task distribution information to perform resource scheduling;
the machine node leasing requirements comprise the number of machine nodes and the machine node leasing duration; the machine node leasing requirement and node task allocation information determining module comprises:
The number determination submodule is used for determining the number of the participation nodes required by each aggregation grouping by adopting the minimum resource demand of each aggregation grouping and the upper limit of the preset node resources;
The time length determining submodule is used for splicing the participation nodes of each aggregation group according to the number of the participation nodes of each aggregation group and the start-stop execution time, generating a plurality of participation node combinations and determining the time length of each participation node combination;
The machine node number and machine node lease time determination submodule is used for determining the number of the participating node combinations as the number of the machine nodes and determining the time of the participating node combinations as the machine node lease time;
The node task allocation information determination submodule is used for determining node task allocation information according to the tasks in each participating node combination.
7. The apparatus of claim 6, wherein the traffic timing diagram generation module comprises:
An execution resource demand graph generation sub-module for generating an execution resource demand graph by adopting the scheduling arrangement information;
The task starting time acquisition sub-module is used for acquiring the task starting time of each execution resource demand graph;
and the service time sequence diagram generating sub-module is used for generating a service time sequence diagram according to the task starting time and each execution resource demand diagram.
8. An electronic device, the device comprising a processor and a memory:
The memory is used for storing program codes and transmitting the program codes to the processor;
the processor is configured to perform the service resource scheduling method of any one of claims 1-5 according to instructions in the program code.
9. A computer readable storage medium for storing program code for performing the traffic resource scheduling method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211743243.5A CN115981865B (en) | 2022-12-29 | 2022-12-29 | Service resource scheduling method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211743243.5A CN115981865B (en) | 2022-12-29 | 2022-12-29 | Service resource scheduling method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115981865A CN115981865A (en) | 2023-04-18 |
| CN115981865B true CN115981865B (en) | 2024-05-07 |
Family
ID=85966459
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211743243.5A Active CN115981865B (en) | 2022-12-29 | 2022-12-29 | Service resource scheduling method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115981865B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109725993A (en) * | 2018-06-01 | 2019-05-07 | 中国平安人寿保险股份有限公司 | Task executing method, device, equipment and computer readable storage medium |
| CN112884319A (en) * | 2021-02-10 | 2021-06-01 | 腾讯大地通途（北京）科技有限公司 | Task allocation method and device, computer equipment and storage medium |
| CN114896067A (en) * | 2022-05-26 | 2022-08-12 | 中国平安人寿保险股份有限公司 | Automatic generation method and device of task request information, computer equipment and medium |
| CN114896049A (en) * | 2022-06-16 | 2022-08-12 | 中国电力科学研究院有限公司 | Method, system, equipment and medium for scheduling operation tasks of electric power artificial intelligence platform |
| CN115098240A (en) * | 2022-07-25 | 2022-09-23 | 中诚华隆计算机技术有限公司 | Multiprocessor application scheduling method and system and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11372690B2 (en) * | 2019-10-03 | 2022-06-28 | Microsoft Technology Licensing, Llc | Multi-phase distributed task coordination |
-
2022
- 2022-12-29 CN CN202211743243.5A patent/CN115981865B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109725993A (en) * | 2018-06-01 | 2019-05-07 | 中国平安人寿保险股份有限公司 | Task executing method, device, equipment and computer readable storage medium |
| CN112884319A (en) * | 2021-02-10 | 2021-06-01 | 腾讯大地通途（北京）科技有限公司 | Task allocation method and device, computer equipment and storage medium |
| CN114896067A (en) * | 2022-05-26 | 2022-08-12 | 中国平安人寿保险股份有限公司 | Automatic generation method and device of task request information, computer equipment and medium |
| CN114896049A (en) * | 2022-06-16 | 2022-08-12 | 中国电力科学研究院有限公司 | Method, system, equipment and medium for scheduling operation tasks of electric power artificial intelligence platform |
| CN115098240A (en) * | 2022-07-25 | 2022-09-23 | 中诚华隆计算机技术有限公司 | Multiprocessor application scheduling method and system and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115981865A (en) | 2023-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP4386410B2 (en) | Method and system for dynamically allocating computer system resources | |
| CN105912399B (en) | Task processing method, device and system | |
| US20110202657A1 (en) | Method for scheduling cloud-computing resource and system applying the same | |
| CN111459641B (en) | Method and device for task scheduling and task processing across machine room | |
| CN111258745B (en) | Task processing method and device | |
| CN110347515B (en) | Resource optimization allocation method suitable for edge computing environment | |
| CN112130966A (en) | Task scheduling method and system | |
| CN110636388A (en) | Service request distribution method, system, electronic equipment and storage medium | |
| CN111798113A (en) | Resource allocation method, device, storage medium and electronic equipment | |
| CN109254913A (en) | Collaborative share method, apparatus, electronic equipment and computer storage medium | |
| Denninnart et al. | Improving robustness of heterogeneous serverless computing systems via probabilistic task pruning | |
| Mokhtari et al. | Autonomous task dropping mechanism to achieve robustness in heterogeneous computing systems | |
| CN106775975B (en) | Process scheduling method and device | |
| CN111580974A (en) | GPU instance distribution method and device, electronic equipment and computer readable medium | |
| CN115981865B (en) | Service resource scheduling method and device, electronic equipment and storage medium | |
| CN109347982A (en) | A kind of dispatching method and device of data center | |
| CN112286623B (en) | Information processing method and device and storage medium | |
| CN110502343B (en) | Resource allocation method, system, device and computer readable storage medium | |
| CN111143063A (en) | Task resource reservation method and device | |
| WO2016206441A1 (en) | Method and device for allocating virtual resource, and computer storage medium | |
| CN115129481B (en) | Computing resource allocation method and device and electronic equipment | |
| CN116820729A (en) | Offline task scheduling method and device and electronic equipment | |
| CN114745278B (en) | Method and device for expanding and shrinking capacity of service system, electronic equipment and storage medium | |
| JP2020154391A (en) | Information processing system and program | |
| CN114124763A (en) | Method and system for measuring broadband access rate |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |