CN110365799B - Cluster multi-node scheduling method and device and computer readable storage medium - Google Patents
Cluster multi-node scheduling method and device and computer readable storage medium Download PDFInfo
- Publication number
- CN110365799B CN110365799B CN201910760105.XA CN201910760105A CN110365799B CN 110365799 B CN110365799 B CN 110365799B CN 201910760105 A CN201910760105 A CN 201910760105A CN 110365799 B CN110365799 B CN 110365799B
- Authority
- CN
- China
- Prior art keywords
- scheduling
- node
- cost value
- value
- nodes
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 14
- 238000004364 calculation method Methods 0.000 claims abstract description 8
- 238000004891 communication Methods 0.000 claims description 91
- 230000006870 function Effects 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 11
- 239000002131 composite material Substances 0.000 claims 3
- 238000012545 processing Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011158 quantitative evaluation Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
Abstract
The embodiment of the invention discloses a cluster multi-node scheduling method, a cluster multi-node scheduling device and a computer readable storage medium, which are used for acquiring scheduling tasks; and determining the cost value of each node in the cluster resource pool according to a preset node cost table and the service requirement of the scheduling task on each node. And traversing the cluster resource pool by using a depth-first algorithm, determining scheduling nodes required by the scheduling task, and calculating the comprehensive cost value of the current scheduling scheme according to the cost value of each scheduling node. The scheduling scheme comprises nodes used for executing the scheduling tasks, and the comprehensive cost value represents the cost value of executing the scheduling tasks according to the nodes contained in the scheduling scheme. And when all scheduling nodes required by the scheduling task are obtained through traversal, selecting a scheduling scheme meeting the requirement of the comprehensive cost value as the scheduling scheme of the scheduling task. The influence of different factors on the scheduling task is skillfully integrated by a cost value calculation mode, so that a more reasonable and efficient scheduling scheme can be selected.
Description
Technical Field
The present invention relates to the field of resource scheduling technologies, and in particular, to a cluster multi-node scheduling method, apparatus, and computer-readable storage medium.
Background
With the increasing use of distributed computing, situations are often encountered in the development process where multiple resource nodes are scheduled at once, which is the case for substantially all clusters. When the prior art processes the situation of multi-node scheduling, a feasible scheme is generally obtained according to a certain condition as a basis, and scheduling is performed according to the scheme.
In practice, however, the requirements for each node may vary for the plurality of nodes performing the task. If a scheduling scheme is obtained according to a certain condition in the prior art, the scheduled nodes are often unreasonable, and the cluster performance cannot achieve a better effect.
For example, the following steps are carried out: the cluster has a total of three nodes, node1, node2, node 3; assuming that all three node resources are available, the resources are generally referred to as a Central Processing Unit (CPU), a memory, and a Graphics Processing Unit (GPU). The cluster now needs to perform a three-node task, node a has the highest performance requirement, and A, C the communication between them is frequent. For node1, node2 and node3, in terms of performance node1 is slightly larger than node2, both of which are much larger than node 3; the communication speed between the nodes 2, 3 is much faster than that between the node1 and the nodes 2, 3 in the aspect of communication between the nodes. Scheduling a to node2, B to node1, and C to node3 would be done by taking into account the various factors scheduling. However, according to the prior art, the scheduling scheme is obtained only based on a single condition, for example, based on the aspect that the node a has high performance requirement, a is scheduled to the node1, which often results in poor rationality of the scheduling scheme.
Therefore, how to improve the rationality of the scheduling scheme is a problem to be solved by those skilled in the art.
Disclosure of Invention
The embodiment of the invention aims to provide a cluster multi-node scheduling method, a cluster multi-node scheduling device and a computer readable storage medium, which can improve the rationality of a scheduling scheme.
To solve the foregoing technical problem, an embodiment of the present invention provides a cluster multi-node scheduling method, including:
acquiring a scheduling task;
determining a cost value of each node in a cluster resource pool according to a preset node cost table and service requirements of the scheduling task on each node;
traversing the cluster resource pool by using a depth-first algorithm, determining scheduling nodes required by the scheduling task, and calculating the comprehensive cost value of the current scheduling scheme according to the cost value of each scheduling node;
and when all scheduling nodes required by the scheduling task are obtained through traversal, selecting a scheduling scheme meeting the requirement of the comprehensive cost value as the scheduling scheme of the scheduling task.
Optionally, the determining the cost value of each node in the cluster resource pool according to the preset node cost table and the service requirement of the scheduling task on each node includes:
determining a communication requirement value between nodes according to the communication frequency between the nodes in the scheduling task;
multiplying the communication grade value between each node in a preset node cost table by the corresponding communication requirement value between the nodes to obtain the communication proxy value between each node;
and determining the performance cost value of each node according to the functional requirements of the scheduling task on the nodes.
Optionally, the calculating a comprehensive cost value of the current scheduling scheme according to the cost value of each scheduling node includes:
according to the following formula, calculating the comprehensive cost value C of the current scheduling scheme,
wherein, r [ i, m [ i ] ] represents the performance cost value of the ith node in the scheduling cluster resource pool of the ith node of the scheduling task, t [ i, j ] represents the communication requirement value of the ith node and the jth node of the scheduling task, s [ m [ i ], m [ j ] represents the communication grade value of the mth node and the mth node in the cluster resource pool, and k represents the number of nodes contained in the current scheduling scheme.
Optionally, after the calculating a comprehensive cost value of the current scheduling scheme according to the cost value of each scheduling node, the method further includes:
when the comprehensive cost value of the first scheduling scheme is larger than the comprehensive cost value of the second scheduling scheme, abandoning the first scheduling scheme; the first scheduling scheme is a current scheduling scheme of all scheduling nodes required by the scheduling task which is not obtained through traversal; the second scheduling scheme is a scheduling scheme of all scheduling nodes required by the scheduling task after traversal is obtained.
Optionally, the determining the performance cost value of each node according to the functional requirement of the scheduling task on the node includes:
acquiring the functional requirements of the scheduling task on the nodes;
and searching performance cost values corresponding to different functions of each node in a preset node cost table according to the function requirement, and determining the performance cost value of each node corresponding to the function requirement.
Optionally, the selecting a scheduling scheme meeting the comprehensive cost value requirement as the scheduling scheme of the scheduling task includes:
and selecting the scheduling scheme with the minimum comprehensive cost value as the scheduling scheme of the scheduling task.
The embodiment of the invention also provides a cluster multi-node scheduling device, which comprises an acquisition unit, a determination unit, a calculation unit and a selection unit;
the acquiring unit is used for acquiring a scheduling task;
the determining unit is configured to determine a cost value of each node in the cluster resource pool according to a preset node cost table and a service requirement of the scheduling task on each node;
the computing unit is used for traversing the cluster resource pool by using a depth-first algorithm, determining scheduling nodes required by the scheduling task, and computing the comprehensive cost value of the current scheduling scheme according to the cost value of each scheduling node;
and the selecting unit is used for selecting a scheduling scheme meeting the requirement of the comprehensive cost value as the scheduling scheme of the scheduling task when all the scheduling nodes required by the scheduling task are obtained through traversal.
Optionally, the determining unit includes a communication requirement value determining subunit, a communication surrogate value determining subunit, and a performance cost value determining subunit;
the communication requirement value determining subunit is used for determining a communication requirement value among the nodes according to the communication frequency among the nodes in the scheduling task;
the communication proxy value determining subunit is used for multiplying the communication grade value between each node in a preset node cost table and the corresponding communication requirement value between the nodes to obtain the communication proxy value between each node;
and the performance cost value determining subunit is used for determining the performance cost value of each node according to the functional requirement of the scheduling task on the node.
Optionally, the calculating unit is specifically configured to calculate a comprehensive cost value C of the current scheduling scheme according to the following formula,
wherein, r [ i, m [ i ] ] represents the performance cost value of the ith node in the scheduling cluster resource pool of the ith node of the scheduling task, t [ i, j ] represents the communication requirement value of the ith node and the jth node of the scheduling task, s [ m [ i ], m [ j ] represents the communication grade value of the mth node and the mth node in the cluster resource pool, and k represents the number of nodes contained in the current scheduling scheme.
Optionally, a discarding unit is further included;
the discarding unit is configured to discard the first scheduling scheme when the comprehensive cost value of the first scheduling scheme is greater than the comprehensive cost value of the second scheduling scheme after the comprehensive cost value of the current scheduling scheme is calculated according to the cost value of each scheduling node; the first scheduling scheme is a current scheduling scheme of all scheduling nodes required by the scheduling task which is not obtained through traversal; the second scheduling scheme is a scheduling scheme of all scheduling nodes required by the scheduling task after traversal is obtained.
Optionally, the performance cost value determining subunit is specifically configured to obtain a functional requirement of the scheduling task on a node; and searching performance cost values corresponding to different functions of each node in a preset node cost table according to the function requirement, and determining the performance cost value of each node corresponding to the function requirement.
Optionally, the selecting unit is specifically configured to select a scheduling scheme with the smallest comprehensive cost value as the scheduling scheme of the scheduling task.
The embodiment of the present invention further provides a cluster multi-node scheduling apparatus, including:
a memory for storing a computer program;
a processor for executing the computer program to implement the steps of the cluster multi-node scheduling method as described in any one of the above.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the cluster multi-node scheduling method according to any of the above-mentioned embodiments are implemented.
According to the technical scheme, the scheduling task is obtained; and determining the cost value of each node in the cluster resource pool according to a preset node cost table and the service requirement of the scheduling task on each node. By calculating the cost value, the influence of different factors on the scheduling task is unified and quantified. The larger the cost value of the node is, the poorer the service capability when the node is called to execute the scheduling task is. And traversing the cluster resource pool by using a depth-first algorithm, determining scheduling nodes required by the scheduling task, and calculating the comprehensive cost value of the current scheduling scheme according to the cost value of each scheduling node. The scheduling scheme comprises nodes used for executing the scheduling tasks, and the comprehensive cost value represents the cost value of executing the scheduling tasks according to the nodes contained in the scheduling scheme. And when all scheduling nodes required by the scheduling task are obtained through traversal, selecting a scheduling scheme meeting the requirement of the comprehensive cost value as the scheduling scheme of the scheduling task. In the technical scheme, the influence of different factors on the scheduling task is skillfully integrated by using a cost value calculation mode, so that a more reasonable and efficient scheduling scheme can be selected.
Drawings
In order to illustrate the embodiments of the present invention more clearly, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings can be obtained by those skilled in the art without inventive effort.
Fig. 1 is a flowchart of a cluster multi-node scheduling method according to an embodiment of the present invention;
fig. 2 is a flowchart of a method for determining a cost value of a node according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a cluster multi-node scheduling apparatus according to an embodiment of the present invention;
fig. 4 is a schematic hardware structure diagram of a cluster multi-node scheduling device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without any creative work belong to the protection scope of the present invention.
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments.
Next, a cluster multi-node scheduling method provided by the embodiment of the present invention is described in detail. Fig. 1 is a flowchart of a cluster multi-node scheduling method provided in an embodiment of the present invention, where the method includes:
s101: and acquiring a scheduling task.
The scheduling task includes the number of nodes to be scheduled and the service requirement of the nodes.
S102: and determining the cost value of each node in the cluster resource pool according to a preset node cost table and the service requirement of the scheduling task on each node.
After the system receives the scheduling task, the system needs to call the nodes in the cluster resource pool to execute the scheduling task.
Different scheduling tasks are used to implement different functions, and corresponding service requirements for the nodes may also differ. For example, when data transmission between nodes needs to be realized, the requirement on communication between nodes is high; when the nodes are required to analyze and process data, the performance requirements on the nodes are high.
The business requirements represent the influence factors of the scheduling tasks, the influence factors of one scheduling task are various, and in order to realize the comprehensive evaluation of different influence factors, a unified quantitative processing mode is adopted in the embodiment of the invention to convert the business requirements into the cost values.
In practical application, a node cost table may be pre-established, where the node cost table may include grades of different service performances of a node, and each grade has a corresponding cost value. According to the service requirements of the scheduling task on each node, the cost value of the node for executing the scheduling task can be determined in a table look-up mode.
The smaller the cost value of the node is, the better the service capability of the node for executing the scheduling task is.
S103: and traversing the cluster resource pool by using a depth-first algorithm, determining scheduling nodes required by the scheduling task, and calculating the comprehensive cost value of the current scheduling scheme according to the cost value of each scheduling node.
Multiple node implementations are often required to be invoked for a scheduled task. The nodes in the cluster resource pool are called and the calling sequence of the nodes is various, and correspondingly, the scheduling scheme corresponding to one scheduling task is various.
To facilitate differentiation from nodes in the cluster resource pool, the nodes scheduled in the scheduling scheme may be referred to as scheduling nodes.
In specific implementation, a depth-first algorithm may be used to traverse the cluster resource pool, determine a complete scheduling scheme, and then calculate the comprehensive cost value of each scheduling scheme according to the cost value of the scheduling node in each scheduling scheme. Under the processing mode, the current scheduling scheme is a complete scheduling scheme containing all scheduling nodes required by the scheduling task.
Except for the mode of firstly determining the complete scheduling scheme and then calculating the comprehensive cost value of each scheduling scheme, the depth-first algorithm can also be utilized to traverse the cluster resource pool, and the comprehensive cost values of all the currently traversed scheduling nodes are calculated while traversing.
Taking the edge traversal edge calculation as an example, when the p-th node is traversed, the comprehensive cost value of the current scheduling scheme is the sum of the cost value of the p-th node and the comprehensive cost value of the previous p-1 nodes. And when all scheduling nodes required by the scheduling task are obtained through traversal, the scheduling scheme at the moment is the complete scheduling scheme.
S104: and when all scheduling nodes required by the scheduling task are obtained through traversal, selecting a scheduling scheme meeting the requirement of the comprehensive cost value as the scheduling scheme of the scheduling task.
The requirement of the comprehensive cost value can be that a scheduling scheme with the comprehensive cost value larger than a preset threshold value is selected, or a scheduling scheme with the minimum comprehensive cost value is selected as a scheduling scheme for scheduling tasks.
In S102, it is mentioned that the service requirement of each node is converted into the cost value of each node. In consideration of practical application, factors influencing scheduling tasks mainly include communication quality between nodes and processing performance of the nodes.
Taking the communication quality between the nodes and the processing performance of the nodes as examples, in the embodiment of the present invention, the communication quality between the nodes can be represented by the communication cost value, and the processing performance of the nodes can be represented by the performance cost value. Fig. 2 is a flowchart of a method for determining a cost value of a node according to an embodiment of the present invention, where the method includes:
s201: and determining a communication requirement value between the nodes according to the communication frequency between the nodes in the scheduling task.
The scheduling task describes the communication frequency between the nodes, and when communication between two nodes is not required, the communication frequency between the two nodes is 0.
In order to achieve quantitative evaluation of the communication frequency, a correspondence relationship between each communication frequency range and the communication requirement value may be constructed in advance, and the higher the communication frequency between the nodes is, the higher the corresponding communication requirement value is. When a scheduling task is subsequently received, according to the communication frequency among the nodes in the scheduling task, the communication requirement value corresponding to the communication frequency can be selected by searching the corresponding relation.
S202: and multiplying the communication grade value between the nodes in a preset node cost table and the corresponding communication requirement value between the nodes to obtain the communication proxy value between the nodes.
And the node cost table records the communication grade value among the nodes in the cluster resource pool. The higher the communication rank value is, the lower the communication efficiency between the nodes is. In the embodiment of the invention, the communication test can be carried out between any two nodes in the cluster resource pool, so that the communication grade value between the nodes is determined.
For example, the time taken to establish a communication connection between two nodes, the rate of data transmission between two nodes, etc. may be tested, and the shorter the time taken and the faster the rate of data transmission, the better the communication efficiency between two nodes is, and a lower communication level value may be set.
In the embodiment of the present invention, the specific value of the communication rank value is not limited, and may be set according to actual requirements.
The communication grade value and the communication requirement value between the nodes are important factors influencing the communication performance between the nodes, and in the embodiment of the invention, the communication grade value and the communication requirement value between the same two nodes are multiplied to be used as the communication proxy value between the two nodes. By analogy, the communication cost value among the nodes can be calculated.
S203: and determining the performance cost value of each node according to the functional requirements of the scheduling task on the nodes.
In the embodiment of the present invention, performance cost values may be set in advance for performance performances of each node in the cluster resource pool in different functions. Setting a lower performance cost value if the performance is better; and if the performance is poorer, setting higher performance cost value. In order to facilitate subsequent query, performance cost values corresponding to different functions of each node in the cluster resource pool may be recorded in the node cost table.
The functional requirements for the nodes in the scheduling task comprise which function of the nodes needs to be called, and according to the functions needing to be called, the performance cost values of the nodes corresponding to the functional requirements are determined by searching the performance cost values corresponding to different functions of the nodes in the node cost table.
Fig. 2 is an illustration of an example where a service requirement includes inter-node communication factors and node performance factors, and in the embodiment of the present invention, the type of the factors included in the service requirement is not specifically limited, and other types of factors may be included besides the communication factors and the performance factors.
In the above introduction it is mentioned that the integrated cost value of the current scheduling scheme can be computed with edge traversal. Taking cost values including the communication cost values among the nodes and the performance cost values of the nodes as an example, the comprehensive cost value C of the current scheduling scheme can be calculated according to the following formula,
wherein, r [ i, m [ i ] ] represents the performance cost value of the ith node in the scheduling cluster resource pool of the ith node of the scheduling task, t [ i, j ] represents the communication requirement value of the ith node and the jth node of the scheduling task, s [ m [ i ], m [ j ] represents the communication grade value of the mth node and the mth node in the cluster resource pool, and k represents the number of nodes contained in the current scheduling scheme.
For example, suppose that the 3 rd node in the scheduling task needs to be allocated to 1 node in the resource pool and the 4 th node needs to be allocated to 2 nodes in the resource pool, in this case, s [1,2] is 2, t [3,4] is 3, t [4,3] is 0, r [3,1] is 5, and r [4,2] is 1. Then with respect to the cost value between the two nodes: the communication cost value is s [1,2] t [3,4] + s [2,1] t [4,3] ═ 6, and the performance cost value is r [3,1] + r [4,2] ═ 6.
According to the technical scheme, the scheduling task is obtained; and determining the cost value of each node in the cluster resource pool according to a preset node cost table and the service requirement of the scheduling task on each node. By calculating the cost value, the influence of different factors on the scheduling task is unified and quantified. The larger the cost value of the node is, the poorer the service capability when the node is called to execute the scheduling task is. And traversing the cluster resource pool by using a depth-first algorithm, determining scheduling nodes required by the scheduling task, and calculating the comprehensive cost value of the current scheduling scheme according to the cost value of each scheduling node. The scheduling scheme comprises nodes used for executing the scheduling tasks, and the comprehensive cost value represents the cost value of executing the scheduling tasks according to the nodes contained in the scheduling scheme. And when all scheduling nodes required by the scheduling task are obtained through traversal, selecting a scheduling scheme meeting the requirement of the comprehensive cost value as the scheduling scheme of the scheduling task. In the technical scheme, the influence of different factors on the scheduling task is skillfully integrated by using a cost value calculation mode, so that a more reasonable and efficient scheduling scheme can be selected.
In the embodiment of the invention, the comprehensive cost value of the current scheduling scheme can be calculated while traversing. Because the completion degrees of the scheduling schemes are different, some scheduling schemes are already traversed and completed, and some scheduling schemes may be in a traversal stage, for convenience of distinguishing, the current scheduling scheme of all the scheduling nodes required by the scheduling task which is not traversed can be called a first scheduling scheme, and the scheduling scheme of all the scheduling nodes required by the scheduling task which is traversed can be called a second scheduling scheme.
In a specific implementation, when the comprehensive cost value of the first scheduling scheme is greater than the comprehensive cost value of the second scheduling scheme, it is indicated that the performance of the first scheduling scheme is inferior to that of the second scheduling scheme, and at this time, the first scheduling scheme may be discarded without performing subsequent traversal operation on the first scheduling scheme.
By comparing the comprehensive cost values of the scheduling schemes which are not traversed and are completed with the scheduling schemes which are traversed, the scheduling schemes which are not traversed and are poor in service performance can be found in time, the scheduling schemes which are poor in service performance are abandoned, the calculation amount of the system is reduced, and the processing efficiency of selecting the scheduling schemes is improved.
Fig. 3 is a schematic structural diagram of a cluster multi-node scheduling apparatus according to an embodiment of the present invention, including an obtaining unit 31, a determining unit 32, a calculating unit 33, and a selecting unit 34;
an obtaining unit 31, configured to obtain a scheduling task;
a determining unit 32, configured to determine a cost value of each node in the cluster resource pool according to a preset node cost table and a service requirement of the scheduling task on each node;
the computing unit 33 is configured to traverse the cluster resource pool by using a depth-first algorithm, determine scheduling nodes required by the scheduling task, and compute a comprehensive cost value of the current scheduling scheme according to the cost value of each scheduling node;
and the selecting unit 34 is configured to select, as the scheduling scheme for scheduling the task, the scheduling scheme that meets the requirement of the comprehensive cost value when all the scheduling nodes required by the scheduling task are obtained by traversal.
Optionally, the determining unit includes a communication requirement value determining subunit, a communication surrogate value determining subunit, and a performance cost value determining subunit;
the communication requirement value determining subunit is used for determining the communication requirement value among the nodes according to the communication frequency among the nodes in the scheduling task;
the communication proxy value determining subunit is used for multiplying the communication grade value between each node in a preset node cost table and the corresponding communication requirement value between the nodes to obtain the communication proxy value between each node;
and the performance cost value determining subunit is used for determining the performance cost value of each node according to the functional requirements of the scheduling task on the nodes.
Optionally, the calculating unit is specifically configured to calculate a comprehensive cost value C of the current scheduling scheme according to the following formula,
wherein, r [ i, m [ i ] ] represents the performance cost value of the ith node in the scheduling cluster resource pool of the ith node of the scheduling task, t [ i, j ] represents the communication requirement value of the ith node and the jth node of the scheduling task, s [ m [ i ], m [ j ] represents the communication grade value of the mth node and the mth node in the cluster resource pool, and k represents the number of nodes contained in the current scheduling scheme.
Optionally, a discarding unit is further included;
the abandoning unit is used for abandoning the first scheduling scheme when the comprehensive cost value of the first scheduling scheme is larger than the comprehensive cost value of the second scheduling scheme after the comprehensive cost value of the current scheduling scheme is calculated according to the cost value of each scheduling node; the first scheduling scheme is a current scheduling scheme of all scheduling nodes required by the scheduling task which is not obtained through traversal; the second scheduling scheme is a scheduling scheme that has traversed all scheduling nodes needed to get the scheduling task.
Optionally, the performance cost value determining subunit is specifically configured to obtain a functional requirement of the scheduling task on the node; and searching performance cost values corresponding to different functions of each node in a preset node cost table according to the function requirement, and determining the performance cost value of each node corresponding to the function requirement.
Optionally, the selecting unit is specifically configured to select a scheduling scheme with the smallest comprehensive cost value as the scheduling scheme for scheduling the task.
The description of the features in the embodiment corresponding to fig. 3 may refer to the related description of the embodiment corresponding to fig. 1, and is not repeated here.
According to the technical scheme, the scheduling task is obtained; and determining the cost value of each node in the cluster resource pool according to a preset node cost table and the service requirement of the scheduling task on each node. By calculating the cost value, the influence of different factors on the scheduling task is unified and quantified. The larger the cost value of the node is, the poorer the service capability when the node is called to execute the scheduling task is. And traversing the cluster resource pool by using a depth-first algorithm, determining scheduling nodes required by the scheduling task, and calculating the comprehensive cost value of the current scheduling scheme according to the cost value of each scheduling node. The scheduling scheme comprises nodes used for executing the scheduling tasks, and the comprehensive cost value represents the cost value of executing the scheduling tasks according to the nodes contained in the scheduling scheme. And when all scheduling nodes required by the scheduling task are obtained through traversal, selecting a scheduling scheme meeting the requirement of the comprehensive cost value as the scheduling scheme of the scheduling task. In the technical scheme, the influence of different factors on the scheduling task is skillfully integrated by using a cost value calculation mode, so that a more reasonable and efficient scheduling scheme can be selected.
Fig. 4 is a schematic hardware structure diagram of a cluster multi-node scheduling apparatus 40 according to an embodiment of the present invention, including:
a memory 41 for storing a computer program;
a processor 42 for executing a computer program for implementing the steps of any one of the above-described clustered multi-node scheduling methods.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned cluster multi-node scheduling method are implemented.
The above details describe a cluster multi-node scheduling method, apparatus, and computer-readable storage medium provided by the embodiments of the present invention. The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Claims (8)
1. A cluster multi-node scheduling method is characterized by comprising the following steps:
acquiring a scheduling task;
determining a cost value of each node in a cluster resource pool according to a preset node cost table and service requirements of the scheduling task on each node;
traversing the cluster resource pool by using a depth-first algorithm, determining scheduling nodes required by the scheduling task, and calculating the comprehensive cost value of the current scheduling scheme according to the cost value of each scheduling node;
when all scheduling nodes required by the scheduling task are obtained through traversal, selecting a scheduling scheme meeting the requirement of the comprehensive cost value as a scheduling scheme of the scheduling task;
the determining the cost value of each node in the cluster resource pool according to the preset node cost table and the service requirement of the scheduling task on each node comprises:
determining a communication requirement value between nodes according to the communication frequency between the nodes in the scheduling task;
multiplying the communication grade value between each node in a preset node cost table by the corresponding communication requirement value between the nodes to obtain the communication proxy value between each node;
and determining the performance cost value of each node according to the functional requirements of the scheduling task on the nodes.
2. The method of claim 1, wherein calculating the composite cost value of the current scheduling scheme according to the cost value of each scheduling node comprises:
according to the following formula, calculating the comprehensive cost value C of the current scheduling scheme,
wherein, r [ i, m [ i ] ] represents the performance cost value of the ith node in the scheduling cluster resource pool of the ith node of the scheduling task, t [ i, j ] represents the communication requirement value of the ith node and the jth node of the scheduling task, s [ m [ i ], m [ j ] represents the communication grade value of the mth node and the mth node in the cluster resource pool, and k represents the number of nodes contained in the current scheduling scheme.
3. The method according to claim 2, further comprising, after said calculating the composite cost value of the current scheduling scheme according to the cost value of each scheduling node:
when the comprehensive cost value of the first scheduling scheme is larger than the comprehensive cost value of the second scheduling scheme, abandoning the first scheduling scheme; the first scheduling scheme is a current scheduling scheme of all scheduling nodes required by the scheduling task which is not obtained through traversal; the second scheduling scheme is a scheduling scheme of all scheduling nodes required by the scheduling task after traversal is obtained.
4. The method of claim 1, wherein determining a performance cost value for each node according to the functional requirements of the scheduling task for the node comprises:
acquiring the functional requirements of the scheduling task on the nodes;
and searching performance cost values corresponding to different functions of each node in a preset node cost table according to the function requirement, and determining the performance cost value of each node corresponding to the function requirement.
5. The method according to any one of claims 1 to 4, wherein the selecting a scheduling scheme satisfying a composite cost value requirement as the scheduling scheme of the scheduling task comprises:
and selecting the scheduling scheme with the minimum comprehensive cost value as the scheduling scheme of the scheduling task.
6. A cluster multi-node scheduling device is characterized by comprising an acquisition unit, a determination unit, a calculation unit and a selection unit;
the acquiring unit is used for acquiring a scheduling task;
the determining unit is configured to determine a cost value of each node in the cluster resource pool according to a preset node cost table and a service requirement of the scheduling task on each node;
the computing unit is used for traversing the cluster resource pool by using a depth-first algorithm, determining scheduling nodes required by the scheduling task, and computing the comprehensive cost value of the current scheduling scheme according to the cost value of each scheduling node;
the selecting unit is used for selecting a scheduling scheme meeting the requirement of the comprehensive cost value as the scheduling scheme of the scheduling task when all scheduling nodes required by the scheduling task are obtained through traversal;
the determining unit comprises a communication requirement value determining subunit, a communication surrogate value determining subunit and a performance cost value determining subunit;
the communication requirement value determining subunit is used for determining a communication requirement value among the nodes according to the communication frequency among the nodes in the scheduling task;
the communication proxy value determining subunit is used for multiplying the communication grade value between each node in a preset node cost table and the corresponding communication requirement value between the nodes to obtain the communication proxy value between each node;
and the performance cost value determining subunit is used for determining the performance cost value of each node according to the functional requirement of the scheduling task on the node.
7. A cluster multi-node scheduling apparatus, comprising:
a memory for storing a computer program;
a processor for executing said computer program for carrying out the steps of the cluster multi-node scheduling method according to any of claims 1 to 5.
8. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the cluster multi-node scheduling method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910760105.XA CN110365799B (en) | 2019-08-16 | 2019-08-16 | Cluster multi-node scheduling method and device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910760105.XA CN110365799B (en) | 2019-08-16 | 2019-08-16 | Cluster multi-node scheduling method and device and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110365799A CN110365799A (en) | 2019-10-22 |
| CN110365799B true CN110365799B (en) | 2022-04-22 |
Family
ID=68224884
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910760105.XA Active CN110365799B (en) | 2019-08-16 | 2019-08-16 | Cluster multi-node scheduling method and device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110365799B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111309479B (en) * | 2020-02-14 | 2023-06-06 | 北京百度网讯科技有限公司 | Method, device, equipment and medium for realizing task parallel processing |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109002359A (en) * | 2018-09-03 | 2018-12-14 | 郑州云海信息技术有限公司 | A kind of dispatching method and relevant apparatus of calculate node server |
| CN110071831A (en) * | 2019-04-17 | 2019-07-30 | 电子科技大学 | Node selecting method based on network cost |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140096256A1 (en) * | 2011-11-01 | 2014-04-03 | University Of Washington Through Its Center For Commercialization | Joint performance-vulnerability metric framework for designing ad hoc routing protocols |
| KR20130088512A (en) * | 2012-01-31 | 2013-08-08 | 한국전자통신연구원 | Apparatus and method for managing resource in clustered computing environment |
-
2019
- 2019-08-16 CN CN201910760105.XA patent/CN110365799B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109002359A (en) * | 2018-09-03 | 2018-12-14 | 郑州云海信息技术有限公司 | A kind of dispatching method and relevant apparatus of calculate node server |
| CN110071831A (en) * | 2019-04-17 | 2019-07-30 | 电子科技大学 | Node selecting method based on network cost |
Non-Patent Citations (1)
| Title |
|---|
| Accelerated Diffusion-Based Recommendation;Mi Li;《2016 17th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD)》;20160601;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110365799A (en) | 2019-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103699445B (en) | Task scheduling method, device and system | |
| CN109981744B (en) | Data distribution method and device, storage medium and electronic equipment | |
| CN108270805B (en) | Resource allocation method and device for data processing | |
| CN110347515B (en) | Resource optimization allocation method suitable for edge computing environment | |
| CN107220108B (en) | Method and system for realizing load balance of cloud data center | |
| CN110365765A (en) | A kind of bandwidth scheduling method and device of cache server | |
| CN103699433B (en) | One kind dynamically adjusts number of tasks purpose method and system in Hadoop platform | |
| CN110502321A (en) | A kind of resource regulating method and system | |
| CN112020098B (en) | Load balancing method, device, computing equipment and computer storage medium | |
| CN113746798B (en) | Cloud network shared resource abnormal root cause positioning method based on multi-dimensional analysis | |
| CN112925637A (en) | Load balancing device and method for edge operation network | |
| CN112261120A (en) | Cloud-side cooperative task unloading method and device for power distribution internet of things | |
| CN105528283B (en) | A kind of method that load value is calculated in mobile application detection load-balancing algorithm | |
| CN104102646B (en) | The method, apparatus and system of data processing | |
| CN110365799B (en) | Cluster multi-node scheduling method and device and computer readable storage medium | |
| CN107566535B (en) | Self-adaptive load balancing method based on concurrent access timing sequence rule of Web map service | |
| CN108683605A (en) | A kind of current-limiting method and device of service call | |
| CN110167031B (en) | Resource allocation method, equipment and storage medium for centralized base station | |
| Tong et al. | A holistic auto-scaling algorithm for multi-service applications based on balanced queuing network | |
| CN110995856B (en) | Method, device and equipment for server expansion and storage medium | |
| Han et al. | An adaptive scheduling algorithm for heterogeneous Hadoop systems | |
| CN111124439A (en) | Intelligent dynamic unloading algorithm with cloud edge cooperation | |
| CN113850346B (en) | Edge service secondary clustering method and system for multi-dimensional attribute perception in MEC environment | |
| CN110113269A (en) | A kind of method and relevant apparatus of the flow control based on middleware | |
| CN110502495A (en) | A kind of log collecting method and device of application server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |