CN117149445B - Cross-cluster load balancing method and device, equipment and storage medium - Google Patents
Cross-cluster load balancing method and device, equipment and storage medium Download PDFInfo
- Publication number
- CN117149445B CN117149445B CN202311422616.3A CN202311422616A CN117149445B CN 117149445 B CN117149445 B CN 117149445B CN 202311422616 A CN202311422616 A CN 202311422616A CN 117149445 B CN117149445 B CN 117149445B
- Authority
- CN
- China
- Prior art keywords
- node
- data request
- cluster
- current data
- cpu
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 230000004044 response Effects 0.000 claims abstract description 39
- 238000012545 processing Methods 0.000 claims abstract description 22
- 230000006870 function Effects 0.000 claims description 37
- 238000004891 communication Methods 0.000 claims description 19
- 102100033121 Transcription factor 21 Human genes 0.000 claims description 16
- 101710119687 Transcription factor 21 Proteins 0.000 claims description 16
- 238000004364 calculation method Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 7
- 238000012216 screening Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 description 6
- 239000003795 chemical substances by application Substances 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 3
- 238000013500 data storage Methods 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000010076 replication Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Computer And Data Communications (AREA)
Abstract
The disclosure relates to a method, a device, equipment and a storage medium for cross-cluster load balancing, wherein the method comprises the following steps: determining service information corresponding to the data request according to the received data request based on the known correspondence between the data request and the service information; acquiring the resource requirement of response data corresponding to the current data request according to the service information; calculating the running state and the load condition of each node on each cluster; according to the running state and the load condition of each node on each cluster and the resource requirement of response data corresponding to the current data request, matching target nodes for the current data request from all nodes of all clusters, and sending task processing requests corresponding to the current data request to the target nodes, so that the requirement of a user on resource management of multiple clusters in a large-scale scene and application of multiple cluster scenes can be met.
Description
Technical Field
The disclosure relates to the technical field of cloud computing, and in particular relates to a cross-cluster load balancing method, a device, equipment and a storage medium.
Background
In the early stage of cloud platform construction, a single cluster can basically meet business requirements of enterprises or governments, but with the continuous increase of the requirements of cloud computing, when a system and business fail and disaster occur, the problems of limited bearing capacity, limited disaster recovery capacity and the like of the single cluster are increasingly prominent, the increasingly-increased requirements cannot be met, and naturally, multi-cluster arrangement becomes a preferred solution. However, in the practical application scenario of multiple clusters, because the resources and the configuration of each cluster are different, the allocation and management of the resources are complex, and the requirement of the multiple clusters for sharing the load is always an important and complex problem that needs to be solved by the software system. How to manage resources for multiple clusters to meet the demands of users in large-scale scenes and multiple cluster scenes is a problem to be solved.
In the related art, only the resource scheduling for a certain multi-node cluster is supported, and the resource scheduling requirement between the multi-clusters cannot be realized in a multi-cluster application scene. For resource scheduling of a certain multi-node cluster, a unified scheduling load distribution technology is generally adopted. The system defines one or more scheduling nodes, the scheduling nodes collect the running states and load conditions of all the computing service nodes, perform calculation analysis in real time according to a preset scheduling strategy, and then distribute calculation tasks to all the computing nodes.
Disclosure of Invention
In order to solve the above technical problems or at least partially solve the above technical problems, embodiments of the present disclosure provide a cross-cluster load balancing method, a platform, a device, and a storage medium.
In a first aspect, embodiments of the present disclosure provide a method for cross-cluster load balancing, applied to a plurality of different clusters of a network, the method including:
determining service information corresponding to the data request according to the received data request based on the known correspondence between the data request and the service information;
acquiring the resource requirement of response data corresponding to the current data request according to the service information;
calculating the running state and the load condition of each node on each cluster;
and matching target nodes for the current data request from all nodes of all clusters according to the running state and the load condition of each node on each cluster and the resource requirement of response data corresponding to the current data request, and sending task processing requests corresponding to the current data request to the target nodes.
In one possible embodiment, the known correspondence between data requests and service information is obtained by:
determining Pod data responsive to the data request;
and taking the service information of the node deployed with the Pod data as the service information corresponding to the data request.
In a possible implementation manner, the obtaining, according to the service information, the resource requirement of the response data corresponding to the current data request includes:
and inquiring the node where the service information is located, and acquiring the resource requirement of the node for responding to the Pod data of the current data request as the resource requirement of the response data corresponding to the current data request.
In one possible implementation manner, the calculating the operation state and the load condition of each node on each cluster includes:
for each cluster data, splitting raster data with the same number as the cluster nodes by taking the nodes as units, so that each node data corresponds to one raster data;
and carrying out real-time calculation on the raster data corresponding to each node, and taking the calculation result as the running state and the load condition of each node.
In one possible implementation manner, the matching the target node for the current data request from all nodes of all clusters according to the running state and the load condition of each node on each cluster and the resource requirement of the response data corresponding to the current data request includes:
according to the running state and load condition of each node on each cluster and the resource requirement of response data corresponding to the current data request, screening out nodes meeting preset conditions from all nodes;
sorting the screened nodes according to the order of the priority from big to small;
and taking the first node in the ordered nodes as a target node.
In one possible embodiment, the priority is calculated by:
acquiring the node resource ratio, the node CPU and memory ratio of each node and the node affinity and stain mark matching degree as priority computing parameters;
according to each priority computing parameter and the corresponding preset weight, computing the priority of the current node,
the priority of the current node is calculated by the following expression:
FSN= (w1NRPP1) + (w2/>NRPP2) + (w3 /> NAP) + (w4 /> TTP)
NRPP1= ( (c cpu - s Pod1 ) 10 / c cpu + (c mem - s Pod2 ) /> 10 / c mem ) / 2
NRPP2 = 10 - | ( (c cpu - s Pod1 ) 10 / c cpu )/ ( (c mem - s Pod2 ) /> 10 / c mem )-1| />10
NAP = [(S/T)10]
TTP=( (TM 1 -TM 2 ) / TM 1 ) 10
wherein TTP is the score of node taint priority function, TM 1 For the number of all stain entries, TM 2 NAP is the score of the node affinity priority function under the condition that the node adopts a soft strategy, NAP is 10 under the condition that the node adopts a hard strategy, S is the number meeting the requirements of preset rules, T is the number meeting the requirements of all preset rules, and NRPP2 is the node resource proportioning priorityScore of function NRPP2, c cpu Is the total capacity of the node CPU, s Pod1 For the sum of CPU capacities of the corresponding Pod of the current data request and other pods on the node, c mem Total capacity of node memory s Pod2 For the sum of the memory capacities of the corresponding Pod of the current data request and other pods on the node, NRPP1 is the score of the node resource duty ratio priority function, c cpu Is the total capacity of the node CPU, s Pod1 For the sum of CPU capacities of the corresponding Pod of the current data request and other pods on the node, c mem Total capacity of node memory s Pod2 And summing the memory capacities of the corresponding Pod and other pods on the node for the current data request.
In one possible implementation manner, the sending the task processing request corresponding to the current data request to the target node includes:
inquiring the node where the service information is located as a first node;
determining whether the first node and the target node are in the same cluster;
under the condition that the first node and the target node are in the same cluster, directly sending a task processing request corresponding to the current data request to a service instance of the target node, and returning corresponding response data to a request end;
and under the condition that the first node and the target node are not in the same cluster, establishing a channel for the cluster where the first node is located and the cluster where the target node is located, calling a service instance of the first node, and submitting a Yaml file corresponding to the service instance of the first node to the target node.
In a second aspect, embodiments of the present disclosure provide a cross-cluster load balancing apparatus applied to a plurality of different clusters of a network, the apparatus comprising:
the determining module is used for determining service information corresponding to the data request according to the received data request based on the known corresponding relation between the data request and the service information;
the acquisition module is used for acquiring the resource requirement of the response data corresponding to the current data request according to the service information;
the computing module is used for computing the running state and the load condition of each node on each cluster;
the matching module is used for matching target nodes for the current data request from all nodes of all clusters according to the running state and the load condition of each node on each cluster and the resource requirement of response data corresponding to the current data request, and sending task processing requests corresponding to the current data request to the target nodes.
In a third aspect, embodiments of the present disclosure provide an electronic device including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus;
a memory for storing a computer program;
and the processor is used for realizing the cross-cluster load balancing method when executing the programs stored in the memory.
In a fourth aspect, embodiments of the present disclosure provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the cross-cluster load balancing method described above.
Compared with the prior art, the technical scheme provided by the embodiment of the disclosure has at least part or all of the following advantages:
the cross-cluster load balancing method is applied to a plurality of clusters with different networks, and based on the known corresponding relation between the data request and the service information, the service information corresponding to the data request is determined according to the received data request; acquiring the resource requirement of response data corresponding to the current data request according to the service information; calculating the running state and the load condition of each node on each cluster; according to the running state and the load condition of each node on each cluster and the resource requirement of response data corresponding to the current data request, matching target nodes for the current data request from all nodes of all clusters, and sending task processing requests corresponding to the current data request to the target nodes, so that the requirement of a user on resource management of multiple clusters in a large-scale scene and application of multiple cluster scenes can be met.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings that are required to be used in the description of the embodiments or the related art will be briefly described below, and it will be apparent to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.
FIG. 1 schematically illustrates a flow diagram of a cross-cluster load balancing method according to an embodiment of the disclosure;
FIG. 2 schematically illustrates an interaction flow diagram between a central cluster, cluster A, and cluster B according to an embodiment of the present disclosure;
FIG. 3 schematically illustrates a block diagram of a cross-cluster load balancing apparatus in accordance with an embodiment of the disclosure;
fig. 4 schematically shows a block diagram of an electronic device according to an embodiment of the disclosure.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure, and it is apparent that the described embodiments are some, but not all, embodiments of the present disclosure. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the disclosure, are within the scope of the disclosure.
Referring to fig. 1, an embodiment of the present disclosure provides a cross-cluster load balancing method applied to a plurality of different clusters of a network, the method including:
s1, based on the known corresponding relation between the data request and the service information, the service information corresponding to the data request is determined according to the received data request.
S2, acquiring the resource requirement of response data corresponding to the current data request according to the service information.
S3, calculating the running state and the load condition of each node on each cluster.
S4, matching target nodes for the current data request from all nodes of all clusters according to the running state and the load condition of each node on each cluster and the resource requirement of response data corresponding to the current data request, and sending task processing requests corresponding to the current data request to the target nodes.
In this embodiment, in step S1, the known correspondence between the data request and the service information is obtained by:
determining Pod data responsive to the data request;
and taking the service information of the node deployed with the Pod data as the service information corresponding to the data request.
In an actual application scene, after each request is initiated, gateway routing or service information (a plurality of gateway routing or service information) of the request response in the history record and the request are maintained together to form a relation corresponding table, and real-time calculation and update are performed. When a request is initiated, firstly, the gateway route or service information corresponding to the request in a relation corresponding table is queried, the domain name of the gateway route is analyzed, the node where the gateway route or service is located is found, then, according to the running state and the load condition of each node analyzed by real-time calculation, a scheduling strategy is formulated to allocate an optimal target node for the request, and a task processing request is allocated to the optimal target node.
In this embodiment, in step S2, the obtaining, according to the service information, the resource requirement of the response data corresponding to the current data request includes:
and inquiring the node where the service information is located, and acquiring the resource requirement of the node for responding to the Pod data of the current data request as the resource requirement of the response data corresponding to the current data request.
In this embodiment, in step S3, the calculating the running state and the load condition of each node on each cluster includes:
for each cluster data, splitting raster data with the same number as the cluster nodes by taking the nodes as units, so that each node data corresponds to one raster data;
and carrying out real-time calculation on the raster data corresponding to each node, and taking the calculation result as the running state and the load condition of each node.
In this embodiment, taking K8s cluster 1 as an example, calculating the running state and load condition of each node on K8s cluster 1 includes:
calling real-time data of the K8s cluster 1 in the distributed data storage, carrying out rasterization on the real-time data, generating raster data with nodes as units, facilitating subsequent calculation and analysis, and splitting raster data with the same number as the nodes, such as K8s1_1, K8s1_2, K8s1_3 and the like, by using the nodes as units;
and calculating the raster data in real time, and analyzing the running states and the load conditions of all current cluster nodes, wherein the running states and the load conditions comprise the total resources and occupied resources of each node.
In this embodiment, in step S4, according to the running state and the load condition of each node on each cluster and the resource requirement of the response data corresponding to the current data request, the matching target node for the current data request from all the nodes of all the clusters includes:
according to the running state and load condition of each node on each cluster and the resource requirement of response data corresponding to the current data request, screening out nodes meeting preset conditions from all nodes;
sorting the screened nodes according to the order of the priority from big to small;
and taking the first node in the ordered nodes as a target node.
In this embodiment, the priority is calculated by:
acquiring the node resource ratio, the node CPU and memory ratio of each node and the node affinity and stain mark matching degree as priority computing parameters;
and calculating the priority of the current node according to each priority calculation parameter and the corresponding preset weight.
Assuming that there are a plurality of nodes nodeA, there are a plurality of scoring functions, such as priority func1, priority func2 (each method can return one score), each method has a corresponding weight factor weight1, weight2. finalscore nodea= (weight 1) priorityFunc1) + (weight2 />priority func 2) +. Wherein the scoring function comprises:
1. node resource occupation ratio priority function NodeResourceProportionpriority is simplified to NRPP1: preferably, selecting a node with smaller overall resource occupation;
2. node CPU, memory proportion priority function NodeResourceRatioPriority, simplify as NRPP2: preferentially selecting nodes with the CPU and memory utilization rates close to each other and each resource more balanced;
3. node affinity priority function NodeAffinityPriority, simplified to NAP: according to the node affinity ranking, nodes with higher affinity indexes are preferred;
4. the node stain priority function, taintTolerationpriority, is simplified as TTP: preferably, the node with the lowest stain matching rate and highest score is selected. (the more blemish matches, the lower the node score)
Processing each through the preselected nodes with the four priority functions, each priority function returning a score of 0-10, the higher the score the better the node, and each function corresponding to a value representing the weight.
And calculating all the priority function scores and weights through an algorithm to obtain a final result and sequencing. The calculated score for each preferred function is first multiplied by a weight, and then the scores for all preferred functions are added to obtain the final priority score for the node. The weights can be customized, and the ability of the function tendency is optimized, and the scoring formula for calculating the priority is specifically as follows:
1. node resource occupancy priority function NRPP1 (weight factor w 1):
the node resource ratio is the ratio of the node idle resource to the node total capacity, i.e., is determined by (total capacity-sum of capacities of Pod on node-capacity of new Pod)/total capacity). The CPU and the memory have the same weight, and the node score is higher when the resource idle ratio is higher.
NRPP1= ( (c cpu - s Pod1 ) 10 / c cpu + (c mem - s Pod2 ) /> 10 / c mem ) / 2
Wherein NRPP1 is the score of the node resource duty ratio priority function NRPP1, c cpu Is the total capacity of the node CPU, s Pod1 For the sum of CPU capacities of the corresponding Pod of the current data request and other pods on the node, c mem Total capacity of node memory s Pod2 And summing the memory capacities of the corresponding Pod and other pods on the node for the current data request.
2. Node resources (CPU, memory) match priority function NRPP2 (weight factor w 2):
the closer the CPU and the memory use ratio is, the higher the node weight is, and the more balanced machines of all resources after Pod deployment are selected as much as possible.
NRPP2 = 10 - | ( (c cpu - s Pod1 ) 10 / c cpu )/ ( (c mem - s Pod2 ) /> 10 / c mem )-1| />10
Wherein NRPP2 is the score of the node resource (CPU, memory) proportioning priority function NRPP2, c cpu Is the total capacity of the node CPU, s Pod1 For the sum of CPU capacities of the corresponding Pod of the current data request and other pods on the node, c mem Total capacity of node memory s Pod2 And summing the memory capacities of the corresponding Pod and other pods on the node for the current data request.
3. Node affinity priority function NAP (weight factor w 3):
affinity is divided into two strategies:
1) Hard strategy (HardStrategy): the selected host computer needs to meet the rule requirement of Pod on the host computer; (hosts that do not meet the requirements have been excluded during the node pre-selection phase.)
2) Soft policy (SoftStrategy): the selected host needs to meet the rule requirement of pod on the host as much as possible but not guaranteed.
The NodeAffinityPriority function performs node affinity ordering on all nodes in the preference queue according to the Pod corresponding to the current data request. If the rule requirement is completely met, the score is 10 points; and if part of the requirements of the rules are met but the soft strategy is adopted, the scores are obtained according to the meeting ratio of the soft strategy, for example, less than 10% is 0 score, less than 10% is 1 score, less than 20% is 2 score, less than 30% is 30 score, less than 30% is 3 score, and the scores of all the sections are analogized downwards in sequence.
In the case of a soft policy for a node, the score of the node affinity priority function is calculated by the following expression:
NAP = [(S/T)10]
wherein NAP is the score of the node affinity priority function, [ (S/T)10]The score is obtained by rounding the numerical value, such as 2.1 score of 2 score, wherein S is the number meeting the requirement of the preset rule, and T is the preset valueThere are a number of preset rule requirements.
In the case of a node employing a hard policy, nap=10
4. Node taint priority function TTP (weight factor w 4):
matching the tolerated blobs with the node blobs in the corresponding Pod using the current data request, the more items that are successfully paired, the lower the score, i.e., the more blobs are matched, the lower the score (completely unmatched node data has been excluded during the node pre-selection stage).
TTP=( (TM 1 -TM 2 ) / TM 1 ) 10
Wherein TTP is the score of node taint priority function TTP, TM 1 For the number of all stain entries, TM 2 The number of successful spot items for pairing.
The Node priority is divided into: fsn= (w 1)NRPP1) + (w2/>NRPP2) + (w3 /> NAP) + (w4 TTP)
In this embodiment, in step S4, the sending, to the target node, a task processing request corresponding to the current data request includes:
inquiring the node where the service information is located as a first node;
determining whether the first node and the target node are in the same cluster;
under the condition that the first node and the target node are in the same cluster, directly sending a task processing request corresponding to the current data request to a service instance of the target node, and returning corresponding response data to a request end;
and under the condition that the first node and the target node are not in the same cluster, establishing a channel for the cluster where the first node is located and the cluster where the target node is located, calling a service instance of the first node, and submitting a Yaml file corresponding to the service instance of the first node to the target node.
In this embodiment, establishing a channel for a cluster where a first node is located and a cluster where a target node is located, calling a service instance of the first node, and submitting a Yaml file corresponding to the service instance of the first node to the target node includes:
and controlling the cluster where the target node is located and the cluster where the first node is located to initiate an STCP connection establishment channel to communicate, calling a Service interface of a Service instance corresponding to the first node, and submitting the obtained Yaml file corresponding to the Service instance to the cluster where the target node is located through a POST request.
In this embodiment, the communication between the cluster where the control target node is located and the cluster where the first node is located initiates the STCP connection establishment channel, including:
deploying a preset server component in a central cluster;
the method comprises the steps that a preset access component is deployed in a cluster where a target node is located, a preset proxy component is deployed in the cluster where a first node is located, a deployment file of the proxy component comprises a server component service address and a service ID (identity) connected with the proxy component, and the deployment file of the access component comprises the server component service address, the service ID and a mapped virtual proxy service address corresponding to each service ID connected with the access component;
after the deployment of the server component is completed, exposing the TCP service address of the server component to the outside, wherein the server component is used for providing a cross-cluster network connection channel;
after the deployment of the proxy component is completed, registering the service metadata information bound by the proxy component into a metadata base of the server component through a server component service address in a deployment file of the proxy component, wherein the service metadata information comprises a service ID;
the server component receives the access request of the access component, and based on the metadata base, a channel between the access component and the proxy component corresponding to the same service ID is constructed through the server component deployed in the central cluster, so that data transmission among clusters with different networks is realized.
In this embodiment, the server component receives an access request of an access component, constructs a channel between the access component and an agent component corresponding to the same service ID through a server component deployed in a central cluster based on a metadata base, and realizes data transmission between clusters of different networks, including:
after the deployment of the access component is completed, receiving a data query request of business service bound with the access component, establishing an access channel with the server component by using an STCP protocol according to a server component service address in an access component deployment file, and sending the access request to the server component through the access channel;
the server component checks whether a service ID in the access request exists in the metadata base, and if the service ID in the access request exists in the metadata base, checks whether a service key in the access request corresponds to the service ID in the metadata base, and if the service key in the access request corresponds to the service ID, agents an access channel to an agent component address corresponding to the service ID;
the agent component accesses the service corresponding to the service ID and returns the service data returned by the service to the access component through the access channel.
In this embodiment, the proxy component address that proxies the access channel to the service ID corresponds to the service ID includes:
and according to the proxy component address corresponding to the service ID, querying an address mapping relation comparison table in a known data storage library in the proxy component corresponding to the service ID, and converting the virtual proxy service address in the access request into the service address bound by the proxy component so as to access the service corresponding to the service ID through the proxy component.
In this embodiment, the method further includes:
controlling the cluster to generate the same instance according to the Yaml file deployment through the Apiserver address and Token information of the cluster where the target node is located, and selecting the target node to dispatch and run the instance Pod;
and after the Pod is deployed, the Pod is successfully operated, the relation corresponding table is adjusted according to the scheduling condition of the current successful operation, the request is scheduled to be a Service instance Service address corresponding to the access target node, and response data is returned to the request terminal.
In the embodiment, the request scheduling is distributed to the node with the optimal resource, and the cross-cluster scheduling of the resource can be realized through the cross-cluster Pod copy drifting technology.
Referring to fig. 2, taking an example that the first node and the target node are both in the cluster a and the first node is in the cluster a and the target node is in the cluster B, the cross-cluster load balancing method of the present embodiment is described as follows:
step1: central cluster uniformly manages node data of cluster A and cluster B
The center cluster calls real-time cluster data of the K8sA cluster and the K8sB cluster in the distributed data storage, performs rasterization processing on the real-time cluster data, processes the real-time cluster data of the K8sA cluster and the K8sB cluster, generates raster data, facilitates subsequent calculation and analysis, splits raster data of which the number is the same as that of the nodes such as K8sA_1, K8sA_2 and the like by taking the nodes as units, calculates the raster data in real time, and analyzes the running state and the load condition of each node of the current K8sA cluster and the current K8sB cluster, wherein the running state and the load condition comprise total resources and occupied resources of each node.
Step2: implementing cross-cluster resource scheduling
1. When a request is initiated by a request end, 2, firstly, inquiring gateway route or service information of the request in a relation corresponding table, analyzing a domain name of the gateway route, finding a cluster K8sA where the gateway route or service is located, then, according to the running state and the load condition of each node in the K8sA and the K8sB which are analyzed by real-time calculation, formulating a scheduling strategy to allocate an optimal target node for the request, and distributing a task processing request to the optimal target node.
3. If the distributed cluster of the target node is K8sA, further determining whether the target node and the first node are the same node, and if the target node and the first node are the same node, directly operating a Service instance pod on the first node; under the condition that the target node and the first node are not the same node, directly distributing the request to a Service instance of the target node in the K8sA, 4. Returning corresponding response data to the request end;
5. under the condition that the distributed cluster of the target node is K8sB, 6. Controlling the K8sB to communicate with the cluster K8sA of the first node to initiate an STCP connection establishment channel, calling a Service interface of a Service instance corresponding to a gateway route in the K8sA cluster, and submitting a Yaml file corresponding to the acquired Service instance to the K8sB through a POST request;
7. controlling the K8sB to generate the same instance according to the Yaml file deployment through the Apiserver address and Token information of the K8sB, and selecting a target node to schedule and run the instance Pod;
8. and after the instances Pod are deployed, successfully operating, and adjusting a corresponding relation table of the request and the gateway route according to the latest scheduling condition, wherein the request is scheduled to access a Service instance Service address corresponding to the gateway route in the K8sB, and response data is returned to the request end.
According to the cross-cluster load balancing method, based on a grid technology, multi-cluster node data are managed and analyzed, data such as running states and load conditions of all nodes in all clusters are calculated and analyzed in real time, a scheduling strategy is formulated to allocate optimal target nodes for requests, STCP connection establishment channels among the clusters are initiated according to the scheduling strategy as required, pod replication drifting of the cross-clusters is conducted, cross-cluster scheduling requirements of resources under a multi-cluster scene are achieved, enterprises are helped to achieve resource management scheduling under the multi-cluster environment, and the capacity of managing the resources and controlling flow of the clusters is achieved.
According to the cross-cluster load balancing method, the cross-cluster resource scheduling method is realized through the grid technology, for K8s clusters deployed in different areas and on different cloud platforms, unified data management and cross-cluster resource scheduling can be realized, enterprises are helped to manage and monitor a plurality of K8s clusters more effectively, resources are supported to be scheduled according to needs, and service quality and efficiency, reliability and safety of a system are improved.
Referring to fig. 3, an embodiment of the present disclosure provides a cross-cluster load balancing apparatus applied to a plurality of different clusters of a network, the apparatus comprising:
a determining module 11, configured to determine service information corresponding to the data request according to the received data request based on a known correspondence between the data request and the service information;
an obtaining module 12, configured to obtain, according to the service information, a resource requirement of response data corresponding to a current data request;
a calculating module 13, configured to calculate an operation state and a load condition of each node on each cluster;
the matching module 14 is configured to match a target node for a current data request from all nodes of all clusters according to an operation state and a load condition of each node on each cluster and a resource requirement of response data corresponding to the current data request, and send a task processing request corresponding to the current data request to the target node.
The implementation process of the functions and roles of each unit in the above device is specifically shown in the implementation process of the corresponding steps in the above method, and will not be described herein again.
For the device embodiments, reference is made to the description of the method embodiments for the relevant points, since they essentially correspond to the method embodiments. The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purposes of the present invention. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
In the above embodiment, any of the determining module 11, the acquiring module 12, the calculating module 13, and the matching module 14 may be combined and implemented in one module, or any of the modules may be split into a plurality of modules. Alternatively, at least some of the functionality of one or more of the modules may be combined with at least some of the functionality of other modules and implemented in one module. At least one of the determination module 11, the acquisition module 12, the calculation module 13 and the matching module 14 may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or in hardware or firmware such as any other reasonable way of integrating or packaging the circuits, or in any one of or a suitable combination of three of software, hardware and firmware. Alternatively, at least one of the determining module 11, the acquiring module 12, the calculating module 13 and the matching module 14 may be at least partly implemented as a computer program module, which when executed may perform the respective functions.
Referring to fig. 4, an electronic device provided by an embodiment of the present disclosure includes a processor 1110, a communication interface 1120, a memory 1130, and a communication bus 1140, where the processor 1110, the communication interface 1120, and the memory 1130 perform communication with each other through the communication bus 1140;
a memory 1130 for storing a computer program;
processor 1110, when executing programs stored on memory 1130, implements a cross-cluster load balancing method as follows:
determining service information corresponding to the data request according to the received data request based on the known correspondence between the data request and the service information;
acquiring the resource requirement of response data corresponding to the current data request according to the service information;
calculating the running state and the load condition of each node on each cluster;
and matching target nodes for the current data request from all nodes of all clusters according to the running state and the load condition of each node on each cluster and the resource requirement of response data corresponding to the current data request, and sending task processing requests corresponding to the current data request to the target nodes.
The communication bus 1140 may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, among others. The communication bus 1140 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface 1120 is used for communication between the electronic device and other devices described above.
The memory 1130 may include random access memory (Random Access Memory, simply RAM) or may include non-volatile memory (non-volatile memory), such as at least one magnetic disk memory. Optionally, the memory 1130 may also be at least one storage device located remotely from the processor 1110.
The processor 1110 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but also digital signal processors (Digital Signal Processing, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field-programmable gate arrays (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
Embodiments of the present disclosure also provide a computer-readable storage medium. The computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements a cross-cluster load balancing method as described above.
The computer-readable storage medium may be embodied in the apparatus/means described in the above embodiments; or may exist alone without being assembled into the apparatus/device. The computer-readable storage medium carries one or more programs that, when executed, implement a cross-cluster load balancing method according to an embodiment of the present disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example, but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
It should be noted that in this document, relational terms such as "first" and "second" and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing is merely a specific embodiment of the disclosure to enable one skilled in the art to understand or practice the disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (8)
1. A method of cross-cluster load balancing, applied to a plurality of different clusters of a network, the method comprising:
determining service information corresponding to the data request according to the received data request based on the known correspondence between the data request and the service information;
acquiring the resource requirement of response data corresponding to the current data request according to the service information;
calculating the running state and the load condition of each node on each cluster;
matching target nodes for the current data request from all nodes of all clusters according to the running state and the load condition of each node on each cluster and the resource requirement of response data corresponding to the current data request, sending task processing requests corresponding to the current data request to the target nodes,
the matching of the target node for the current data request from all nodes of all clusters according to the running state and the load condition of each node on each cluster and the resource requirement of the response data corresponding to the current data request includes:
according to the running state and load condition of each node on each cluster and the resource requirement of response data corresponding to the current data request, screening out nodes meeting preset conditions from all nodes;
sorting the screened nodes according to the order of the priority from big to small;
taking the first node in the ordered nodes as a target node,
wherein the priority is calculated by:
acquiring the node resource ratio, the node CPU and memory ratio of each node and the node affinity and stain mark matching degree as priority computing parameters;
according to each priority computing parameter and the corresponding preset weight, computing the priority of the current node,
the priority of the current node is calculated by the following expression:
FSN= (w1NRPP1) + (w2/>NRPP2) + (w3 />NAP) + (w4 /> TTP)
NRPP1= ( (c cpu - s Pod1 ) 10 / c cpu + (c mem - s Pod2 ) />10 / c mem ) / 2
NRPP2 = 10 - | ( (c cpu - s Pod1 ) 10 / c cpu )/ ( (c mem - s Pod2 ) /> 10 / c mem )-1| 10
NAP = [(S/T)10]
TTP=( (TM 1 -TM 2 ) / TM 1 ) 10
wherein TTP is the score of node taint priority function, TM 1 For the number of all stain entries, TM 2 NAP is the score of the node affinity priority function under the condition that the node adopts a soft strategy, NAP is 10 under the condition that the node adopts a hard strategy, S is the number meeting the requirements of preset rules, T is the number meeting the requirements of all preset rules, NRPP2 is the score of the node resource proportion priority function NRPP2, and c cpu Is the total capacity of the node CPU, s Pod1 For the sum of CPU capacities of the corresponding Pod of the current data request and other pods on the node, c mem Total capacity of node memory s Pod2 For the sum of the memory capacities of the corresponding Pod of the current data request and other pods on the node, NRPP1 is the score of the node resource duty ratio priority function, c cpu Is the total capacity of the node CPU, s Pod1 For the sum of CPU capacities of the corresponding Pod of the current data request and other pods on the node, c mem Total capacity of node memory s Pod2 And summing the memory capacities of the corresponding Pod and other pods on the node for the current data request.
2. The method according to claim 1, wherein the known correspondence between data requests and service information is obtained by:
determining Pod data responsive to the data request;
and taking the service information of the node deployed with the Pod data as the service information corresponding to the data request.
3. The method according to claim 1, wherein the obtaining, according to the service information, the resource requirement of the response data corresponding to the current data request includes:
and inquiring the node where the service information is located, and acquiring the resource requirement of the node for responding to the Pod data of the current data request as the resource requirement of the response data corresponding to the current data request.
4. The method of claim 1, wherein said calculating the operational status and load condition of each node on each cluster comprises:
for each cluster data, splitting raster data with the same number as the cluster nodes by taking the nodes as units, so that each node data corresponds to one raster data;
and carrying out real-time calculation on the raster data corresponding to each node, and taking the calculation result as the running state and the load condition of each node.
5. The method of claim 1, wherein the sending a task processing request corresponding to the current data request to the target node comprises:
inquiring the node where the service information is located as a first node;
determining whether the first node and the target node are in the same cluster;
under the condition that the first node and the target node are in the same cluster, directly sending a task processing request corresponding to the current data request to a service instance of the target node, and returning corresponding response data to a request end;
and under the condition that the first node and the target node are not in the same cluster, establishing a channel for the cluster where the first node is located and the cluster where the target node is located, calling a service instance of the first node, and submitting a Yaml file corresponding to the service instance of the first node to the target node.
6. A cross-cluster load balancing apparatus for use with a plurality of different clusters of networks, the apparatus comprising:
the determining module is used for determining service information corresponding to the data request according to the received data request based on the known corresponding relation between the data request and the service information;
the acquisition module is used for acquiring the resource requirement of the response data corresponding to the current data request according to the service information;
the computing module is used for computing the running state and the load condition of each node on each cluster;
a matching module, configured to match a target node for a current data request from all nodes of all clusters according to an operation state and a load condition of each node on each cluster and a resource requirement of response data corresponding to the current data request, and send a task processing request corresponding to the current data request to the target node,
the matching of the target node for the current data request from all nodes of all clusters according to the running state and the load condition of each node on each cluster and the resource requirement of the response data corresponding to the current data request includes:
according to the running state and load condition of each node on each cluster and the resource requirement of response data corresponding to the current data request, screening out nodes meeting preset conditions from all nodes;
sorting the screened nodes according to the order of the priority from big to small;
taking the first node in the ordered nodes as a target node,
wherein the priority is calculated by:
acquiring the node resource ratio, the node CPU and memory ratio of each node and the node affinity and stain mark matching degree as priority computing parameters;
according to each priority computing parameter and the corresponding preset weight, computing the priority of the current node,
the priority of the current node is calculated by the following expression:
FSN= (w1NRPP1) + (w2/>NRPP2) + (w3 />NAP) + (w4 />TTP)
NRPP1= ( (c cpu - s Pod1 ) 10 / c cpu + (c mem - s Pod2 ) />10 / c mem ) / 2
NRPP2 = 10 - | ( (c cpu - s Pod1 ) 10 / c cpu )/ ( (c mem - s Pod2 ) />10 / c mem )-1| />10
NAP = [(S/T)10]
TTP=( (TM 1 -TM 2 ) / TM 1 ) 10
wherein TTP is the score of node taint priority function, TM 1 For the number of all stain entries, TM 2 For the number of successful taint items paired, NAP is the score of the node affinity priority function in case of soft policy for the node, NAP is 10 in case of hard policy for the node, S is the number meeting the preset rule requirement,t is the number required by all preset rules, NRPP2 is the score of a node resource proportion priority function NRPP2, c cpu Is the total capacity of the node CPU, s Pod1 For the sum of CPU capacities of the corresponding Pod of the current data request and other pods on the node, c mem Total capacity of node memory s Pod2 For the sum of the memory capacities of the corresponding Pod of the current data request and other pods on the node, NRPP1 is the score of the node resource duty ratio priority function, c cpu Is the total capacity of the node CPU, s Pod1 For the sum of CPU capacities of the corresponding Pod of the current data request and other pods on the node, c mem Total capacity of node memory s Pod2 And summing the memory capacities of the corresponding Pod and other pods on the node for the current data request.
7. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
a memory for storing a computer program;
a processor configured to implement the cross-cluster load balancing method of any one of claims 1-5 when executing a program stored on a memory.
8. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements the cross-cluster load balancing method of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311422616.3A CN117149445B (en) | 2023-10-31 | 2023-10-31 | Cross-cluster load balancing method and device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311422616.3A CN117149445B (en) | 2023-10-31 | 2023-10-31 | Cross-cluster load balancing method and device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117149445A CN117149445A (en) | 2023-12-01 |
| CN117149445B true CN117149445B (en) | 2024-01-12 |
Family
ID=88908470
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311422616.3A Active CN117149445B (en) | 2023-10-31 | 2023-10-31 | Cross-cluster load balancing method and device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117149445B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103207814A (en) * | 2012-12-27 | 2013-07-17 | 北京仿真中心 | Decentralized cross cluster resource management and task scheduling system and scheduling method |
| CN104461740A (en) * | 2014-12-12 | 2015-03-25 | 国家电网公司 | Cross-domain colony computing resource gathering and distributing method |
| WO2023109067A1 (en) * | 2021-12-15 | 2023-06-22 | 深圳前海微众银行股份有限公司 | Pod load balancing method, apparatus and device and storage medium |
| CN116737378A (en) * | 2023-06-13 | 2023-09-12 | 平安科技（深圳）有限公司 | Dynamic resource allocation method and device, electronic equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9225806B2 (en) * | 2012-06-15 | 2015-12-29 | Citrix Systems, Inc. | Systems and methods for generating IPID across a cluster network |
-
2023
- 2023-10-31 CN CN202311422616.3A patent/CN117149445B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103207814A (en) * | 2012-12-27 | 2013-07-17 | 北京仿真中心 | Decentralized cross cluster resource management and task scheduling system and scheduling method |
| CN104461740A (en) * | 2014-12-12 | 2015-03-25 | 国家电网公司 | Cross-domain colony computing resource gathering and distributing method |
| WO2023109067A1 (en) * | 2021-12-15 | 2023-06-22 | 深圳前海微众银行股份有限公司 | Pod load balancing method, apparatus and device and storage medium |
| CN116737378A (en) * | 2023-06-13 | 2023-09-12 | 平安科技（深圳）有限公司 | Dynamic resource allocation method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117149445A (en) | 2023-12-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112000448A (en) | Micro-service architecture-based application management method | |
| CN112532675B (en) | Method, device and medium for establishing network edge computing system | |
| US20190034084A1 (en) | Selecting controllers based on affinity between access devices and storage segments | |
| US7502850B2 (en) | Verifying resource functionality before use by a grid job submitted to a grid environment | |
| US8352634B2 (en) | On-demand propagation of routing information in distributed computing system | |
| JP5998206B2 (en) | Scalable centralized dynamic resource distribution in cluster data grids | |
| CN107545338B (en) | Service data processing method and service data processing system | |
| US20060048157A1 (en) | Dynamic grid job distribution from any resource within a grid environment | |
| CN110798517B (en) | Decentralized cluster load balancing method and system, mobile terminal and storage medium | |
| US20100138540A1 (en) | Method of managing organization of a computer system, computer system, and program for managing organization | |
| CN109525410B (en) | Distributed storage system upgrading management method and device and distributed storage system | |
| WO2021227999A1 (en) | Cloud computing service system and method | |
| CN110071965B (en) | Data center management system based on cloud platform | |
| CN110221920B (en) | Deployment method, device, storage medium and system | |
| US20230222006A1 (en) | Container Orchestration System | |
| EP4167539A1 (en) | Resource management method and system, proxy server, and storage medium | |
| US20220318071A1 (en) | Load balancing method and related device | |
| US20220329651A1 (en) | Apparatus for container orchestration in geographically distributed multi-cloud environment and method using the same | |
| CN112532674A (en) | Method, device and medium for establishing network edge computing system | |
| CN115134371A (en) | Scheduling method, system, equipment and medium containing edge network computing resources | |
| CN112187864A (en) | Load balancing method and device, storage medium and electronic equipment | |
| US20110153826A1 (en) | Fault tolerant and scalable load distribution of resources | |
| CN117149445B (en) | Cross-cluster load balancing method and device, equipment and storage medium | |
| CN112532667A (en) | Network edge computing method, device and medium | |
| CN112001800B (en) | Method and device for processing business in block chain system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |