CN114338695A - Leader election method, device, equipment and storage medium based on multiple copies - Google Patents
Leader election method, device, equipment and storage medium based on multiple copies Download PDFInfo
- Publication number
- CN114338695A CN114338695A CN202210228188.XA CN202210228188A CN114338695A CN 114338695 A CN114338695 A CN 114338695A CN 202210228188 A CN202210228188 A CN 202210228188A CN 114338695 A CN114338695 A CN 114338695A
- Authority
- CN
- China
- Prior art keywords
- application
- leader
- copy
- copies
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The application provides a leader election method, a device, equipment and a storage medium based on multiple copies, and relates to the technical field of communication. The method can be applied to a target cluster, and comprises the following steps: acquiring a plurality of resource observation vectors at the current moment through each first application copy; acquiring a prediction submission delay sum corresponding to each first application copy at the current moment according to a plurality of resource observation vectors and a preset predictor at the current moment; the method and the device have the advantages that the first target leader application copy is determined in the multiple first application copies at the current moment according to the sum of the predicted submission delays corresponding to the first application copies at the current moment, so that the sum of the predicted submission delays corresponding to the first application copies at the current moment can be quickly obtained according to the preset predictor and the multiple resource observation vectors at the current moment, the first target leader application copy can be quickly elected in the multiple first application copies at the current moment, and the method and the device have the advantages of being simple in election mode and high in election efficiency.
Description
Technical Field
The present application relates to the field of communications technologies, and in particular, to a leader election method, apparatus, device, and storage medium based on multiple copies.
Background
A cluster is a group of mutually independent computers interconnected by a high-speed network, which form a group and are managed in a single system mode. A client interacts with a cluster, which appears as a stand-alone server.
Currently, to enhance cluster reliability, applications in a cluster are typically deployed in multiple copies to tolerate a downtime or distributed failure. In order to ensure consistency of data in multiple application copies, a leader in the multiple application copies is often determined based on a Raft algorithm, so that based on the application leader, it can be ensured that client requests are ordered in an ordered log and can be forwarded to other application copies.
However, because the Raft algorithm is relatively complex, when determining the leader of multiple application copies based on the Raft algorithm, the election efficiency is often low.
Disclosure of Invention
The present application aims to provide a leader election method, device, equipment and storage medium based on multiple copies, which can improve election efficiency, in view of the above disadvantages in the prior art.
In order to achieve the above purpose, the technical solutions adopted in the embodiments of the present application are as follows:
in a first aspect, the present invention provides a leader election method based on multiple copies, applied to a target cluster, where the target cluster includes: the method comprises the following steps that a plurality of working nodes comprise a plurality of first application copies corresponding to target application programs, and the method comprises the following steps:
acquiring a plurality of resource observation vectors at the current moment through each first application copy, wherein each resource observation vector comprises communication round-trip time between the corresponding first application copy and other first application copies;
acquiring a sum of predicted submission delays corresponding to the first application copies at the current moment according to the plurality of resource observation vectors and a preset predictor at the current moment, wherein the preset predictor is used for predicting and acquiring the predicted submission delay of each first application copy initiating a request to a leader application copy at the previous deadline;
and determining a first target leader application copy in the plurality of first application copies at the current moment according to the sum of the predicted submission delays corresponding to the first application copies at the current moment.
In an optional implementation manner, before the obtaining, according to the plurality of resource observation vectors and the preset predictor at the current time, a sum of predicted submission delays corresponding to the first application copies at the current time, the method further includes:
acquiring a resource observation vector of each first application copy and a submission delay time corresponding to each first application copy in a history deadline time period corresponding to a history deadline leader application copy, and taking a plurality of first application copies as a plurality of training samples, wherein the submission delay time is used for representing the delay time of a request message sent by the first application copy to a history deadline leader application copy forwarding client;
and training to obtain the predictor according to a plurality of training samples.
In an optional embodiment, the working node further includes: determining a first target leader application copy from the plurality of first application copies at the current time according to the predicted submission delay sums corresponding to the first application copies at the current time, wherein the determining the plurality of second application copies corresponding to other application programs comprises:
determining at least one first candidate leader application copy in the plurality of first application copies at the current moment according to the sum of the predicted submission delays corresponding to the first application copies at the current moment;
and determining a first target leader application copy in the at least one first candidate leader application copy according to the at least one first candidate leader application copy and a preset leader equilibrium distribution condition, wherein the preset leader equilibrium distribution condition is used for representing the maximum number of leader application copies in the working nodes.
In an optional embodiment, the target cluster is configured with a leader management object, the leader management object comprising: the determining a first target leader application copy among at least one of the candidate leader application copies according to the at least one candidate leader application copy and a preset leader balancing distribution condition comprises:
acquiring the number of application copies of the leader on each working node at the current moment through the leader management object;
acquiring target working nodes meeting preset leader balance distribution conditions according to the number of application copies of the leader on each working node at the current moment;
determining a first target leader application copy among at least one of the candidate leader application copies based on at least one of the candidate leader application copy and the target worker node.
In an optional embodiment, a plurality of application copies corresponding to each type of application program are configured to correspond to one copy management object, the copy management object includes leader record information of the corresponding plurality of application copies, and the leader record information includes: the leader applies the working node identification and update time at which the copy resides, the method further comprising:
acquiring leader record information corresponding to various application copies through the copy management object;
and acquiring the number of the leader application copies on each working node according to the leader record information corresponding to each application copy and recording the number of the leader application copies into the leader management object.
In an alternative embodiment, the method further comprises:
respectively acquiring a first number of application copies of a leader in a target cluster and a second number of working nodes;
and determining the preset leader balance distribution condition according to the first quantity and the second quantity.
In an alternative embodiment, each of the resource observation vectors further comprises: the CPU resource occupancy rate of the corresponding first application copy, and/or the RAM resource occupancy rate of the corresponding first application copy.
In a second aspect, the present invention provides a leader election device based on multiple copies, applied to a target cluster, where the target cluster includes: a plurality of working nodes, wherein the working nodes comprise a plurality of first application copies corresponding to the target application program, and the device comprises:
a first obtaining module, configured to obtain, through each first application copy, a plurality of resource observation vectors at a current time, where each resource observation vector includes a communication round-trip time between the corresponding first application copy and other first application copies;
a second obtaining module, configured to obtain, according to the multiple resource observation vectors at the current time and a preset predictor, a sum of predicted submission delays corresponding to the first application copies at the current time, where the preset predictor is configured to predict a predicted submission delay for obtaining that each first application copy initiates a request to a previous application copy of a leader in an arbitrary period;
and the determining module is used for determining a first target leader application copy from the plurality of first application copies at the current moment according to the sum of the predicted submission delays corresponding to the first application copies at the current moment.
In an optional embodiment, the second obtaining module is further configured to obtain a resource observation vector of each first application copy and a submission delay time corresponding to each first application copy in a history tenure period corresponding to a history tenure leader application copy, and use a plurality of first application copies as a plurality of training samples, where the submission delay time is used to characterize a delay time for the first application copy to forward a request message sent by a client to the history tenure leader application copy;
and training to obtain the predictor according to a plurality of training samples.
In an optional embodiment, the working node further includes: the determining module is specifically configured to determine at least one first candidate leader application copy from among the plurality of first application copies at the current time according to a sum of predicted submission delays corresponding to the respective first application copies at the current time;
and determining a first target leader application copy in the at least one first candidate leader application copy according to the at least one first candidate leader application copy and a preset leader equilibrium distribution condition, wherein the preset leader equilibrium distribution condition is used for representing the maximum number of leader application copies in the working nodes.
In an optional embodiment, the target cluster is configured with a leader management object, the leader management object comprising: the determining module is specifically configured to acquire, by the leader management object, the number of leader application copies on each working node at the current time;
acquiring target working nodes meeting preset leader balance distribution conditions according to the number of application copies of the leader on each working node at the current moment;
determining a first target leader application copy among at least one of the candidate leader application copies based on at least one of the candidate leader application copy and the target worker node.
In an optional embodiment, a plurality of application copies corresponding to each type of application program are configured to correspond to one copy management object, the copy management object includes leader record information of the corresponding plurality of application copies, and the leader record information includes: the leader applies the working node identification and the update time at which the copy is located, the apparatus further comprising: the recording module is used for acquiring leader recording information corresponding to various application copies through the copy management object;
and acquiring the number of the leader application copies on each working node according to the leader record information corresponding to each application copy and recording the number of the leader application copies into the leader management object.
In an optional embodiment, the determining module is further configured to obtain a first number of leader application copies and a second number of working nodes in the target cluster, respectively;
and determining the preset leader balance distribution condition according to the first quantity and the second quantity.
In an alternative embodiment, each of the resource observation vectors further comprises: the CPU resource occupancy rate of the corresponding first application copy, and/or the RAM resource occupancy rate of the corresponding first application copy.
In a third aspect, the present invention provides an electronic device comprising: a processor, a storage medium and a bus, the storage medium storing machine readable instructions executable by the processor, the processor and the storage medium communicating via the bus when the electronic device is operating, the processor executing the machine readable instructions to perform the steps of the multi-copy based leader election method according to any of the preceding embodiments.
In a fourth aspect, the present invention provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the multi-copy based leader election method according to any one of the preceding embodiments.
The beneficial effect of this application is:
the leader election method, device, equipment and storage medium based on multiple copies provided in the embodiments of the present application may be applied to a target cluster, where the target cluster may include multiple working nodes, and the working nodes may include multiple first application copies corresponding to a target application program, and the method includes: acquiring a plurality of resource observation vectors at the current moment through each first application copy, wherein each resource observation vector comprises communication round-trip time between the corresponding first application copy and other first application copies; the method comprises the steps that the sum of the predicted submission delay corresponding to each first application copy at the current moment is obtained according to a plurality of resource observation vectors at the current moment and a preset predictor, wherein the preset predictor is used for predicting the predicted submission delay of the request initiated by each first application copy to the application copy of the leader in the previous period; the method and the device have the advantages that the first target leader application copy is determined in the multiple first application copies at the current moment according to the sum of the predicted submission delays corresponding to the first application copies at the current moment, so that the sum of the predicted submission delays corresponding to the first application copies at the current moment can be quickly obtained according to the preset predictor and the multiple resource observation vectors at the current moment, the first target leader application copy can be quickly elected in the multiple first application copies at the current moment, and the method and the device have the advantages of being simple in election mode and high in election efficiency.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
FIG. 1 is a schematic flowchart of a leader election method based on multiple copies according to an embodiment of the present application;
FIG. 2 is a schematic flowchart of another leader election method based on multiple copies according to an embodiment of the present application;
fig. 3 is a schematic architecture diagram of a cluster according to an embodiment of the present application;
FIG. 4 is a schematic flowchart of another leader election method based on multiple copies according to an embodiment of the present application;
FIG. 5 is a schematic flowchart of another leader election method based on multiple copies according to an embodiment of the present application;
FIG. 6 is a schematic flowchart of yet another leader election method based on multiple copies according to an embodiment of the present application;
FIG. 7 is a schematic flowchart of another leader election method based on multiple copies according to an embodiment of the present application;
FIG. 8 is a functional block diagram of a leader election device based on multiple copies according to an embodiment of the present application;
fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
In the prior art, leaders in multiple application copies are often determined based on a raw algorithm, but the raw algorithm is relatively complex, and particularly, uniform leader election in the raw algorithm may cause slow placement speed or low efficiency of the leaders, so that a synchronization process is decelerated and response time of a client request is lost. The problem of inefficiency for the leader is also exacerbated in geographically isolated Raft deployments.
Therefore, the existing leader election method based on multiple copies has the problem of low election efficiency.
In view of this, the present application provides a leader election method based on multiple copies, and by applying the method, election efficiency of a leader copy in multiple copies can be improved.
Fig. 1 is a schematic flowchart of a leader election method based on multiple copies according to an embodiment of the present application, where the method may be applied to a target cluster, and optionally, the target cluster may be a Kafka cluster, a Kubernetes cluster, or the like, which is not limited herein. The target cluster may include: the system comprises a plurality of working nodes, wherein the working nodes can comprise a plurality of first application copies corresponding to target type applications, and each first application copy can be understood as a complete copy of the target type application to tolerate the fault shutdown or distributed fault. Alternatively, the target class application may be a game application, a shopping application, a social application, etc., and is not limited herein. As shown in fig. 1, the method may include:
s101, a plurality of resource observation vectors at the current moment are obtained through the first application copies, and each resource observation vector comprises communication round-trip time between the corresponding first application copy and other first application copies.
Each first application copy may be configured to periodically send a preset signal to other first application copies to obtain Round-Trip Time (RTT) between the first application copy and the other first application copies, and a resource observation vector may be generated according to the Round-Trip Time (RTT), where the first application copy may be configured to generate a resource observation vector
Corresponding resource observation vector
Can be recorded as
Wherein, in the step (A),
representing a first application copy
And a first set of application replicas
Copy of a mid-application
The round-trip time of the communication between,
，
。
based on this description, in some embodiments, each first application copy may obtain a resource observation vector at the current time and send the resource observation vector to the target cluster, and of course, in some embodiments, the target cluster may also actively obtain a corresponding resource observation vector from each first application copy, which is not limited herein. Optionally, the target cluster may store the obtained resource observation vector in a preset queue.
S102, obtaining a prediction submission delay sum corresponding to each first application copy at the current moment according to the plurality of resource observation vectors and the preset predictor at the current moment.
The preset predictor is used for predicting and acquiring the predicted submission delay of each first application copy initiating a request to the previous application copy of the deadline leader. That is, according to the multiple resource observation vectors at the current time and the predicted submission delay of each first application copy initiating a request to the previous application copy of the deadline leader, the sum of the predicted submission delays corresponding to each first application copy at the current time can be obtained. The sum of the predicted submission delays corresponding to the first application copies at the current time may represent the communication state of each first application copy at the current time, and it may be understood that, at the current time, if the sum of the predicted submission delays corresponding to a certain first application copy is lower, the communication state of the first application copy is better, and otherwise, the communication state of the first application copy is poorer.
S103, determining a first target leader application copy from the plurality of first application copies at the current moment according to the sum of the predicted submission delays corresponding to the first application copies at the current moment.
The total predicted submission delay corresponding to each first application copy at the current moment can represent the probability that each first application copy becomes a leader in a plurality of first application copies to a certain extent, and can also be regarded as the fitness score of each first application copy as a first target leader application copy, wherein the total predicted submission delay corresponding to each first application copy and the probability that the total predicted submission delay corresponding to each first application copy becomes a leader in the plurality of first application copies can be in inverse proportion, that is, the lower the total predicted submission delay corresponding to each first application copy, the higher the probability that the total predicted submission delay corresponding to each first application copy becomes a leader in the plurality of first application copies.
Based on the above description, it can be understood that, at this time, a first target leader application copy may be determined from the multiple first application copies at the current time according to the predicted submission delay sum corresponding to each first application copy at the current time, and then based on the first target leader application copy, it may be ensured that the client requests are ordered in the ordered log to forward the requests to other non-target leader first application copies, and the consistency of data in the multiple first application copies is ensured to tolerate a downtime or a distributed failure, so as to improve the reliability of the target-class application program. In addition, the leader election method provided by the application has the characteristic of simplicity in election, and compared with a method for determining the leader in multiple application copies based on a Raft algorithm in the prior art, the leader election method can improve election efficiency.
To sum up, the leader election method based on multiple copies provided by the embodiment of the present application may be applied to a target cluster, where the target cluster may include multiple working nodes, and the working nodes may include multiple first application copies corresponding to a target application program, and the method includes: acquiring a plurality of resource observation vectors at the current moment through each first application copy, wherein each resource observation vector comprises communication round-trip time between the corresponding first application copy and other first application copies; the method comprises the steps that the sum of the predicted submission delay corresponding to each first application copy at the current moment is obtained according to a plurality of resource observation vectors at the current moment and a preset predictor, wherein the preset predictor is used for predicting the predicted submission delay of the request initiated by each first application copy to the application copy of the leader in the previous period; the method and the device have the advantages that the first target leader application copy is determined in the multiple first application copies at the current moment according to the sum of the predicted submission delays corresponding to the first application copies at the current moment, so that the sum of the predicted submission delays corresponding to the first application copies at the current moment can be quickly obtained according to the preset predictor and the multiple resource observation vectors at the current moment, the first target leader application copy can be quickly elected in the multiple first application copies at the current moment, and the method and the device have the advantages of being simple in election mode and high in election efficiency.
FIG. 2 is a flowchart illustrating another leader election method based on multiple copies according to an embodiment of the present application. Optionally, as shown in fig. 2, before obtaining a total of the predicted submission delays corresponding to the first application copies at the current time according to the plurality of resource observation vectors and the preset predictor, the method further includes:
s201, acquiring a resource observation vector of each first application copy and a submission delay time corresponding to each first application copy in a history deadline time period corresponding to the history deadline leader application copy, and taking the plurality of first application copies as a plurality of training samples.
Wherein the submission delay time is used to characterize a delay time for the first application copy to forward the request message sent by the client to the historical due leader application copy.
The submission delay time corresponding to each first application copy may be obtained as follows: in the history deadline time period corresponding to the history deadline leader application copy, an application client communicating with each first application copy transmits a request message of the local end to the first application copy closest to the application client, records a timestamp of arrival of the request message by the first application copy closest to the application client, forwards and submits the request message to the history deadline leader application copy, and records a submitted timestamp, where a time delay between the submitted timestamp and the timestamp of arrival of the request message may be referred to as a submission delay time. The obtaining manner of the resource observation vector of each first application copy in the historical time period may refer to the obtaining manner of the resource observation vector of each first application copy at the current time, which is not described herein again.
Based on the above description, in the history deadline time period corresponding to the history deadline leader application copy, after the resource observation vector of each first application copy in the multiple first application copies and the commit delay time corresponding to each first application copy are obtained, the multiple first application copies may be used as multiple training samples.
Alternatively, in some embodiments, the plurality of training samples may be represented by the following formula:
，
. Wherein R1 represents a training sample, x represents a set of resource observation vectors corresponding to a plurality of training samples,
a copy is applied for the history tenure leader,
representing a first application copy
Apply copies with historical tenure leader
Y represents a set of submission delay times corresponding to the plurality of training samples,
representing a first application copy
Applying a copy to a historical tenure leader
And forwarding the delay time of the request message sent by the client.
S202, training and obtaining a predictor according to the plurality of training samples.
It can be seen that the obtained training samples may include multiple first application copies within a history tenure period corresponding to the history tenure leader application copy, each application copy may be labeled with a resource observation vector and a commit delay time, and then the predictor may be trained according to the training samples. It can be appreciated that the trained predictor can predict a predicted commit delay for obtaining the first application copy to be predicted to initiate a request to its corresponding current tenure leader application copy based on the resource observation vector corresponding to the first application copy to be predicted.
Optionally, in some embodiments, the training samples may be trained by using an Ordinary Least Squares (OLS), and the predictor may be obtained by training based on the idea of minimizing the sum of squares of errors. The current specific training mode is not limited to this.
It should be noted that, in order to improve the accuracy of the prediction result of the predictor, optionally, the training samples may be updated according to a preset time interval, and then the predictor may be updated according to the updated training samples, so that the accuracy of the prediction result may be improved.
In some embodiments, based on the above description, the process of obtaining the sum of the predicted submission delays corresponding to the first application copies at the current time according to the plurality of resource observation vectors and the preset predictor at the current time may be completed with reference to the following formula:
wherein, in the step (A),
representing an arbitrary time
First application copy
The sum of the corresponding predicted commit delays,
representing a first application copy
Applying a predicted commit delay of the replica initiating requests to the historical tenure leader,
representing a first set of application replicas.
Fig. 3 is a schematic architecture diagram of a cluster according to an embodiment of the present disclosure. As shown in fig. 3, the cluster may include a first working Node1 and a second working Node2, where the first working Node1 and the second working Node2 respectively run a first application copy corresponding to the first type of application APP1 and a second application copy corresponding to the second type of application APP 2. If the first type application APP1 is a target type application, as can be seen from the above description, by applying the embodiment of the present application, a first target Leader application copy in a quick election may be selected from the first working Node1 and the second working Node2, as shown in fig. 3, in some embodiments, the first application copy running on the first working Node1 may be determined to be a first target Leader application copy (Leader), and optionally, the first application copy running on the second working Node1 at this time may be referred to as a Follower application copy (Follower).
FIG. 4 is a flowchart illustrating another leader election method based on multiple copies according to an embodiment of the present application. As can be seen from fig. 3, the working node further includes: and a plurality of second application copies corresponding to other types of application programs, for example, a second application copy corresponding to the second type of application program APP 2. As shown in fig. 4, the step of determining a first target leader application copy from among the plurality of first application copies at the current time according to the sum of the predicted submission delays corresponding to the first application copies at the current time may include:
s301, determining at least one first candidate leader application copy in the plurality of first application copies at the current moment according to the sum of the predicted submission delays corresponding to the first application copies at the current moment.
Referring to the above description, after determining the predicted submission delay sums corresponding to the first application copies at the current time, the predicted submission delay sums may be sorted (in an ascending or descending order), an ordered list may be obtained, and a first candidate leader application copy may be determined from the plurality of first application copies at the current time according to the ordered list. Optionally, in the specific determination, multiple first application copies with a lower total sum of ordering prediction submission delays may be obtained as the first candidate leader application copy, and taking an ascending order as an example, W first application copies with a top ordering may be obtained as the first candidate leader application copies, where a value of W may be 3, 5, and the like, which is not limited herein.
S302, determining a first target leader application copy in the at least one first candidate leader application copy according to the at least one first candidate leader application copy and a preset leader balance distribution condition.
And the preset leader balance distribution condition is used for representing the maximum number of the leader application copies in the working nodes. Through the preset Leader balance distribution condition, the distribution of Leader application copy leaders in all the working nodes can be balanced, and the Leader application copy is prevented from being stacked on a single working node.
Optionally, when the determination is specifically performed, a candidate leader application copy meeting the preset leader balance distribution condition may be screened from the at least one first candidate leader application copy according to the preset leader balance distribution condition as the first target leader application copy. In some embodiments, if the candidate leader application copies meeting the preset leader balancing distribution condition include a plurality of candidate leader application copies, any one of the candidate leader application copies may be selected as the first target leader application copy, or alternatively, the first target leader application copy may be determined according to a preset selection rule. By applying the embodiment of the application, the problem that the running performance of the target cluster is reduced due to the fact that a plurality of target leader application copies corresponding to various application programs are concentrated on the same working node can be solved, in addition, the problem that the response time of the target cluster is long due to network congestion can be reduced, and the running performance of the target cluster is guaranteed.
FIG. 5 is a flowchart illustrating another leader election method based on multiple copies according to an embodiment of the present application. Optionally, referring to fig. 3, the target cluster is further configured with a Leader Management object (LEMP), where the Leader Management object may include: the number of leader application copies on each worker node. In some embodiments, the leader management object may configure the target cluster in the form of a configuration file. As shown in fig. 5, the step of determining a first target leader application copy among the at least one first candidate leader application copy according to the at least one first candidate leader application copy and the preset leader balanced distribution condition may include:
s401, acquiring the number of leader application copies on each working node at the current time through the leader management object.
S402, obtaining target working nodes meeting preset leader balance distribution conditions according to the number of the leader application copies on each working node at the current moment.
S403, determining a first target leader application copy in the at least one first candidate leader application copy according to the at least one first candidate leader application copy and the target working node.
The leader management object is stored in a plurality of working nodes, wherein the number of the leader application copies on each working node stored in the leader management object can be updated in real time, that is, if the information of the leader application copies on any working node in the target cluster is determined to be changed, an update message can be generated, and the leader management object is updated according to the update message, so that the real-time performance of the leader management object can be ensured.
Based on the above description, after the number of the leader application copies on each working node at the current time is obtained through the leader management object, a target working node satisfying a preset leader balance distribution condition may be screened out from the plurality of working nodes according to the number, and then a first target leader application copy may be determined according to the target working node and the determined at least one first candidate leader application copy. Of course, it should be noted that if the working node where a first candidate leader application copy is located does not satisfy the preset leader balanced distribution condition, the corresponding first candidate leader application copy, which is still a copy, may periodically attempt to obtain the leader by repeating the above-mentioned correlation process.
FIG. 6 is a flowchart illustrating a further leader election method based on multiple copies according to an embodiment of the present application. Referring to fig. 3, the multiple application copies corresponding to the various types of application programs are configured to correspond to one copy management object (EndPoint, EP), where the copy management object may include leader record information of the corresponding multiple application copies, and the leader record information may include: the working node at which the leader application copy (i.e., the leader application copy of the multiple application copy pairs) resides identifies and updates the time. In some embodiments, each copy management object may manage its corresponding plurality of application copies in the form of a configuration file. As shown in fig. 6, the method further includes:
s501, leader record information corresponding to various application copies is obtained through the copy management object.
S502, acquiring the number of the leader application copies on each working node according to the leader recording information corresponding to each application copy and recording the number of the leader application copies to a leader management object.
Based on the above description, it can be understood that after the leader record information corresponding to each type of application copy is obtained by the copy management object, the number of leader application copies on each working node corresponding to the latest update time can be obtained according to the update time in each leader record information, and the number of leader application copies on each working node corresponding to the acquired latest update time is recorded in the leader management object, so that the number of leader application copies on each working node stored by the leader management object can be ensured to be updated in real time.
Optionally, in some embodiments, if it is determined that the copy management object is not configured for the multiple application copies corresponding to the various types of application programs, optionally, the copy management object may be configured by using a configuration file corresponding to the copy management object.
FIG. 7 is a flowchart illustrating a further leader election method based on multiple copies according to an embodiment of the present application. Optionally, as shown in fig. 7, the method further includes:
s601, respectively obtaining a first number of application copies of the leader in the target cluster and a second number of working nodes.
S602, determining a preset leader balance distribution condition according to the first quantity and the second quantity.
Optionally, the first number of the leader application copies and the second number of the working nodes in the target cluster may be obtained through a preset configuration file, and of course, a specific obtaining manner is not limited thereto. For example, the number of worker nodes may be retrieved through an Application Programming Interface (API) in the target cluster.
In some embodiments, the determined preset leader balancing distribution condition may be expressed by the following formula according to the first number and the second number:
wherein, in the step (A),
applying a first number of replicas for the leader in the target cluster,
for the second number of worker nodes in the target cluster,
the maximum number of copies to apply for the leader in each worker node. Of course, the expression formula for presetting the leader balance distribution condition is not limited to this, and can be flexibly adjusted according to the actual application scenario.
Optionally, each of the resource observation vectors further includes: the CPU resource occupancy rate of the corresponding first application copy, and/or the RAM resource occupancy rate of the corresponding first application copy.
Referring to the foregoing description, that is, each resource observation vector may include a communication round trip time between the corresponding first application copy and other first application copies, a CPU resource occupancy of the corresponding first application copy, and/or a RAM resource occupancy of the corresponding first application copy. Of course, the present application does not limit the specific content in each resource observation vector, and the combination can be flexible according to the actual application scenario.
For example, the first application copy
Corresponding resource observation vector
Can be recorded as
Wherein, in the step (A),
representing a first application copy
The corresponding occupancy rate of the CPU resource is,
representing a first application copy
For the corresponding RAM resource occupancy, the related description for other parameters can be referred to, and will not be described herein again.
Of course, it should be noted that, according to an actual application scenario, the resource observation vector in the first application copy corresponding to each training sample may also include the CPU resource occupancy rate of the corresponding first application copy, and/or the RAM resource occupancy rate of the corresponding first application copy, which is not limited herein.
By applying the embodiment of the application, the first target leader application copy can be determined from the plurality of first application copies at the current moment by combining a plurality of resources, so that the problem of long response time of the target cluster caused by unbalanced resource allocation can be avoided, and the operation performance of the target cluster is further ensured.
Fig. 8 is a functional module schematic diagram of a multi-copy-based leader election device according to an embodiment of the present application, where the device may be an apparatus deployed with the foregoing target cluster, where the target cluster includes: the basic principle and the generated technical effect of the device are the same as those of the corresponding method embodiments, and for the sake of brief description, no part mentioned in this embodiment may refer to corresponding contents in the method embodiments. As shown in FIG. 8, the leader election device 100 may include:
a first obtaining module 110, configured to obtain, through each first application copy, a plurality of resource observation vectors at a current time, where each resource observation vector includes a communication round-trip time between the corresponding first application copy and other first application copies;
a second obtaining module 120, configured to obtain, according to the multiple resource observation vectors at the current time and a preset predictor, a sum of predicted submission delays corresponding to the first application copies at the current time, where the preset predictor is configured to predict and obtain a predicted submission delay for each first application copy initiating a request to a previous application copy of an arbitrary leader;
a determining module 130, configured to determine, according to a sum of predicted submission delays corresponding to the first application copies at the current time, a first target leader application copy from among the first application copies at the current time.
In an optional embodiment, the second obtaining module 120 is further configured to obtain a resource observation vector of each first application copy and a submission delay time corresponding to each first application copy in a historical deadline time period corresponding to a historical deadline leader application copy, and use a plurality of first application copies as a plurality of training samples, where the submission delay time is used to characterize a delay time of a request message sent by the first application copy to a historical deadline leader application copy forwarding client;
and training to obtain the predictor according to a plurality of training samples.
In an optional embodiment, the working node further includes: a plurality of second application copies corresponding to other application programs, where the determining module 130 is specifically configured to determine at least one first candidate leader application copy from among the plurality of first application copies at the current time according to a sum of predicted submission delays corresponding to the first application copies at the current time;
and determining a first target leader application copy in the at least one first candidate leader application copy according to the at least one first candidate leader application copy and a preset leader equilibrium distribution condition, wherein the preset leader equilibrium distribution condition is used for representing the maximum number of leader application copies in the working nodes.
In an optional embodiment, the target cluster is configured with a leader management object, the leader management object comprising: the determining module is specifically configured to acquire, by the leader management object, the number of leader application copies on each working node at the current time;
acquiring target working nodes meeting preset leader balance distribution conditions according to the number of application copies of the leader on each working node at the current moment;
determining a first target leader application copy among at least one of the candidate leader application copies based on at least one of the candidate leader application copy and the target worker node.
In an optional embodiment, a plurality of application copies corresponding to each type of application program are configured to correspond to one copy management object, the copy management object includes leader record information of the corresponding plurality of application copies, and the leader record information includes: the leader applies the working node identification and the update time at which the copy is located, the apparatus further comprising: the recording module is used for acquiring leader recording information corresponding to various application copies through the copy management object;
and acquiring the number of the leader application copies on each working node according to the leader record information corresponding to each application copy and recording the number of the leader application copies into the leader management object.
In an optional embodiment, the determining module 130 is further configured to obtain a first number of leader application copies and a second number of working nodes in the target cluster, respectively;
and determining the preset leader balance distribution condition according to the first quantity and the second quantity.
In an alternative embodiment, each of the resource observation vectors further comprises: the CPU resource occupancy rate of the corresponding first application copy, and/or the RAM resource occupancy rate of the corresponding first application copy.
The above-mentioned apparatus is used for executing the method provided by the foregoing embodiment, and the implementation principle and technical effect are similar, which are not described herein again.
These above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors, or one or more Field Programmable Gate Arrays (FPGAs), etc. For another example, when one of the above modules is implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
Fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application, where the electronic device may be the leader election device described above. As shown in fig. 9, the electronic device may include: a processor 210, a storage medium 220, and a bus 230, wherein the storage medium 220 stores machine-readable instructions executable by the processor 210, and when the electronic device is operated, the processor 210 communicates with the storage medium 220 via the bus 230, and the processor 210 executes the machine-readable instructions to perform the steps of the above-mentioned method embodiments. The specific implementation and technical effects are similar, and are not described herein again.
Optionally, the present application further provides a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the computer program performs the steps of the above method embodiments. The specific implementation and technical effects are similar, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present application. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (10)
1. A leader election method based on multiple copies is applied to a target cluster, and the target cluster comprises the following steps: the method comprises the following steps that a plurality of working nodes comprise a plurality of first application copies corresponding to target application programs, and the method comprises the following steps:
acquiring a plurality of resource observation vectors at the current moment through each first application copy, wherein each resource observation vector comprises communication round-trip time between the corresponding first application copy and other first application copies;
acquiring a sum of predicted submission delays corresponding to the first application copies at the current moment according to the plurality of resource observation vectors and a preset predictor at the current moment, wherein the preset predictor is used for predicting and acquiring the predicted submission delay of each first application copy initiating a request to a leader application copy at the previous deadline;
and determining a first target leader application copy in the plurality of first application copies at the current moment according to the sum of the predicted submission delays corresponding to the first application copies at the current moment.
2. The method of claim 1, wherein before obtaining a predicted commit delay sum corresponding to each of the first application copies at a current time according to the plurality of resource observation vectors and a preset predictor at the current time, the method further comprises:
acquiring a resource observation vector of each first application copy and a submission delay time corresponding to each first application copy in a history deadline time period corresponding to a history deadline leader application copy, and taking a plurality of first application copies as a plurality of training samples, wherein the submission delay time is used for representing the delay time of a request message sent by the first application copy to a history deadline leader application copy forwarding client;
and training to obtain the predictor according to a plurality of training samples.
3. The method of claim 1, wherein the working node further comprises: determining a first target leader application copy from the plurality of first application copies at the current time according to the predicted submission delay sums corresponding to the first application copies at the current time, wherein the determining the plurality of second application copies corresponding to other application programs comprises:
determining at least one first candidate leader application copy in the plurality of first application copies at the current moment according to the sum of the predicted submission delays corresponding to the first application copies at the current moment;
and determining a first target leader application copy in the at least one first candidate leader application copy according to the at least one first candidate leader application copy and a preset leader equilibrium distribution condition, wherein the preset leader equilibrium distribution condition is used for representing the maximum number of leader application copies in the working nodes.
4. The method of claim 3, wherein the target cluster is configured with a leader management object, the leader management object comprising: the determining a first target leader application copy among at least one of the candidate leader application copies according to the at least one candidate leader application copy and a preset leader balancing distribution condition comprises:
acquiring the number of application copies of the leader on each working node at the current moment through the leader management object;
acquiring target working nodes meeting preset leader balance distribution conditions according to the number of application copies of the leader on each working node at the current moment;
determining a first target leader application copy among at least one of the candidate leader application copies based on at least one of the candidate leader application copy and the target worker node.
5. The method of claim 4, wherein a plurality of application copies corresponding to each type of application program are configured to correspond to a copy management object, the copy management object comprising leader record information for the corresponding plurality of application copies, the leader record information comprising: the leader applies the working node identification and update time at which the copy resides, the method further comprising:
acquiring leader record information corresponding to various application copies through the copy management object;
and acquiring the number of the leader application copies on each working node according to the leader record information corresponding to each application copy and recording the number of the leader application copies into the leader management object.
6. The method of claim 3, further comprising:
respectively acquiring a first number of application copies of a leader in a target cluster and a second number of working nodes;
and determining the preset leader balance distribution condition according to the first quantity and the second quantity.
7. The method of claim 2, wherein each of the resource observation vectors further comprises: the CPU resource occupancy rate of the corresponding first application copy, and/or the RAM resource occupancy rate of the corresponding first application copy.
8. A multi-copy based leader election device, applied to a target cluster, the target cluster comprising: a plurality of working nodes, wherein the working nodes comprise a plurality of first application copies corresponding to the target application program, and the device comprises:
a first obtaining module, configured to obtain, through each first application copy, a plurality of resource observation vectors at a current time, where each resource observation vector includes a communication round-trip time between the corresponding first application copy and other first application copies;
a second obtaining module, configured to obtain, according to the multiple resource observation vectors at the current time and a preset predictor, a sum of predicted submission delays corresponding to the first application copies at the current time, where the preset predictor is configured to predict a predicted submission delay for obtaining that each first application copy initiates a request to a previous application copy of a leader in an arbitrary period;
and the determining module is used for determining a first target leader application copy from the plurality of first application copies at the current moment according to the sum of the predicted submission delays corresponding to the first application copies at the current moment.
9. An electronic device, comprising: a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, the processor and the storage medium communicating over the bus when the electronic device is operating, the processor executing the machine-readable instructions to perform the steps of the multi-copy based leader election method according to any one of claims 1-7.
10. A computer-readable storage medium, having stored thereon, a computer program for performing, when executed by a processor, the steps of the multi-copy based leader election method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210228188.XA CN114338695B (en) | 2022-03-10 | 2022-03-10 | Leader election method, device, equipment and storage medium based on multiple copies |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210228188.XA CN114338695B (en) | 2022-03-10 | 2022-03-10 | Leader election method, device, equipment and storage medium based on multiple copies |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114338695A true CN114338695A (en) | 2022-04-12 |
| CN114338695B CN114338695B (en) | 2022-05-24 |
Family
ID=81033982
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210228188.XA Active CN114338695B (en) | 2022-03-10 | 2022-03-10 | Leader election method, device, equipment and storage medium based on multiple copies |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114338695B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115495250A (en) * | 2022-11-17 | 2022-12-20 | 北京集度科技有限公司 | Flow balancing method, electronic equipment and computer program product |
| CN116611506A (en) * | 2023-07-18 | 2023-08-18 | 中国电信股份有限公司 | User analysis model training method, user label determining method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120124412A1 (en) * | 2010-11-15 | 2012-05-17 | Microsoft Corporation | Systems and Methods of Providing Fast Leader Elections in Distributed Systems of Simple Topologies |
| CN107832138A (en) * | 2017-09-21 | 2018-03-23 | 南京邮电大学 | A kind of implementation method of the High Availabitity namenode models of flattening |
| US20180349039A1 (en) * | 2017-06-05 | 2018-12-06 | International Business Machines Corporation | Implicit leader election in a distributed storage network |
| CN110855793A (en) * | 2019-11-19 | 2020-02-28 | 南昌航空大学 | Distributed system consensus method |
| CN112822239A (en) * | 2020-12-30 | 2021-05-18 | 深圳市宝能投资集团有限公司 | Block chain consensus method, block chain consensus device, electronic equipment and storage medium |
| CN113873030A (en) * | 2021-09-24 | 2021-12-31 | 支付宝(杭州)信息技术有限公司 | Block chain consensus method, consensus node and electronic equipment |
-
2022
- 2022-03-10 CN CN202210228188.XA patent/CN114338695B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120124412A1 (en) * | 2010-11-15 | 2012-05-17 | Microsoft Corporation | Systems and Methods of Providing Fast Leader Elections in Distributed Systems of Simple Topologies |
| US20180349039A1 (en) * | 2017-06-05 | 2018-12-06 | International Business Machines Corporation | Implicit leader election in a distributed storage network |
| CN107832138A (en) * | 2017-09-21 | 2018-03-23 | 南京邮电大学 | A kind of implementation method of the High Availabitity namenode models of flattening |
| CN110855793A (en) * | 2019-11-19 | 2020-02-28 | 南昌航空大学 | Distributed system consensus method |
| CN112822239A (en) * | 2020-12-30 | 2021-05-18 | 深圳市宝能投资集团有限公司 | Block chain consensus method, block chain consensus device, electronic equipment and storage medium |
| CN113873030A (en) * | 2021-09-24 | 2021-12-31 | 支付宝(杭州)信息技术有限公司 | Block chain consensus method, consensus node and electronic equipment |
Non-Patent Citations (3)
| Title |
|---|
| AIKY哇: "CockroachDB的raft优化", 《CSDN》 * |
| FARHAD SOLEIMANIAN GHAREHCHOPOGH: "A Survey and Taxonomy of leader election algorithms in distributed systems", 《INDIA JOURNAL OF SCIENCE AND TECHNOLOGY》 * |
| 赵亚辉: "混合云环境下动态副本创建与一致性策略研究", 《知网》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115495250A (en) * | 2022-11-17 | 2022-12-20 | 北京集度科技有限公司 | Flow balancing method, electronic equipment and computer program product |
| CN115495250B (en) * | 2022-11-17 | 2023-03-10 | 北京集度科技有限公司 | Flow balancing method, electronic equipment and computer program product |
| CN116611506A (en) * | 2023-07-18 | 2023-08-18 | 中国电信股份有限公司 | User analysis model training method, user label determining method and device |
| CN116611506B (en) * | 2023-07-18 | 2023-10-03 | 中国电信股份有限公司 | User analysis model training method, user label determining method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114338695B (en) | 2022-05-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114338695B (en) | Leader election method, device, equipment and storage medium based on multiple copies | |
| US10673763B2 (en) | Learning or emulation approach to traffic engineering in information-centric networks | |
| Shah et al. | On the design of adaptive and decentralized load balancing algorithms with load estimation for computational grid environments | |
| US8024740B2 (en) | Acquisition system for distributed computing resources | |
| US10055262B1 (en) | Distributed load balancing with imperfect workload information | |
| US8499069B2 (en) | Method for predicting performance of distributed stream processing systems | |
| Dam et al. | Genetic algorithm and gravitational emulation based hybrid load balancing strategy in cloud computing | |
| US20190319881A1 (en) | Traffic management based on past traffic arrival patterns | |
| US8805991B1 (en) | System and method for routing service requests | |
| Wada et al. | Evolutionary deployment optimization for service‐oriented clouds | |
| US20060200251A1 (en) | Systems and methods for optimal component composition in a stream processing system | |
| US10715638B2 (en) | Method and system for server assignment using predicted network metrics | |
| JP2008027442A (en) | Sub-task processor distribution scheduling | |
| US9501326B2 (en) | Processing control system, processing control method, and processing control program | |
| Nahir et al. | Replication-based load balancing | |
| WO2015194182A1 (en) | Service chain management apparatus, service chain management system, service chain management method, and program recording medium | |
| JP2012222402A (en) | Relay device, relay program, and relay method | |
| JP4265377B2 (en) | Load balancing method, apparatus, system, and program | |
| CN113010313A (en) | Load balancing method and device, electronic equipment and computer storage medium | |
| CN110601994A (en) | Load balancing method for micro-service chain perception in cloud environment | |
| Mostafa | Cooperative fog communications using a multi-level load balancing | |
| WO2018037930A1 (en) | Data store device and data management method | |
| Saravanakumar et al. | A novel load balancing algorithm for computational grid | |
| US7676561B1 (en) | Method and apparatus for adaptive load balancing | |
| Zhou et al. | TAP: Timeliness‐aware predication‐based replica selection algorithm for key‐value stores |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |