WO2012055243A1

WO2012055243A1 - Method and apparatus for implementing load balancing of distributed hash table network

Info

Publication number: WO2012055243A1
Application number: PCT/CN2011/075564
Authority: WO
Inventors: 胡学川; 李漓春
Original assignee: 中兴通讯股份有限公司
Priority date: 2010-10-27
Filing date: 2011-06-10
Publication date: 2012-05-03
Also published as: CN102457428B; CN102457428A

Abstract

The present invention provides a method and an apparatus for implementing load balancing of a Distributed Hash Table (DHT) network, wherein the method includes: a load space corresponding to each first virtual identifier of N first virtual identifiers distributed to each node in the DHT network is divided into M portions equally, and M×N second virtual identifiers are respectively used to identify each portion of the load space after equal division, wherein M and N are natural numbers, and M≧2; the M×N second virtual identifiers are distributed to each node in the current DHT network, and each node is responsible for the load space corresponding to the second virtual identifier distributed to itself. The present invention can implement data storage load balancing of DHT networks, and meanwhile reduce workloads of virtual identifier management and virtual identifier transfer.

Description

The present invention relates to the field of communications, and in particular to a load balancing implementation method and apparatus for a distributed hash table (DHT) network . BACKGROUND Currently, a distributed hash table (DHT) has become a main organization form of a peer-to-peer (P2P) overlay network, and its storage load balancing is mainly implemented by using a virtual identifier. In the distributed hash overlay network, there are two different attributes of the node ID and the virtual identifier. The node ID is used to distinguish different nodes and maintain a distributed hash table overlay network. The virtual identifier is used to divide the hash table space into multiple intervals and allocate them to nodes. In a distributed hash table overlay network, each node ID is different, and each virtual identifier is also different. Allows for the case where the node ID is the same as the virtual ID. In the overlay network, the hash table space is divided into multiple intervals according to all virtual identifiers of all nodes and a certain segmentation rule. The number of intervals is equal to the number of virtual identifiers, and each interval is associated with a virtual identifier. The hash table interval is the responsibility of the node to which the virtual identity associated with it belongs. Since each node has multiple virtual identities, each node needs to be responsible for multiple hash table intervals. Each data resource in the overlay network is assigned a unique resource identifier. If the resource identifier of a data resource is located in a certain hash table interval, the data resource is located by the node corresponding to the virtual identifier that is responsible for the hash table interval. save. In this way, all node IDs and virtual identifiers in the overlay network form two different logical loops. Virtual logo ^! The hash table space is divided into finer divisions, and each virtual identifier is responsible for the corresponding hash block, and the data resource whose resource identifier falls in the hash block is stored by the virtual identifier, thereby improving the network load balancing effect. effect. As shown in FIG. 1, it is assumed that the hash space is 0-63 and is equally divided into 12 by 12 virtual identifiers. There are four nodes A, B, C and D in the network. The four nodes have their own node IDs: Nl, N9, N28, N49, and assume that the four nodes have the same capabilities. Then each node has three virtual identifiers, such as Node B has three virtual identifiers: P8, P28, and P43. The relationship between the resource object's resource identifier and the virtual identifier is saved by the corresponding virtual identifier (for example, if the resource identifier is K5, the virtual identifier P8 is responsible). As shown in Table 1, the hash blocks of the three virtual identifiers of the node B are: (1, 8], (23, 28], (38, 43], and the resources of the resource identifier in this space belong to Node B is responsible. Therefore, under this kind of load balancing method, the data resource space stored in the four nodes is 15 . The details are shown in Table 1. Since the resource identifiers are uniformly distributed throughout the hash space, the data resources that each node is responsible for are balanced. Table 1.

The inventors have found that in the related art, since the virtual identification load space is aliquoted, the theoretical load space value of the node is hardly equal to the sum of the space occupied by the X virtual identifiers. Therefore, the actual space of the node is different from the space responsible for the theory. When there are fewer network scale nodes, the node theory is responsible for a larger space, and the node is actually responsible for more or less one virtual identifier. The maximum deviation of the load balance is still relatively small. . However, with the continuous development of the business, the scale of the nodes continues to expand, and the theoretical responsibility space of each node is continuously smaller. At this time, if the node is actually responsible for one virtual identifier, the load space corresponding to each virtual identifier is fixed. The maximum difference in load balancing is relatively large, which will cause the load imbalance of the node to exceed the load imbalance threshold, and the load between nodes is not balanced. SUMMARY OF THE INVENTION A primary object of the present invention is to provide a load balancing implementation scheme for a DHT network to address at least the above-mentioned problem of load imbalance between nodes of a DHT network due to an increase in nodes. According to an aspect of the present invention, a load balancing implementation method for a distributed hash table network is provided, including: first, each of the N first virtual identifiers that have been allocated to each node in the distributed hash table network The load space corresponding to the virtual identifier is divided into M shares, and each of the load spaces after the equalization is identified by using the second virtual identifier, where Μ and Ν are natural numbers, and Μ ≥ 2; The second virtual identifiers are assigned to the respective nodes in the current DHT network, and each node is responsible for the load space corresponding to the second virtual identifier assigned thereto. Before the load space corresponding to each first virtual identifier is divided into M shares, the method further includes: the load imbalance of one node in the DHT network exceeds a load imbalance degree threshold. Allocating the MXN second virtual identifiers to each node in the current DHT network includes: adjusting the number of second virtual identifiers that are responsible for each node according to the load imbalance of each node in the current DHT network, so that the load imbalance is exceeded. The load imbalance of nodes with load imbalance thresholds decreases. Assigning MXN second virtual identifiers to each node in the current DHT network includes: adding a new node to the DHT network; calculating the number of second virtual identifiers that the new node should be responsible for according to the capabilities of the new node; The second virtual identifier, which is responsible for the node whose second virtual identifier number exceeds the theoretical value, is assigned to the new node, so that the number of second virtual identifiers that the new node is responsible for is close to the number of second virtual identifiers that the new node should be responsible for. Before the load space corresponding to each first virtual identifier is divided into M shares, the method further includes: adding a new node to the DHT network. Allocating the MXN second virtual identifiers to each node includes: calculating the number of second virtual identifiers that the new node should be responsible for according to the capability of the new node; and the node responsible for the second virtual identifier number in the DHT network exceeding the theoretical value is responsible for Part of the second virtual identity is assigned to the new node, so that the number of second virtual identities that the new node is responsible for is close to the number of second virtual identities that the new node should be responsible for. Allocating a part of the second virtual identifier that is responsible for the second virtual identifier number in the DHT network to the new node is responsible for: including: according to the capabilities of each node in the DHT network, calculating the number that each node should be responsible for after joining the new node The theoretical value of the second virtual identifier; determining whether the number of the second virtual identifier currently responsible for each node exceeds the theoretical value of the node, and if so, randomly selecting all of the second virtual identifiers responsible for the node beyond the theoretical value portion The second virtual identity is assigned to the new node. When the DHT network is initially built, N is determined according to the node planning and load imbalance threshold of the DHT network. Determine N by: obtaining the node with the worst capability in the DHT network; determining the number N of the first virtual identifier according to the load unbalance threshold, so that the load imbalance of the node with the worst capability is less than the load imbalance Threshold. According to another aspect of the present invention, a load balancing implementation apparatus for a DHT network is provided, including: a splitting module, configured to correspond to each of the first virtual identifiers of the N first virtual identifiers that have been allocated to the respective nodes in the DHT network The load space is divided into M parts, and each of the load spaces after the equalization is identified by using the second virtual identifier, wherein Μ and Ν are natural numbers, and Μ ≥ 2; the allocation module is set to be the first The two virtual identifiers are assigned to the respective nodes of the current DHT network, and each node is responsible for the load space corresponding to the second virtual identifier assigned thereto. The device further includes: a determining module, configured to determine whether the DHT network has a node whose load imbalance exceeds a load imbalance degree threshold, and if yes, trigger the splitting module. The apparatus further includes: a discovery module configured to discover whether a new node is added to the DHT network, and if so, triggering the splitting module. With the present invention, the virtual identifiers assigned to the respective nodes are split, so that the load space corresponding to each virtual identifier is reduced, so that the difference between the theoretical load space of each node and the actual load space is reduced, thereby ensuring DHT. Load balancing of each node in the network. BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are set to illustrate,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, In the drawings: FIG. 1 is a schematic diagram of a single-hop DHT network according to the related art using a virtual identifier to balance a load; FIG. 2 is a load balancing diagram in which the number of virtual identifiers in the related art is much larger than the number of nodes; FIG. 3 is a schematic diagram of the load balancing according to the present invention; FIG. 4 is a schematic structural diagram of a device for load balancing implementation of a preferred DHT network according to Embodiment 1 of the present invention; FIG. 5 is a schematic structural diagram of a load balancing implementation apparatus of another preferred DHT network according to Embodiment 1 of the present invention; FIG. 6 is a flowchart of a load balancing implementation method of a DHT network according to Embodiment 2 of the present invention; FIG. 8 is a flowchart of a new node joining and virtual identity splitting of a distributed topology network according to Embodiment 4 of the present invention; FIG. 9 is a flow chart of the virtual identification splitting of the centralized topology network according to the fifth embodiment of the present invention; FIG. 10 is a schematic diagram of the virtual identification before splitting according to the sixth embodiment of the present invention; FIG. 12 is a schematic diagram of a split virtual identifier assigned to a new node according to Embodiment 6 of the present invention. FIG. BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. It should be noted that the embodiments in the present application and the features in the embodiments may be combined with each other without conflict. In order to facilitate the understanding of the present invention, the definition of the node load related parameters involved in the present invention will be described below first: Node capacity one C: A node with a large capacity can bear more load. In practical applications, the capacity of a node can be measured by resources such as the CPU, memory, bandwidth, and connection stability of the node. Specific measures can be measured by means of real-risk measurement, risk-of-risk data or dynamic updates. In general, the capacity of a node is limited by the most resource-intensive resources of its CPU, memory, bandwidth, and so on. In the embodiment of the present invention, the ID space is used to measure the actual load of the node. Therefore, the node capacity G can be expressed as the size of the load space (which can be simply referred to as ID space) corresponding to the virtual identifier (ID) that the node is responsible for. For example, the node with the lowest performance in the DHT network can be defined as a reference value, denoted as C _mm , and the capacity C of other nodes can be expressed as mC _mm . Where m is the performance ratio of the node to the worst performing node. The load imbalance of the node - LB: indicates the degree of deviation between the actual load of the node and the equilibrium load under ideal conditions. In the embodiment of the present invention, the size of the ID space that the node is responsible for represents the section. The storage load of the point. Ideally, the size of the ID space that the node is actually responsible for is equal to the size of the ID space that should be responsible for its node capacity. Therefore, the load imbalance of a node can be defined as: The size of the ID space that the node is actually responsible for/the size of the ID space that the node i should be responsible for - 1 where the size of the ID space that the node is actually responsible for is equal to all the virtual IDs owned by the node. The sum of the size of the ID space. The size of the ID space that the node should be responsible for should be based on the node capacity of all nodes in the network. The size of the ID space that node i should be responsible for is:

N ',"

k=\ , where Ν is the number of nodes in the network and S is the total ID space size. Load Unbalance Threshold - LN: The equilibrium goal to be achieved by the load balancing algorithm. That is, the absolute value of the load imbalance LB of all nodes in the network should be less than the threshold LN. Number of virtual nodes - V: Indicates the number of virtual nodes (ie, virtual IDs) owned by each physical node. For example, you can define the worst performing node in the network (node capacity is c _mm ). The number of virtual nodes owned by the node is V _mm and the node with the node capacity of mC _mm is [mV _mm ]. Embodiment 1 FIG. 3 is a schematic structural diagram of a device for load balancing implementation of a DHT network according to an embodiment of the present invention. The device may split a virtual identifier assigned by each node in a DHT network when the DHT network is expanded. As shown in FIG. 3, the device mainly includes: a splitting module 10 and a distribution module 20. The splitting module 10 is configured to divide the load space corresponding to each of the first virtual identifiers of the Z first virtual identifiers that have been allocated to the nodes in the DHT network into M shares, and use the second virtual identifier pair. Each of the load spaces after the equalization is identified, wherein Μ and Ν are natural numbers, JL Μ>2; and the allocation module 20 is configured to allocate the second virtual identifiers to each node of the current DHT network, Each node is responsible for the load space corresponding to the second virtual identifier assigned thereto. For example, if the number of the first virtual identifiers that have been assigned to the node i is η, the splitting module 10 may divide the load space corresponding to each of the first virtual identifiers that have been assigned to the node i into M shares, and add ηχ (Ml) virtual identifiers, the newly added ηχ(M-l) virtual identifiers and the original n first virtual identifiers identify some of the split load spaces. In a preferred embodiment of the present invention, as shown in FIG. 4, the apparatus may further include: a determining module 30, configured to determine whether the DHT network has a node whose load imbalance exceeds a load imbalance degree threshold, If so, the splitting module 10 is triggered. In an actual application, after the first virtual identifier is allocated to each node of the DHT network, when a new node joins the DHT network, the allocation module 20 may determine, according to the capability of the new node, the first node that the new node should be responsible for. The number of virtual identities, and then calculate the difference between the number of first virtual identities that the original nodes should be responsible for after the new node joins the DHT network and the number of first virtual identities that each node is actually responsible for. The first virtual identifier of the virtual identifier is allocated to the new node, and after the allocation is completed, the load unbalance degree of each node is calculated, and the determining module 30 determines whether the load imbalance of each node exceeds the load unbalance degree threshold; Alternatively, after the initial construction of the DHT network is completed, the load imbalance of each node is calculated, and the determining module 30 determines whether the load imbalance of each node exceeds the load unbalance threshold. When the determining module 30 determines that the load imbalance of one or more nodes exceeds the load unbalance degree threshold, the splitting module 10 is triggered, and the splitting module 10 divides the load space corresponding to each first virtual identifier that each node is responsible for. Μ ,, and newly added ( Μ - 1 ) 虚拟 a virtual logo, using the newly added ( Μ - 1 ) 虚拟 a virtual id and the original 虚拟 first virtual identities to identify the divided load spaces. After the virtual module is configured, the splitting module 10 can directly trigger the allocating module 20, and the allocating module 20 adjusts the number of second virtual identifiers that each node is responsible for according to the load imbalance degree of each node, so that the load imbalance is more than the load. The load imbalance of the nodes of the equalization threshold is reduced. For example, suppose there are three nodes in the DHT network: Α, Β, and C. The entire hash space is 0-63, and the load imbalance degree threshold is 10%. Before splitting, the entire hash space is evenly divided into 8 parts. 8 virtual ID identifiers, assuming each node's capabilities are c, then each node theory should be responsible for the space: c / (c + c + c) * 64 = 21.33, therefore, the number of virtual IDs assigned by the three nodes is 3, 3, 2, the load imbalance of each node is: 12.49 %, 12.49 % and - 14.98%, the judgment module 30 judges that the load imbalance has exceeded the load imbalance threshold, triggers the split module 10, splits The module 10 divides the load space corresponding to the three virtual identifiers that the A node is responsible for into 12, and is identified by 12 virtual identifiers. The load space corresponding to the three virtual identifiers that the node B is responsible for is divided into 12 copies, and 12 virtual The identifier is identified, and the load space corresponding to the two virtual identifiers that the C node is responsible for is divided into eight, which are identified by eight virtual identifiers, and then the distribution module 20 adjusts the number of virtual identifiers that each node is responsible for, and takes the node A and the node. B is responsible A virtual identifier assigned to the node C, so that the load of each node of the inequality is: 3.1%, 3.1% and -6.2%, less than threshold load imbalance. Or, the splitting module 10 is performing virtual After the identification, the allocation module 20 may not be triggered. When the new node joins, the allocation module 20 calculates the number of the second virtual identifiers that the new node should be responsible for according to the capabilities of the new node, and then The portion of the second virtual identifier that is responsible for the second virtual identifier in the DHT network is responsible for the portion of the second virtual identifier assigned to the new node, and the number of the second virtual identifier that the new node is responsible for. Approaching the number of the second virtual identities that the new node should be responsible for. In another preferred embodiment of the present invention, as shown in FIG. 5, the apparatus may further include: a discovery module 40, configured to discover whether the DHT network has a new node to join, and if yes, trigger the splitting Module 10. After the splitting module 10 performs the splitting function, the allocating module 20 may first calculate the number of second virtual identifiers (ie, theoretical values) that each node should be responsible for after the new node joins, and then the second virtual identifier that is actually responsible for exceeding the theoretical value. The second virtual identifier that is responsible for the node is taken out (the number of second virtual identifiers that are actually responsible for - the number of second virtual identifiers that the node should be responsible for) the second virtual identifiers are assigned to the new node. In practical applications, the above device may be set in the management service node of the DHT network, or may be set in other nodes of the DHT network (for example, the responsible node of the new node;). The above device provided by the embodiment of the present invention can be ensured by splitting the virtual identifier.

Load balancing of each node in the DHT network. Embodiment 2 FIG. 6 is a flowchart of a method for implementing load balancing of a DHT network according to an embodiment of the present invention. The method may be implemented by the apparatus described in Embodiment 1 above. As shown in FIG. 6, the method mainly includes the following steps: Step S602, the load space corresponding to each of the first virtual identifiers of the Z first virtual identifiers that have been allocated to each node in the DHT network is equally divided. M shares, using a second virtual identifier to identify each load space after sharing, where Μ and Ζ are natural numbers, and Μ ≥ 2; where Ζ is the number of virtual identities in the current DHT network, ie DHT The number of virtual spaces that are evenly divided across the entire load space in the network. For example, when the DHT network is initially built, the node can be determined according to the node planning of the DHT network and the preset load imbalance threshold. The node planning of the DHT network includes but is not limited to: the number of nodes in the DHT network and each The capabilities of the node. When the node capabilities are different, the theoretical value of the virtual ID of each node load is proportional to its capability. Since the virtual ID that the node is actually responsible for is at most one deviation from its theoretical value, the node with the smallest capacity has the largest load imbalance. Therefore, in the embodiment of the present invention, when determining the virtual IZ in the DHT network, it is only necessary to ensure that the total number of Zs can enable the node with the smallest capacity to meet the load balancing requirement, even if the load imbalance of the node with the smallest capacity is less than Load imbalance threshold. For node i of the DHT network, the following formula needs to be satisfied: (the size of the ID space that node i is actually responsible for / the size of the ID space that node i should be responsible for) - 1 < load imbalance threshold. Assuming that the number of virtual identifiers is Z, the node i with the smallest capacity is overloaded with a Δ space (S/Z), where S is the entire load space of the DHT network, and Z can be determined by the following formula:

Solution, available:

k

C _t xLN Preferably, Z may take a value of 2 to the power of n. For example, the initial network planning scale has 100 nodes, and the maximum load imbalance LN is controlled below 10%. If the node capacity capacity value C is the same, according to the formula, the number of virtual identifiers Z must be greater than 1000. When the virtual load is used to identify the equally divided load spaces, M-1 new virtual identifiers may be generated in the middle of the two connected pre-split virtual identifiers, and the two connected splits are used. The first virtual identifier in the virtual identifier and the M-1 new virtual identifiers are used as the M split virtual identifiers (ie, the second virtual identifier;), and the M split virtual identifiers are used to The load space corresponding to the virtual identifier is identified by the M payload space. In an actual application, when it is found that the load imbalance of a node in the DHT network exceeds the load unbalance threshold, step S602 is triggered. Alternatively, it is also possible to trigger step 4 S602 when a new node joins in the DHT network. In step S604, the second virtual identifiers are allocated to the respective nodes in the current DHT network, and each node is responsible for the load space corresponding to the second virtual identifiers allocated thereto. If it is found that the load unbalance degree of a node in the DHT network exceeds the load unbalance degree threshold to trigger the foregoing step S602, the second virtual identifier that is responsible for each node may be according to the load unbalance degree of each node in the current DHT network. The quantity is adjusted to reduce the load imbalance of the node whose load imbalance is greater than the load imbalance degree threshold; or, after performing the foregoing step S602, the second virtual identifier of each node in the current DHT network may not be adjusted, and When a new node joins, the number of second virtual identifiers that the new node should be responsible for is calculated according to the capability of the new node, and then the number of the second virtual identifiers responsible for the DHT network exceeds the theoretical value. The second virtual identifier that is responsible for the node is assigned to the new node, and the number of the second virtual identifier that the new node is responsible for is close to the number of the second virtual identifier that the new node should be responsible for. . If there is a new node joining the network to trigger step S602, the ability of the new node, calculate the number of split virtual IDs that the new node i should be responsible for:

X=^^x (ZxM) ( 1 ),

Then calculate the theoretical value Xj of the split virtual identity that each node is responsible for after the new node joins: X = ^ (ZxM), k=\ and then all the second responsible for the node from the actual responsible second virtual identifier exceeding its theoretical value The virtual identifier is taken out (the number of second virtual identifiers that are actually responsible - the number of second virtual identifiers that the node should be responsible for) the second virtual identifiers are assigned to the new node. With the above method provided by the embodiment of the present invention, the virtual identifier can be automatically segmented, so that the network has scalability, and the load balancing of the nodes of the DHT network can be ensured. Embodiment 3 In the embodiment of the present invention, a centralized topology network is taken as an example to describe a splitting process of a virtual identifier. In the embodiment of the present invention, when the load unbalance degree of the node exceeds the load unbalance degree threshold in the DHT network, the splitting is performed, and after the splitting, the number of responsible virtual identifiers of each node is adjusted to make the DHT The load imbalance of nodes in the network decreases. FIG. 7 is a flowchart of splitting a virtual identifier according to an embodiment of the present invention. As shown in FIG. 7, the method mainly includes the following steps: Step S701: As a new node continuously joins the overlay network, the network scale is continuously expanded, and the load imbalance is continuously increased. When the unbalanced threshold is reached, the management service node generates an alarm; Step S702: The management service node sends an alarm notification to the decision center (such as the network management platform); Step S703: After receiving the network load imbalance alarm, the decision center may automatically The processing may also be performed by the manual number, indicating the virtual identifier in the overlay network, such as the split M; step S704: the decision center sends a command to the management service node, requesting the management service node to split the virtual identifier; Step S705: the management service node according to the indication The virtual identifier is equally divided into M parts; for example, there are 8 original virtual identifiers in the overlay network, and after equal division of 4 copies, 32 virtual identifiers in the overlay network are superimposed. Step S706: The management service node adjusts the virtual identifiers that are responsible for each node according to the imbalance degree of each node, so that the imbalance is decreased. Step S707: The management service node sends a resource data adjustment notification to the overlay network access node. Step S708: The nodes in the data are adjusted. In the embodiment of the present invention, the virtual identifier may be split when the load imbalance exceeds the threshold, thereby reducing the load imbalance of the DHT network. The fourth embodiment of the present invention uses a distributed topology network as an example to describe the joining of new nodes and the splitting of virtual identifiers. FIG. 8 is a flowchart of a new node joining and virtual identity splitting in the embodiment of the present invention. As shown in FIG. 8, the method mainly includes the following steps: Step S801: A new node generates a node ID, and locates according to the size of the node ID. The access node is the direct successor node of the new node in the current network; wherein the node ID can be obtained by hashing the feature information of the node, and the feature information of the node refers to the uniqueness information of the node. The selection of the hash algorithm may be arbitrary, such as the IP address, the network card number, or the machine serial number; or, the node ID may be uniformly allocated by the network operation manager. Step S802: The new node sends a join request message to the admission node, where the join request message carries its capability value and node identifier. Step S803: The admission node locates the responsible node of the new node by using the DHT algorithm (ie, the successor node of the new node) And then forwarding the join request message to the responsible node; Step S804: The responsible node allocates X newly split virtual identifiers to the new node according to the new node capability value; wherein, the calculation formula of X is:

, 标识 identifies the number of all virtual identities after splitting for the entire overlay network. Specifically, in this step, the responsible node may first split the virtual identifiers responsible for each node, and then calculate the number of split virtual identifiers that each node is responsible for after the new nodes in the overlay network are joined, and the nodes will be in each node. The number of virtual IDs that are currently responsible for the split, minus the number of split virtual IDs that the theory is responsible for, and the number of split virtual identifiers that the node should assign to the new node (assumed to be R), and then The R virtual identifiers randomly obtained from the node are allocated to the new node until the X nodes are reached; or, the responsible node may first calculate the number of virtual identifiers in the overlay network that are theoretically responsible for each node after the new node joins. The number of original virtual identifiers in each node minus the difference between the number of theoretically responsible virtual identifiers is the number of virtual identifiers (set to R) that each node should assign to the new node. Randomly selecting R virtual identifiers from each node to assign virtual identifiers to new nodes until X's are reached (where X' is the number of original virtual identifiers that the new node calculated by the new node's ability is responsible for) There are festivals If the load imbalance of the point is greater than the load unbalance threshold, the virtual identifier is split, and then the virtual identifiers responsible for each node are adjusted to reduce the load imbalance. Steps S805, S806: Responsible for the node to return the response message of the new node joining request; Step S807: The responsible node instructs the relevant node to perform resource migration adjustment; Step S808: The responsible node notifies the node in the overlay network to update the query routing table; Step S809: Each node in the overlay network updates its respective query routing table according to the indication. In the embodiment of the present invention, the virtual identifier may be split when the new node joins, so that the load imbalance of the DHT network after the new node joins is not excessive. Embodiment 5 The embodiment of the present invention uses a centralized topology network as an example to describe a splitting process of a virtual identifier. In the embodiment of the present invention, when the load unbalance degree of the node exceeds the load unbalance degree threshold in the DHT network, the splitting is performed, and after the splitting, the number of responsible virtual identifiers of each node is not adjusted, and After a new node is added, the virtual space is allocated by the split virtual identifier. FIG. 9 is a flowchart of splitting and allocating virtual identifiers according to an embodiment of the present invention. As shown in FIG. 9, the method includes the following steps: Step S901: As new nodes continuously join the overlay network, the network scale is continuously expanded and the load is unbalanced. The degree is continuously increased. When the load imbalance threshold is reached, the management service node generates an alarm. Step S902: The management service node sends an alarm notification to the decision center (such as the network management platform). Step S903: The decision center receives the network load imbalance. After the alarm, the alarm may be automatically processed, or may be performed by the manual number, indicating the virtual identifier in the overlay network, etc.; Step S904: The decision center sends a command to the management service node, requesting the management service node to split the virtual identifier; Step S905: Management service According to the indication, the node splits the virtual identifier into M shares. For example, the overlay network has the original 8 virtual identifiers, and after equally splitting 4 copies, 32 virtual identifiers in the overlay network are superimposed. Step S906: The management service node receives the request message with the new node joining; Step S907: The management service node allocates the X split virtual identifiers to the new node according to the new node capability value; wherein, the calculation formula of X is:

, 标识 identifies the number of all virtual identities after splitting for the entire overlay network. Similarly, the service management node calculates the number of virtual identities that each node in the overlay network is responsible for. The number of virtual identifiers currently responsible for each node minus the difference between the number of theoretically responsible virtual identifiers is the number of virtual identifiers (set to R) that each node should assign to the new node. R virtual identities assigned to the new node are randomly selected from each node until X is reached. Step S908: The management service node sends a resource data adjustment notification to the overlay network access node. Step S909: The node in the overlay network performs data adjustment. Embodiment 6 The embodiment of the present invention describes a virtual identity split provided by an embodiment of the present invention by using a specific example. In the embodiment of the present invention, the entire hash space (assumed to be 0-63) is assumed, and the load imbalance degree threshold (LN) is 10%. As shown in Figure 9, the network has two nodes, A and B. The capabilities of the nodes are the same, and the entire hash space is evenly split by 8 virtual identifiers. The virtual identifiers P0, P16, P32, and P48 are assigned to the node A, and the virtual identifiers P8, P24, P40, and P56 are assigned to the node B. Because the two nodes have the same capability, each node is assigned 4 virtual identifiers, and the virtual identifier is responsible for the same space, so the storage load of the two nodes is balanced. When a new node C (capable of the same as A and B) requests to join the overlay network, then each node theory should be responsible for the space: Node A = Node B = Node C = c / (c + c + c) * 64 Since the identity is indivisible, the assigned virtual identities of the three nodes are: 3, 3, 2, at which point two nodes are actually responsible for 24 and the other is 16. Then the load imbalance of each node is 12.49%, 12.49% and - 24.98%. The load imbalance has exceeded the threshold and a split virtual ID is required. If the load space corresponding to each virtual identifier is split into M (assumed M = 4) equal parts, the entire hash space is divided by 32 virtual identifiers. As shown in Figure 10, node A is responsible for 16 virtual identifiers, and node B is responsible for 16 Virtual logo. After the port is added to the node C, the three node theoretical assignment identifiers are 11, 11 and 10. In this case, five virtual identifiers need to be extracted from the virtual identifiers responsible for the nodes A and B to be assigned to the node C. Thus, there are two nodes that are actually responsible for 22 and the other only 20. The load imbalances are 3.1%, 3.1% and -6.2%, respectively, to meet the demand. The responsible node randomly selects 5 virtual identifiers from node A and node B to allocate to the new node C, as shown in FIG. From the above description, it can be seen that, through the present invention, the entire DHT overlay network system can well implement data storage load balancing, and the virtual identifier can be automatically segmented according to the scale of the network node, thereby reducing the work of identity management and identity migration. The amount, at the same time, makes the network scalable. Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general-purpose computing device, which can be concentrated on a single computing device or distributed over a network composed of multiple computing devices. Alternatively, they may be implemented by program code executable by the computing device, such that they may be stored in the storage device by the computing device and, in some cases, may be different from the order herein. The steps shown or described are performed, or they are separately fabricated into individual integrated circuit modules, or a plurality of modules or steps are fabricated as a single integrated circuit module. Thus, the invention is not limited to any specific combination of hardware and software. The above is only the preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes can be made to the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the scope of the present invention are intended to be included within the scope of the present invention.

Claims

Claim

1. A distributed hash table DHT network load balancing implementation method, including:

All the load spaces corresponding to the first virtual identifiers of the N first virtual identifiers that have been allocated to the respective nodes in the DHT network are equally divided into M shares, and the shared workloads are divided by using the second virtual identifiers. Space is identified, where Μ and Ν are natural numbers, and Μ ≥ 2;

Each of the second virtual identifiers is assigned to each node in the current DHT network, and each node is responsible for the load space corresponding to the second virtual identifier assigned thereto.

The method according to claim 1, wherein before the load space corresponding to each of the first virtual identifiers is equally divided, the method further includes: a load imbalance of a node in the DHT network The load imbalance threshold is exceeded.

The method according to claim 2, wherein the allocating the second virtual identifiers to each node in the current DHT network comprises:

Adjusting the number of the second virtual identifiers that are responsible for each node according to the load imbalance of each node in the current DHT network, so that the load imbalance of the node whose load imbalance exceeds the load imbalance degree threshold is decreased. .

a new node joins the DHT network;

Calculating, according to the capability of the new node, the number of the second virtual identifiers that the new node should be responsible for;

Allocating a portion of the second virtual identifier that is responsible for the second virtual identifier number in the DHT network to a theoretical value is assigned to the new node, and the second virtual identifier responsible for the new node is responsible for The number is close to the number of the second virtual identities that the new node should be responsible for.

The method according to claim 1, wherein before the load space corresponding to each of the first virtual identifiers is equally divided, the method further comprises: adding a new node to the DHT network.

6. The method of claim 5, wherein the assigning each of the second virtual identifiers to each of the nodes comprises:

7. The method according to claim 6, wherein allocating a portion of the second virtual identifier that is responsible for a node whose number of the second virtual identifiers in the DHT network exceeds a theoretical value to the new node is responsible for including :

Calculating a theoretical value of the second virtual identifier that each of the nodes should be responsible for after joining the new node according to the capabilities of each node in the DHT network;

Determining whether the number of the second virtual identifiers currently being responsible for each of the nodes exceeds the theoretical value of the node, and if so, randomly selecting all of the second virtual identifiers that are responsible for the node to exceed the theoretical value portion The second virtual identity is assigned to the new node.

The method according to any one of claims 1 to 7, wherein, when the DHT network is initially built, the UI is determined according to a node plan and a load imbalance degree threshold of the DHT network.

9. The method according to claim 8, wherein the Ν is determined in the following manner:

Obtaining the least capable node in the DHT network;

And determining, according to the load imbalance degree threshold, the number of the first virtual identifiers, so that the load imbalance of the node with the worst capability is less than the load imbalance degree threshold.

10. A load balancing implementation device for a DHT network, comprising:

The splitting module is configured to divide the load space corresponding to each of the first virtual identifiers that are allocated to each node in the DHT network into two parts, and use the second virtual identifier pair The load space after the equalization is identified, wherein Μ and Ν are natural numbers, and Μ>2; And an allocating module, configured to allocate the second virtual identifier to each node of the current DHT network, where each node is responsible for a load space corresponding to the second virtual identifier allocated thereto.

The device according to claim 10, wherein the device further comprises:

The determining module is configured to determine whether the DHT network has a node whose load imbalance is greater than a load imbalance degree threshold, and if yes, trigger the splitting module.

12. The device according to claim 10, wherein the device further comprises:

The discovery module is configured to discover whether the DHT network has a new node to join, and if so, trigger the splitting module.