JP2012123544A - Load distribution device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To eliminate deviation in the management range length of data in respective node devices while suppressing the increase of storage capacity used for managing the node device participating in a system, in the system of distributing and managing the data by the plurality of node devices.SOLUTION: Each node device 1 constituting a Key-Value type data management system by a distribution hash table includes a load distribution device 10. When a node table for managing the node device 1 participating in the data management system is updated, the load distribution device 10 writes correspondence of the node ID of the node device 1 and an address to a node table for key retrieval. The load distribution device 10 obtains the management range length of a key in each node device 1 on the basis of the node table for the key retrieval, allocates a virtual node to the node device 1 of the shortest management range length until the management range lengths get nearly equal, and adds the address of the node device 1 and the node ID of the virtual node to the node table for the key retrieval.

Description

  The present invention relates to load distribution in a distributed database system and a distributed file system, and more particularly to a load distribution apparatus and program for a key-value data management system based on DHT (Distributed Hash Table).

  There are a distributed database system, a distributed file system, and the like as a system for managing data and files by regarding a plurality of nodes connected via a network as if they are apparently regarded as one storage device. In such a distributed database system or distributed file system, data including a combination of “key” and “value” (hereinafter, a combination of “key” and “value” is described as (key, value)) is managed. A key-value type data management system is used. As a method for realizing a large-scale key-value type data management system, DHT has been proposed. In DHT, (key, value) is distributed and managed in a plurality of nodes constituting the system.

  Chord is a typical DHT system (see, for example, Non-Patent Document 1). In Chord, a node participating in the system generates a unique node ID in the system from a node address or the like using a hash function, and forms an overlay network in which the node IDs are arranged in a circle in the order of values. When storing (key, value) in the system, the node generates a key ID from the key using a hash function, and the node (key, value) has a node ID close to the key ID in the clockwise direction on the overlay network. ). When a node searches for a key, a key ID is generated from the key using a hash function, a node having a node ID close to the key ID is found, and a value corresponding to the key is acquired from the found node.

  For example, “key” is a data name of data to be distributed and managed by the system, and “value” is an address of a node storing the data. The node storing the data generates a key ID from the data name by a hash function, and a node having a node ID close to the key ID in the clockwise direction on the overlay network, a “key” indicating the data name and the own node The association with the “value” indicating the node address is registered. The usage node that uses the data generates a key ID from the data name of the data that it wants to use, and queries the node with a node ID close to the clockwise direction to this key ID to obtain a “value” corresponding to the “key”. To do. The using node can know the address of the node storing the data from the acquired “value”.

  In DHT, it is necessary to find a node having a node ID close to the key ID. As a node discovery method, each node participating in the system has a node table that manages a part of the nodes participating in the system, and the node table is distributed and stored throughout the system. And a method having a complete node table in which all nodes participating in the system are registered in all nodes.

In the method of storing node tables in a distributed manner, it is possible to configure a large-scale system consisting of a large number of nodes. The method of storing complete node tables is a method of storing node tables in a distributed manner. Compared to, it is difficult to configure a large-scale system. On the other hand, in the method of storing the node table in a distributed manner, when searching for a key, it takes time to search while routing a plurality of nodes. However, in the method having a complete node table, there is no need for routing and O ( 1) Search is possible. In other words, a constant order can be searched. Chord is a method of storing the node table in a distributed manner at each node. If the number of nodes participating in the system is N, the number of hops until the node managing the key is found is O (log ₂ N) (logarithm order of N).

  Non-Patent Document 2 and Non-Patent Document 3 have been proposed as methods for storing a complete node table. In a method in which all nodes store a complete node table in common, when a node joins or leaves the system, all nodes modify the node table accordingly. Therefore, in Non-Patent Document 2, nodes of Slice Leader and Unit Leader are allocated while the hash space is divided, and notification of participation or withdrawal is sent to all nodes hierarchically via those nodes. . Each node periodically checks the existence of the nodes in the node table and the adjacent nodes in order to keep the node table correct. In Non-Patent Document 3, each node randomly selects a node from the node table and periodically exchanges the node table.

  The range managed by each node in the hash space in DHT is from the ID of an adjacent node (predecessor) in the reverse direction to the clockwise direction to the own node ID. The node ID generated by using a hash function such as SHA-1 is a random value, but in general, the generated node ID is biased. The bias of the node ID on the hash space is a bias of the management range length of the key ID managed by each node, and as a result, the difference in the amount of data (key, value) managed by the nodes participating in the system. Will occur.

  There is a virtual node method as a method for dealing with this. In the virtual node method, one node generates a plurality of different node IDs and participates in the overlay network, thereby equalizing the key management range length of each node. The number of virtual nodes to be generated is an empirical value of log (N) per node in Non-Patent Document 1, and 60 per node in Non-Patent Document 3. As a method for generating a node ID of a virtual node, a predetermined method, for example, a method of generating from a value obtained by adding 1, 2, 3,... As proposed, there is a method of arranging the nodes at equidistant positions in the hash space from the original node ID.

  The virtual node method can be used not only to equalize the key management range length of each node but also to adjust the key management range length of the node according to the performance when the node performance differs. . That is, by increasing the number of virtual nodes of a node with high performance, the key management range length of that node can be increased. However, when the number of virtual nodes is controlled according to the difference in node performance, it is necessary to increase the number of virtual nodes if the performance difference between the nodes in the system is large. Therefore, problems such as an increase in node table size and an increase in management overhead occur. In order to solve this, in Patent Document 1, a boundary managed by a node on a hash space is adjusted between nodes according to a performance difference between adjacent nodes.

JP 2004-252663 A

I. Stoica et al., “Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications,” Proc. ACM SIGCOMM, 2001 Kaneko et al., “OneHop-P2P extension method considering node locality and management fairness,” FIT2008, no.4, RL-006, 2008 Maebashi et al., “Implementation and application of distributed systems for large-scale data processing”, IEICE D, Vol.J93-D, No.7, 2010

As described above, in the DHT, when the node ID generated using the hash function is used, there is a bias in the arrangement of the nodes on the hash space, and as a result, there is a difference in the key management range length managed by each node. Arise. In order to solve this problem, a method of assigning a virtual node to each node is used. However, if a fixed number of virtual nodes are allocated to each node, the required storage capacity increases due to an increase in the node table size, an increase in management overhead, and the like, resulting in a resource load. Therefore, there is a problem in unnecessarily allocating a large number of virtual nodes.
Further, even if the allocation range between adjacent nodes is adjusted as in Patent Document 1, if the arrangement of nodes is originally biased, the allocation range adjustment range is too narrow or too wide. For this reason, it is necessary to use a virtual node method for assigning a fixed number, which is not a fundamental solution.

  The present invention has been made in consideration of such circumstances, and in a system in which data is distributed and managed by a plurality of node devices, the storage capacity used for managing the node devices participating in the system is reduced. Provided are a load distribution device and a program capable of eliminating a bias in a data management range length in each node device while suppressing an increase.

[1] One aspect of the present invention stores a node table including node specifying information for specifying a node device constituting a data management system, and the node specified by the node specifying information and the node specifying information. A storage unit for storing a key search node table indicating a correspondence with a node ID generated for the device; and node identification information of a node device participating in the data management system is written to the node table, and the data management is performed. A node table management unit that deletes node specifying information of a node device that has left the system from the node table, a node ID set in the key search node table, and the node specifying information associated with the node ID Based on the above, the management range length of the key to be managed by each node device is calculated. A key search node table in which a correspondence between a management range length calculation unit, node identification information in the node table, and a node ID generated for the node device identified by the node identification information is set. The management range length until it is determined that the difference in management range length of each node device calculated by the management range length calculation unit based on the generated key search node table is written in the storage unit and meets a predetermined condition A new node ID is generated for the node device selected based on the node, and a correspondence between the node specifying information of the node device and the newly generated node ID is added to the key search node table. Based on the virtual node registration unit that repeats the updating process, the node ID in the key search node table, and the specified key A load balancer, characterized in that it comprises a key table processing unit for acquiring node identification information of the node device which manages a value corresponding to the key.
According to this aspect, in the key-value type data management system using the key search node table in which the node device specific information and the node ID are associated with each other in order to search for the node device that holds the value corresponding to the key. When a node device joins or leaves, the load balancer determines that the difference in management range length between node devices satisfies a predetermined condition (for example, the difference in key management range length of each node device is a predetermined value). Until the difference in key management range length of each node device weighted by the node performance value falls within a predetermined range, in other words, the key management range length for the node performance value of each node device Until the difference in the ratios falls within a predetermined range), the node of the virtual node is selected with respect to the appropriately selected node device (for example, the node device having the narrowest management range length). Repeating a process of increasing the control range key length by assigning ID, and generate a key search node table comprising a node ID generated for a particular information and the node device of the node device.
The key device management range is determined based on the node ID registered in the key search node table corresponding to the node specifying information of the node device. As the node ID assigned to the node device, a hash value calculated based on the node identification information of the node device and information unique to each node ID already assigned to the node device can be used.
As an example, the management range corresponding to the node ID is from the adjacent node ID to the own node ID. However, which range boundary belongs to is determined in advance as appropriate.
The management range length is the length of the management range described above, and the management range length of the key of each node device is the total of the lengths of the management ranges corresponding to the respective node IDs assigned to the node device.
Making the difference in the key management range length of each node within a predetermined range means that the load of each node device is made nearly equal.
Thereby, the imbalance of the key management range length of each node apparatus can be corrected efficiently, and the number of virtual nodes allocated to the entire system can be suppressed. Correcting the imbalance in the key management range length also corrects the load imbalance of each node device. Therefore, it is possible to distribute the load of each node device while suppressing an increase in the key search node table. .

[2] One aspect of the present invention is the load distribution device described above, wherein the management range length calculation unit weights the management range length of the node device according to a value indicating the performance of the node device, The virtual node registration unit determines whether a difference in the management range length weighted according to a value indicating the performance of the node device matches a predetermined condition.
According to this aspect, the load distribution device weights the key management range length in each node device by the node performance index value, and the weighted management range length is the narrowest until the weighted management range length becomes nearly equal. A node ID of the virtual node is assigned to the node device. As an example, the key management range length is weighted by multiplying the key management range length by the reciprocal of a value indicating the performance of the CPU or the reciprocal of the total disk capacity. In other words, the value of the ratio of the key management range length to the node performance value is used.
Therefore, it is possible to assign a management range length corresponding to the processing capability and memory capacity of the node device to each node device.

[3] One aspect of the present invention is the load distribution apparatus described above, wherein the management range length calculation unit is configured to perform node identification information of the node device and the node ID newly generated by the virtual node registration unit. When the association is added to the key search node table, the node ID adjacent to the node ID added to the key search node table is specified to calculate the key management range length. Update the generated management range length of the node device to a value obtained by adding the calculated management range length, and specify a node corresponding to a node ID adjacent to the node ID of the virtual node from the key search node table Information is read, and the key management range length of the node device identified by the read node identification information is subtracted from the calculated management range length. It is characterized by updating.
According to this aspect, the load distribution device stores the key management range length of each node device once calculated, and the node ID of the virtual node is added to the node device and the key search node table is updated. If so, the key management range length is calculated for the virtual node with the added node ID. Then, of the stored management range length, only the key management range length stored for the node device to which the virtual node is added and the node device corresponding to the node adjacent to the virtual node is changed.
This makes it possible to calculate the key management range length of each node device at high speed.

[4] One aspect of the present invention is the load distribution apparatus described above, wherein the node specifying information is an address of the node apparatus.
According to this aspect, the node address is set as the node specifying information in the key search node table.
As a result, the key table processing unit can acquire the address of the node device that manages the key from the key search node table, so that the value corresponding to the key can be inquired using the read address as the destination. It becomes possible.

[5] According to one aspect of the present invention, a computer used as a load distribution apparatus stores a node table including node specification information for specifying a node device constituting a data management system, and the node specification information; A storage unit for storing a key search node table indicating a correspondence with a node ID generated for the node device specified by the node specifying information, and node specifying information of the node device participating in the data management system Corresponds to the node ID that is written in the node table, deletes node identification information of the node device that has left the data management system from the node table, the node ID set in the key search node table, and the node ID Each node device based on the node identification information attached. A management range length calculation unit for calculating a management range length of a key to be managed, a correspondence between node identification information in the node table and a node ID generated for a node device identified by the node identification information A key search node table in which an attachment is set is written in the storage unit, and a difference in management range length of each node device calculated by the management range length calculation unit based on the generated key search node table is a predetermined condition A node ID is newly generated for the node device selected based on the management range length, and the node identification information of the node device is associated with the newly generated node ID Is added to the key search node table, and the virtual node registration unit that repeats the process of updating the key search node table, A program that functions as a key table processing unit that acquires node identification information of the node device that manages a value corresponding to the key based on a node ID and a specified key.

  According to the present invention, in a key-value type data management system that distributes and manages data by a plurality of node devices, while suppressing an increase in storage capacity used to manage node devices participating in the system, It is possible to eliminate the bias in the data management range length in each node device.

1 is a configuration diagram of a data management system according to a first embodiment of the present invention. It is a block diagram which shows the structure of the load distribution apparatus by the embodiment. It is a figure which shows the data structural example of the node table by the embodiment. It is a figure which shows the data structural example of the node table for key searches by the embodiment. It is a figure which shows the data structural example of the key table by the embodiment. It is a figure which shows the node table update process flow of the node table management part by the embodiment. It is a figure which shows the node table update process flow for a key search of the virtual node registration part by the embodiment. It is a figure for demonstrating the management range length calculation process of the management range length calculating part by the embodiment. It is a figure which shows the experimental result of the data management system by the embodiment. It is a figure for demonstrating the management range length calculation process of the management range length calculating part by 2nd Embodiment. It is a figure which shows the data structural example of the node table by 3rd Embodiment. It is a figure which shows the data structural example of the node table for key searches by the embodiment. It is a figure for demonstrating the management range length calculation process of the management range length calculating part by the embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[1. First Embodiment]
[1.1 Overall configuration]
FIG. 1 is a configuration diagram of a data management system according to the first embodiment of the present invention. The data management system of this embodiment is a key-value (key (value) -value) type data management system using DHT, for example, a distributed database system or a distributed file system. As shown in the figure, the data management system is configured by connecting a plurality of node devices 1 via a communication network 5. The communication network 5 is an IP (Internet Protocol) communication network composed of various network devices such as communication devices such as routers and communication lines such as cables. The node device 1 includes a load distribution device 10 and a data storage unit 30. The data storage unit 30 stores data such as files to be managed in the data management system.

[1.2 Relationship between load balancer and data management system]
The load balancer 10 operates within each node device 1. The node device 1 is, for example, a computer device that executes a node, and the load distribution device 10 corresponds to a process that is executed when the node device 1 executes an application program. Each load balancer 10 holds a node table for managing the node devices 1 participating in the key-value type data management system and a key search node table used for discovery of the node devices 1 performing key management. ing. When a new node device 1 participates in the key-value type data management system or when the node device 1 that has participated in the key-value type data management system is disconnected, the load distribution device 10 operates and the node Modify the table and the key retrieval node table. Since all the load distribution apparatuses 10 perform the same operation, the node table and the key search node table of all the load distribution apparatuses 10 in the key-value type data management system are kept the same.

  In the present embodiment, only the node devices 1 participating in the key-value type data management system are registered as nodes in the node table. On the other hand, nodes constituting a circular overlay network in DHT are registered in the key search node table used for key registration and search. The nodes constituting the overlay network include a node that has an entity like the node device 1 and a virtual node that is virtually generated for the node device 1. In the present embodiment, load distribution is realized by registering virtual nodes in the key search node table.

  The existing key-value type data management system technology is applied to the configuration method of the key-value type data management system using DHT and the node joining / leaving notification method.

[1.3 Configuration of load balancer]
FIG. 2 is a block diagram showing a configuration of the load distribution apparatus 10 according to the present embodiment, and only functional blocks related to the present embodiment are extracted and shown. As shown in the figure, the load distribution apparatus 10 includes a node table management unit 11, a virtual node registration unit 12, a node ID generation unit 13, a management range length calculation unit 14, a storage unit 15, and a key table processing unit 16. It is prepared for.

  The storage unit 15 is realized by a hard disk device, a semiconductor memory, or the like, and records a node table 21, a key search node table 22, and a key table 23. The node table 21 is data in which the node ID and the address of the node device 1 constituting the data management system are associated with each other. The key search node table 22 is data in which node IDs and addresses of nodes participating in the overlay network are associated with each other. The key table 23 is data indicating a set of “key” and “value”.

  The node table management unit 11 stores the node table 21 stored in the storage unit 15 when a new node device 1 participates in the data management system or when the node device 1 that has participated in the data management system leaves. Update. The node ID generation unit 13 generates a node ID based on the address of the node device 1 and the number of virtual nodes. The management range length calculation unit 14 calculates a management range length that is the width of the key range managed by each node device 1 based on the node ID set in the key search node table 22.

  The virtual node registration unit 12 generates a virtual node for the node device 1 with the shortest management range length calculated by the management range length calculation unit 14 and updates the key search node table 22 stored in the storage unit 15. This process is repeated until the difference in the management range length of each node device 1 becomes equal to or less than a predetermined value or the number of virtual nodes reaches the upper limit. The memory 121 included in the virtual node registration unit 12 includes a management range threshold Th that is a condition for determining that a difference in management range length of each node device 1 is equal to or less than a predetermined value, and a maximum indicating the upper limit of the number of virtual nodes. The virtual node number Vmax is stored. The management range threshold Th is a threshold for the ratio of the narrowest management range length to the widest management range length. The key table processing unit 16 refers to the key table 23 stored in the storage unit 15 and reads a value corresponding to the key.

[1.4 Data structure]
FIG. 3 is a diagram illustrating a data configuration example of the node table 21 according to the present embodiment.
As shown in the figure, the node table 21 is a list of data associated with the node ID and node address of the node device 1 constituting the data management system. The node table 21 is sorted in numerical order of node IDs. The node ID of the node table 21 does not include the node ID of the virtual node. In the present embodiment, a combination of an IP (Internet Protocol) address and a port number of the node device 1 is used as the node address.

FIG. 4 is a diagram illustrating a data configuration example of the key search node table 22 according to the present embodiment.
As shown in the figure, the key search node table 22 is a list of data in which node IDs, node addresses, and virtual node numbers of nodes constituting the overlay network are associated with each other. The key search node table 22 is sorted in numerical order of node IDs. In addition to the node ID of the node device 1, the node ID of the key search node table 22 includes the node ID of the virtual node. The data management system of the present embodiment, which is a key-value type data management system using DHT, uses this key search node table 22 for finding a node that performs key management. That is, the key search node table 22 indicates a circular overlay network in DHT. The node address can be used as node specifying information for specifying the node device 1.

FIG. 5 is a diagram illustrating a data configuration example of the key table 23 according to the present embodiment.
As shown in the figure, the key table 23 is a list of data in which combinations of keys and values managed by the data management system are associated. In the present embodiment, a case where the key is a data ID for specifying data and the value is the node address of the node device 1 storing the data will be described as an example.

[1.5 Operation of load balancer]
Next, the operation of the load distribution apparatus 10 will be described.
First, an example of a joining / leaving notification procedure in a key-value data management system using DHT will be described. When the node device 1 participates in the overlay network, the load distribution device 10 in the node device 1 requests participation from the load distribution device 10 in any of the node devices 1 that have already participated in the overlay network. . The load balancer 10 that has received the request for participation notifies the participation of the node to the slice leader (Slice Leader), and the slice leader notifies other slice leaders. Notification to the unit leader. In this way, the node participation information is notified to all nodes in the overlay network. Note that the load balancer 10 determines a notification destination such as a slice leader or a unit leader based on the node ID registered in the node table 21.

  When the load balancer 10 detects the detachment of the node device 1, the detachment of the node is notified to the slice leader as in the case of the notification of participation, and the slicing leader notifies the other slice leaders. In addition, the slice leader notifies the unit leader in his area. The load balancer 10 does not receive a notification of leaving from the load balancer 10 in the other node device 1 that has left the overlay network or does not receive a response to the survival confirmation signal that is periodically transmitted to the other node device 1. To detect withdrawal. The survival confirmation signal is transmitted with the node address registered in the node table 21 as the destination.

  The above-described participation / leaving notification procedure is an example of One Hoop, which is one of the overlay network configuration methods using DHT, and other existing technologies of the key-value type data management system using DHT. Thus, it is possible to notify participation and withdrawal to all the node devices 1.

[1.5.1 Operation of node table manager]
FIG. 6 is a diagram showing a node table update processing flow of the node table management unit 11 according to the present embodiment.
The node table management unit 11 receives a node join notification or a leave notification from another load balancer 10. The notification of participation includes a notification type indicating “participation” and the node address of the participating node apparatus 1. The leave notification includes a notification type indicating “leave” and the node address of the node device 1 that has left. The node table management unit 11 outputs the node address included in the notification of participation or the notification of withdrawal and the virtual node number “0” to the node ID generation unit 13 and instructs generation of the node ID. The node ID generation unit 13 generates a node ID from the combination of the node address and the number of virtual nodes, and notifies the node table management unit 11 (step S11).

  Subsequently, when the node table management unit 11 determines that the notification type is “participation” (step S12: YES), the node table management unit 11 stores the management range length calculation unit in step S11. Data in which the node ID received from 14 is associated with the node address set in the notification of participation is added (step S13).

  On the other hand, when the node table management unit 11 determines that the notification type is “leave” (step S12: NO), the management table length calculation unit 14 in step S11 from the node table 21 stored in the storage unit 15. The data of the set of the node ID received from the node ID and the node address associated with the node ID is deleted (step S14).

  After the process of step S13 or step S14, the node table management unit 11 outputs an execution instruction to the virtual node registration unit 12, and instructs generation of a key search node table based on the updated node table 21 (step S15). ).

The node table management unit 11 periodically transmits a survival confirmation signal to another load balancer 10 with the node address registered in the node table 21 as a destination, and when there is no response, the node address of the destination The above processing is performed on the assumption that the notification of leaving of the node for which is set is received.
Further, when the node table management unit 11 receives the leave notification, the node table management unit 11 does not perform the process of step S11, and in step S13, the node address set in the leave notification and the node associated with the node address. The data of the ID set may be deleted.

  With the above operation, the node table 21 in which the nodes participating in the key-value type data management system using DHT are recorded is maintained.

[1.5.2 Operation of virtual node registration unit]
FIG. 7 is a diagram showing a key search node table update processing flow of the virtual node registration unit 12 according to the present embodiment. By this key search node table update process, the virtual node registration unit 12 generates a key search node table 22 from the node table 21. In the memory 121 of the virtual node registration unit 12, a management range threshold Th set by an input unit (not shown) and the maximum virtual node number Vmax are stored in advance.

  The virtual node registration unit 12 receives the instruction to generate the key search node table output from the node table management unit 11 in step S15 of FIG. The virtual node registration unit 12 deletes the contents of the key search node table 22 currently stored in the storage unit 15, and copies the current setting contents of the node table 21 to the key search node table 22 (step S21). . At this time, the virtual node registration unit 12 sets “0” in the field of the number of virtual nodes in association with the set of the node ID and the node address of each data copied to the key search node table 22.

  Subsequently, the virtual node registration unit 12 sets “0” to a parameter N used as a counter of the number of generated virtual nodes (step S22). The virtual node registration unit 12 determines whether or not the parameter N is smaller than the maximum virtual node number Vmax stored in the memory 121 (step S23). When the virtual node registration unit 12 determines that the parameter N is smaller than the maximum virtual node number Vmax (step S23: NO), the management range of the node devices 1 participating in the key-value type data management system using DHT The management range length calculator 14 is instructed to output the maximum management range length Lmax, which is the largest management range length, and the minimum management range length Lmin, which is the smallest management range length (step S24). The management range length calculation unit 14 refers to the key search node table 22, and refers to the node device 1 having the maximum management range length Lmax, the minimum management range length Lmin, and the minimum management range length Lmin (hereinafter, “minimum management range length”). The node address of “node device” and the number of virtual nodes of the minimum management range length node device are returned.

  The virtual node registration unit 12 determines whether or not the maximum management range length Lmax / minimum management range length Lmin is greater than the management range threshold Th (step S25). When the virtual node registration unit 12 determines that the maximum management range length Lmax / minimum management range length Lmin is greater than the management range threshold Th (step S25: YES), the minimum management range having the minimum management range length at the present time One virtual node is added to the long node device (step S26). Specifically, it operates as follows.

  The virtual node registration unit 12 calculates the number of virtual nodes to be added by adding 1 to the number of virtual nodes received in step S24. The virtual node registration unit 12 outputs the node address of the minimum management range length node device and the number of virtual nodes to be added to the management range length calculation unit 14 and instructs generation of a node ID. The management range length calculation unit 14 generates a node ID from the combination of the node address and the number of virtual nodes, and notifies the virtual node registration unit 12 of the node ID. The virtual node registration unit 12 uses a key search for data in which the node ID of the virtual node received from the management range length calculation unit 14, the node address of the minimum management range length node device, and the number of virtual nodes to be added are associated with each other. Add to the node table 22.

After adding the virtual node, the virtual node registration unit 12 adds 1 to the current value of the parameter N (step S27), and repeats the processing from step S23.
Finally, when it is determined in step S23 that the total number of virtual nodes has reached the maximum number of virtual nodes Vmax and the parameter N is equal to or greater than the maximum number of virtual nodes Vmax (step S23: YES), or step S25 When it is determined that the maximum management range length Lmax / minimum management range length Lmin is equal to or less than the management range threshold Th (step S25: NO), the virtual node registration unit 12 ends the process.

As described above, in the present embodiment, the process of adding a virtual node to the node device 1 with the smallest key management range length is repeated. Thereby, the imbalance of the management range length can be corrected efficiently, and the number of virtual nodes can be determined so that the management range length of the key of each node device 1 becomes approximately the same.
Note that the closer the management threshold Th is to 1, the more the number of keys managed in each node device 1 becomes closer, but on the other hand, the number of virtual nodes increases, resulting in the size of the key search node table 22 as a result. Will increase. Therefore, the management threshold value Th is determined in a range in which the size of the key search node table 22 does not become too large based on the results of experiments and the like. Further, the larger the maximum virtual node number Vmax, the larger the size of the key search node table 22 may be. Therefore, the maximum virtual node number Vmax is determined based on the capacity of the memory used as the storage unit 15.

  In the present embodiment, in step S25, the maximum management range length Lmax / minimum management range length Lmin is compared with the threshold value Th of the management range, and it is determined whether the processing has been completed. It is not a thing. For example, the maximum management range length Lmax may be compared with a predetermined threshold value, or the minimum management range length Lmin may be compared with a predetermined threshold value. Alternatively, the standard deviation or variance of the management range lengths of all the node devices 1 and virtual nodes registered in the key search node table 22 may be compared with a predetermined threshold value.

[1.5.3 Operation of node ID generator]
Next, node ID generation processing of the node ID generation unit 13 according to the present embodiment will be described.
Upon receipt of the node address and the number of virtual nodes from the node table management unit 11 or the virtual node registration unit 12, the node ID generation unit 13 generates a node ID by using a hash function for data obtained by combining the node address and the number of virtual nodes. To do. The number of virtual nodes is an integer starting from 0. In the present embodiment, SHA-1 (Secure Hash Algorithm 1) is used as a hash function. Therefore, the node ID generation unit 13 generates a 160-bit value as the node ID. For example, when the node address is “192.168.100.10:3939” and the number of virtual nodes is “3”, the node ID generation unit 13 uses the data “192.168.100.10:3939_3” that combines the node address and the number of virtual nodes. Is generated. If the hash function of SHA-1 is Hash (), the node ID generation unit 13 calculates the node ID using Hash (“192.168.100.10:3939_3”). The node ID generation unit 13 returns the calculated node ID to the node table management unit 11 or the virtual node registration unit 12 that is the output source of the node address and the number of virtual nodes.

[1.5.4 Operation of control range length calculation unit]
Next, the management range length calculation process of the management range length calculation unit 14 according to the present embodiment will be described. The management range length calculation unit 14 calculates the management range length on the hash space of each node participating in the overlay network in the management range length calculation process.

  FIG. 8 is a diagram for explaining the management range length calculation processing of the management range length calculation unit 14. As described above, a node ID having a 160-bit length is used in the present embodiment. However, in order to simplify the description, the node ID is described as having an 8-bit length in FIG. Since the node ID is 8 bits long, the node ID is an integer value in the range of 0 to 255.

FIG. 8A shows the overlay network, and FIG. 8B shows the key search node table 22 when the overlay network shown in FIG. 8A is formed. 8A and 8B show a state in which three node devices 1 are participating in the overlay network, and these three node devices 1 are represented by capital letters A, B, Let C be node A, node B, and node E, respectively. In addition, the virtual nodes of the node A, the node B, and the node E are defined as a node a, a node b, and a node e by using lowercase letters a, b, and c.
Hereinafter, node x with node ID = i is referred to as node x [i].

  FIG. 8C is a diagram showing the management range length of each node device 1. The key management range of each node is a distance from a predecessor that is an adjacent node in the opposite direction in the clockwise direction. For example, the predecessor of the node E [90] is the node b [70]. Accordingly, the key management range of the node E alone is “71 to 90”, and the management range length is “20”. In addition, node E has two virtual nodes, node e [60] and node e [190]. Since the predecessor of the node e [190] is the node a [160], the key management range is “161 to 190”, and the management range length is “30”. Further, since the predecessor of the node e [60] is the node a [10], the key management range is “11 to 60”, and the management range length is “50”. Therefore, the management range length of the entire node E is 20 + 30 + 50 = 100.

Similarly, the management range length of the entire node A is the management range length “26” of the key management range “241 to 255, 0 to 10” of the node A [10] alone and the key management range of the node a [160] alone. The total of the management range length “20” in the range “141 to 160” is “46”.
Further, the management range length of the entire node B is the management range length “50” of the key management range “91 to 140” of the node B [140] alone and the key management range “191 to 240” of the node b [240] alone. And the management range length “10” of the key management range “61 to 70” of the node b [70] alone is “110”.

  Thus, the total management range length of each node device 1 is the sum of the management range length of the node alone of the node device 1 and the management range length of each virtual node generated for the node device 1. As a result, the management range length calculation unit 14 outputs the management range length “110” of the node B as the maximum management range length Lmax and the management range length “46” of the node A as the minimum management range length Lmin.

In order to perform the above operation, the management range length calculation unit 14 refers to the key search node table 22 and calculates the management range length of all the node devices 1. Specifically, it operates as follows.
The management range length computing unit 14 pays attention to the node IDs set in the key search node table 22 in order, and the node ID of interest and the adjacent node ID smaller than the node ID of interest From the difference, the management range length of the node ID of interest is obtained.

  If an adjacent node ID smaller than the node ID of interest is not set in the key search node table 22, the node ID of interest is the smallest node ID participating in the overlay network. is there. Therefore, the node ID of predecessor is the largest node ID among those registered in the key search node table 22. The management range length calculation unit 14 adds the maximum value that the node ID can take to the node ID of interest (for example, node ID = 10 shown in FIG. 8B) (when the node ID is 8 bits long). 255 ") is added, and the management range length is calculated by subtracting the node ID of the predecessor (node ID = 240 shown in FIG. 8B).

  The management range length calculation unit 14 calculates the management range length for each node ID registered in the key search node table 22, and then calculates the node ID associated with the same node address in the key search node table 22. Total the calculated management range lengths. Thereby, the management range length of each node device 1 is calculated. The management range length calculator 14 selects the maximum maximum management range length Lmax and the minimum minimum management range length Lmin among the calculated management range lengths. The management range length calculation unit 14 reads the largest number of virtual nodes from the number of virtual nodes set in the key search node table 22 in association with the node address corresponding to the minimum management range length Lmin. The management range length calculation unit 14 returns the maximum management range length Lmax, the minimum management range length Lmin, the node address of the minimum management range length node device, and the number of read virtual nodes to the virtual node registration unit 12.

[1.5.5 Operation of key table processing unit]
Subsequently, an operation example of the key table processing unit will be described.
When any one of the node devices 1 stores new data in the data storage unit 30, the key table processing unit 16 generates a data ID from the data name of the new data using a SHA-1 hash function. The key table processing unit 16 searches for a node ID close to the generated data ID in the clockwise direction from the key search node table 22 stored in the data storage unit 30, and acquires a node address corresponding to the node ID. . The key table processing unit 16 transmits a key registration request in which the acquired node address is set as the destination and the data ID and the node address of the own node device 1 are set. In the node device 1 that has received the key registration request, the key table processing unit 16 of the load distribution device 10 adds the combination of the data ID and the node address in the key registration request to the key table 23 stored in the data storage unit 30. To do.

  When any one of the node devices 1 accesses data, the key table processing unit 16 of the load distribution device 10 generates an ID from the data name of the data. The key table processing unit 16 searches for a node ID close to the generated data ID in the clockwise direction from the key search node table 22 stored in the data storage unit 30, and acquires a node address corresponding to the node ID. . The key table processing unit 16 transmits an inquiry in which the acquired node address is set as the destination and the data ID is set as the key. In the node device 1 that has received the inquiry, the key table processing unit 16 of the load distribution device 10 reads out a value corresponding to the key set in the inquiry from the key table 23 stored in the data storage unit 30 and returns it. To do. This value indicates the node address of the node device 1 storing the data. The node device 1 as the inquiry transmission source accesses the node device 1 storing the data as the node address destination indicated by the received “value”, and is stored in the data storage unit 30 provided in the node device 1 as the access destination. Reading, rewriting, deleting, etc. of the operation target data.

  In the above description, an example in which the key is a data ID for identifying data and the value is the node address of the node device 1 storing the data has been described. However, a key-value type data management system using DHT has been described. Other key and data sets can be distributed and managed using existing technology.

[1.6 Performance]
FIG. 9 shows experimental results using the load distribution apparatus 10 according to the present embodiment. FIG. 9A shows the relationship between the number of node devices 1 participating in the data management system and the standard deviation of the management range length of all node devices 1, and FIG. The relationship between the number of participating node devices 1 and the size of the key search node table is shown.
In the experiment, the management range threshold value Th = 1.5, the maximum number of virtual nodes Vmax = 10000, and the number of node devices 1 were in the range of 3 to 1000. As a comparison, the standard deviation of the management range length of all the node devices and the size of the key search node table when using a method in which 0, 10, 100 virtual nodes are assigned to all the node devices are used. Show.

From the result of the standard deviation of the management range length shown in FIG. 9A, in the present embodiment, when performing load distribution of each node device 1 participating in the data management system, 100 fixed number of virtual nodes It can be seen that the performance equivalent to the method of assigning is obtained. On the other hand, from the result shown in FIG. 9B, the size of the key search node table 22 when the load distribution apparatus 10 of this embodiment is used is that 10 fixed virtual nodes are allocated to each node apparatus. It can be seen that the size is the same as the size of the key search node table used in the system.
From these experimental results, according to the present embodiment, it is possible to realize a performance equivalent to the case of allocating a fixed number of 100 virtual nodes to each node device by using a key search node table of 1/10 size. I understand.

[2. Second Embodiment]
In the first embodiment described above, the management range length calculation unit 14 obtains the management range lengths of all the nodes registered in the key search node table 22, and then manages the keys managed by each node device 1. However, the calculation method is not limited to this. In this embodiment, the key management range length managed by each node is calculated at high speed.
Hereinafter, differences from the first embodiment will be described.

FIG. 10 is a diagram for explaining the processing of the management range length calculation unit 14 according to the present embodiment.
FIG. 10A illustrates an example in which the node a [230], which is the virtual node of the node A, is added to the overlay network illustrated in FIG. 8A by the process of step S26 of FIG. FIG. 10B shows the management range length of each node device 1 after the addition of the node a [230].

As a result of the addition of the node a [230] shown in FIG. 10A, the management range length of the node A is “40” which is the distance between the added node a [230] and the node e [190] of the predecessor. Is added. On the other hand, the management range length of the node b [240] of the successor (the adjacent node in the clockwise direction) of the added node a [230] is shortened by “40”. In this example, since the node b [240], which is the successor of the added node a [230], is a virtual node of the node B, the management range length of the node B is shortened by “40”.
As a result, as shown in FIG. 10B, the calculation of the management range length is obtained simply by adding “40” to the management range length of node A and subtracting “40” from the management range length of node B. It is possible.

  In order to perform the above operation, the management range length calculation unit 14 writes the management range length calculated for each node address in step S24 of FIG. Then, a virtual node is added in step S26, and when executing step S24 in the next loop, the node ID of the virtual node added in step S26 is received from the virtual node registration unit 12. The added node ID is described as an additional node ID.

  The management range length calculation unit 14 obtains the management range length of the added virtual node from the difference from the adjacent node ID smaller than the additional node ID from the key search node table 22, and the node corresponding to the additional node ID Read the address. The management range length calculation unit 14 updates the management range length stored in the storage unit 15 corresponding to the read node address to a value obtained by adding the management range length of the added virtual node.

  Further, the management range length calculation unit 14 specifies an adjacent node ID larger than the additional node ID from the key search node table 22, and reads a node address corresponding to the specified node ID. The management range length calculation unit 14 updates the management range length stored in the storage unit 15 corresponding to the read node address to a value obtained by subtracting the management range length of the added virtual node.

  After the above processing, the management range length calculation unit 14 reads the management range length corresponding to the address of each node device 1 after the addition of the virtual node from the storage unit 15.

[3. Third Embodiment]
The third embodiment is applied when the node performance of each node device 1 is different. That is, when the performance of the node devices 1 participating in the DHT is not the same, the data amount managed (stored) (key, value) by each load distribution device 10 is controlled according to the node performance. Hereinafter, the difference between the first embodiment and the second embodiment will be described.

[3.1 Data structure]
FIG. 11 is a diagram illustrating a data configuration example of the node table 21 according to the present embodiment. The node table 21 shown in the figure is different from the node table 21 shown in FIG. 3 in that node performance values are set in association with node IDs and node addresses. The node performance value is a quantitative value for an index indicating the performance of the node, and includes, for example, a value indicating the performance of a CPU (central processing unit) and the total disk capacity that can be stored in the node.

  FIG. 12 is a diagram illustrating a data configuration example of the key search node table 22 according to the present embodiment. The key search node table 22 shown in the figure is different from the key search node table 22 shown in FIG. 4 in that a node performance value is set in association with the node ID, the node address, and the number of virtual nodes. Is a point.

[3.2 Operation of load balancer]
Next, the operation of the load distribution apparatus 10 will be described.

[3.2.1 Operation of node table management unit]
In step S11 of FIG. 6, the node participation notification received by the node table management unit 11 includes the notification type “participation”, the node address of the participating node device 1, and the performance value of the participating node device 1. It is. In step S13, the node table management unit 11 adds to the node table 21 stored in the storage unit 15 the node ID received from the management range length calculation unit 14 in step S11, and the node address and node included in the notification of participation. Data that is associated with the performance value is newly added. In step S14, the node table management unit 11 also receives the node ID received from the node ID generation unit 13 in step S11 from the node table 21 stored in the storage unit 15, and the node associated with the node ID. Delete the data of the address / performance value pair.

[3.2.2 Operation of virtual node registration unit]
In step S21 of FIG. 7, when the virtual node registration unit 12 copies the setting contents of the node table 21 to the key search node table 22, the set of the node ID, the node address, and the performance value is copied as it is. Further, in step S26, the virtual node registration unit 12 uses the key search node to store data in which the number of virtual nodes to be added is associated with the node ID of the virtual node, the node address of the minimum management range length node device, and the performance value. Add to table 22.

[3.2.3 Operation of control range length calculation unit]
The management range length calculation unit 14 according to the present embodiment sets the management range length of each node device 1 calculated as in the first embodiment or the second embodiment described above to the performance value of the node device 1. Weighting is performed accordingly.

  The management range length calculation unit 14 calculates the management range length for each node address of each node device 1 as in the first embodiment or the second embodiment. When the management range length calculation unit 14 reads the performance value set in the node table 21 or the key search node table 22 in association with each node address, the management range length calculation unit 14 calculates the management range length calculated for each node address. Divide by the read performance value. The management range length computing unit 14 selects the maximum management range length as the maximum management range length Lmax and the minimum management range length as the minimum management range length Lmin among the management range lengths as a result of the division. The management range length calculation unit 14 reads the largest number of virtual nodes from the number of virtual nodes set in the key search node table 22 in association with the node address corresponding to the minimum management range length Lmin. The management range length calculation unit 14 returns the maximum management range length Lmax, the minimum management range length Lmin, the node address of the minimum management range length node device, and the number of read virtual nodes to the node table management unit 11.

FIG. 13 is a diagram for explaining the management range length calculation in consideration of node performance. The overlay network is configured in the same manner as in FIG. 8A, and the management range lengths calculated by the same processing as in the first embodiment for the nodes A, B, and C are “46” and “110”, respectively. ”,“ 100 ”. The node performance values of the node A, the node B, and the node C are “1”, “2”, and “3”, respectively. In this case, the management range length of the node A is a value “46” obtained by dividing the management range length “46” of the node A by the node performance value “1” of the node A. The management range length of the node B is a value “55” obtained by dividing the management range length “110” of the node B by the node performance value “2” of the node B. The management range length of the node E is a value “33” obtained by dividing the management range length “100” of the node E by the node performance value “3” of the node E. Therefore, the maximum management range length Lmax is the management range length “55” of the node B, and the minimum management range length Lmin is the management range length of the minimum management range length node device of the node E.
As a result, the number of virtual nodes of node E with high performance is increased.

[4. Effects of the embodiment of the present invention]
As described above, according to the embodiment of the present invention, the load balancer installed in the node device constituting the key-value type data management system using DHT is the participation of the node device constituting the system. Or when a node is disconnected, the process of assigning a virtual node to the node device with the narrowest management range length is repeated until the management range length of each node device is nearly equal. A key search node table including the node ID of the generated virtual node is generated. Therefore, compared with the prior art, it is possible to efficiently correct the imbalance of the key management range length of each node device and to reduce the number of virtual nodes, and therefore the increase in the key search node table is small.
In addition, it becomes possible to distribute the load evenly among the node devices, or to distribute the load according to the node performance of each node device.

[5. Others]
In the above-described embodiment, the node table 21 and the key search node table 22 are generated in each load balancer 10. However, in some load balancers 10, the node table 21 and the key search node table 22 are stored. It may be generated and distributed to other load balancers 10.

  The node device 1 in which the load distribution device 10 described above is mounted has a computer system therein. The process of the operation of the load balancer 10 is stored in a computer-readable recording medium in the form of a program, and the above processing is performed by the computer system reading and executing this program. The computer system here includes a CPU, various memories, an OS, and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage unit such as a hard disk built in the computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case is also used to hold a program for a certain period of time. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

DESCRIPTION OF SYMBOLS 1 ... Node apparatus 10 ... Load distribution apparatus 11 ... Node table management part 12 ... Virtual node registration part 121 ... Memory 13 ... Node ID generation part 14 ... Management range length calculating part 15 ... Storage part 16 ... Key table processing part 21 ... Node Table 22 ... Key search node table 23 ... Key table 30 ... Storage unit 5 ... Communication network

Claims (5)

Storing a node table including node specifying information for specifying a node device constituting the data management system; and the node specifying information and a node ID generated for the node device specified by the node specifying information; A storage unit for storing a key search node table indicating the association of
A node table management unit for writing node specifying information of a node device participating in the data management system to the node table, and deleting node specifying information of a node device leaving the data management system from the node table;
A management range for calculating a management range length of a key to be managed by each node device based on the node ID set in the node table for key search and the node specifying information associated with the node ID A length calculator,
The node table for key search that sets the correspondence between the node specifying information in the node table and the node ID generated for the node device specified by the node specifying information is written in the storage unit and generated. The node device selected based on the management range length until it is determined that the difference in management range length of each node device calculated by the management range length calculation unit based on the key search node table matches a predetermined condition Virtual node registration that repeats the process of adding a new node ID to the key search node and updating the key identification node table by associating the node identification information of the node device with the newly generated node ID And
A key table processing unit that acquires node identification information of the node device that manages a value corresponding to the key based on a node ID in the key search node table and a specified key;
A load balancer comprising: The management range length calculation unit weights the management range length of the node device according to a value indicating the performance of the node device,
The virtual node registration unit determines whether a difference in the management range length weighted according to a value indicating the performance of the node device meets a predetermined condition;
The load distribution apparatus according to claim 1. The management range length calculation unit, when an association between node identification information of the node device and the newly generated node ID is added to the key search node table by the virtual node registration unit, The node ID adjacent to the node ID added to the node table is specified to calculate the management range length of the key, and the management range length of the node device for which the node ID has been newly generated is calculated. The node management information is updated to the added value, and node identification information corresponding to the node ID adjacent to the node ID of the virtual node is read from the key search node table, and the key management of the node device identified by the read node identification information Update the range length to a value obtained by subtracting the calculated management range length.
The load distribution apparatus according to claim 1 or 2, wherein   The load distribution apparatus according to claim 1, wherein the node specifying information is an address of the node device. A computer used as a load balancer
Storing a node table including node specifying information for specifying a node device constituting the data management system; and the node specifying information and a node ID generated for the node device specified by the node specifying information; A storage unit for storing a key search node table indicating the association of
A node table management unit for writing node specifying information of a node device participating in the data management system to the node table, and deleting node specifying information of a node device leaving the data management system from the node table;
A management range for calculating a management range length of a key to be managed by each node device based on the node ID set in the node table for key search and the node specifying information associated with the node ID Long operation part,
The node table for key search that sets the correspondence between the node specifying information in the node table and the node ID generated for the node device specified by the node specifying information is written in the storage unit and generated. The node device selected based on the management range length until it is determined that the difference in management range length of each node device calculated by the management range length calculation unit based on the key search node table matches a predetermined condition Virtual node registration that repeats the process of adding a new node ID to the key search node and updating the key identification node table by associating the node identification information of the node device with the newly generated node ID Part,
A key table processing unit that acquires node identification information of the node device that manages a value corresponding to the key based on a node ID in the key search node table and a specified key;
A program characterized by functioning as

Images