JP6353395B2 - Distribution request distribution method, node, and program - Google Patents

Description

  The present invention relates to a distribution request that distributes distribution requests so that distribution requests are not biased to specific nodes when a large amount of data is distributed and managed in a distributed hash table composed of a plurality of nodes (PCs). The present invention relates to a distribution method, a node, and a program.

  The distributed hash table is a technique for sharing and managing one logical ID space by a plurality of nodes, and is used in a distributed database such as Key Value Store. When data is arranged in the distributed hash table, the name of the data is used as a key, and the data is arranged for a node in charge of the key. When acquiring data from the distributed hash table, the name of the data to be acquired is used as a key, the node in charge of the key is searched from the distributed hash table, and the data is acquired. However, when distribution requests are concentrated on specific data or a data group, there is a problem that a distribution load of a node that manages the data (group) increases.

  To solve this problem, the existing method 1 (for example, refer to Non-Patent Document 1), when distribution requests are concentrated on a specific data group, a node having a small distribution request is assigned a part of the ID space of the data group. Move the node to take charge. In FIG. 1, a distributed hash table in which distribution requests are concentrated on nodes B and C is smoothed using the existing method 1. Specifically, the node D with a small number of distribution requests undertakes a part of the range of responsibility of B with a large number of distribution requests: [15, 20], thereby smoothing the number of distribution requests. At this time, D passes its assigned range to the adjacent node C. In the existing method 2 (for example, see Non-Patent Document 2), when each node exceeds the upper limit of the number of distribution requests, its own assigned range is transferred to an adjacent node. In FIG. 2, nodes B and C narrow the assigned range, and A and D widen the assigned range, thereby smoothing the number of distribution requests.

A. R. Bharambe, M.M. Agrawal, and S.A. Seshan, “Mercury: supporting scalable multi-attribute range queries”, pp. 197 353-366, 2004. I. Konstantinou, D.H. Tsusumakos, and N.M. Koziris, “Fast and cost-effective online load-balancing in distributed systems”, IEEE Transactions on Parallels and Distillates. 22, no. 8, pp. 1350-1364, 2011. M.M. Hefeeda and O. Saleh, “Traffic modeling and proportional partial caching for peer-to-peer systems”, IEEE / ACM Transactions on Networking, vol. 16, no. 6, pp. 1447-1460, 2008.

  However, since the existing methods 1 and 2 are performed sequentially and asynchronously, the assigned range of the node is switched many times until the number of distribution requests is smoothed, and data movement between the nodes frequently occurs. In other words, the existing methods 1 and 2 have a problem that data movement frequently occurs between nodes due to switching of the assigned range of nodes.

  Accordingly, the present invention provides a distribution request distribution method, a node, and a program capable of leveling the number of data distribution between nodes while suppressing frequent data movement in order to solve the above problems. With the goal.

  Therefore, the present invention measures the number of distribution requests for each node in a certain time unit, and distributes and manages the data managed by the node having a large number of requests based on the result at a plurality of nodes adjacent to the node. It was.

Specifically, the distribution request distribution method according to the present invention changes the assigned range H in a distributed hash table composed of a plurality of nodes each holding data x in the assigned range H of a common size. A distribution request distribution method for distributing data distribution requests to each node,
A data distribution number investigation step in which one node investigates the number of distribution requests made to other nodes at a certain time;
When the one node receives a distribution request,
Based on the number of distribution requests acquired in the data distribution number checking step, a distribution request function f (x), which is a probability density distribution in which data x is requested to be distributed, is determined, and the probability of the number of nodes holding the data x A charge range determination step for determining the charge range H so that the square error between the inclusion functions g (x, H) and f (x), which are density distributions, is minimized in the range of the desired data x;
It is characterized by performing.

In addition, the node according to the present invention is a node that constitutes a distributed hash table and holds data x in the charge range H of a common size,
A data distribution number survey module that investigates the number of distribution requests made to other nodes at a certain time,
When a delivery request is received,
Based on the number of distribution requests acquired by the data distribution number survey module, a distribution request function f (x), which is a probability density distribution in which data x is requested to be distributed, is determined, and the probability of the number of nodes holding the data x A responsible range determination module for determining the responsible range H such that the square error between the inclusion functions g (x, H) and f (x), which are density distributions, is minimized in the range of the desired data x;
Is provided.

  The present invention uniquely determines the assigned range of each node from which the load can be most smoothed from the number of distribution requests of each node. Specifically, data of a node having a large number of distribution requests at a specific time interval is shared between adjacent nodes (see FIG. 3). As a result, frequent data transfer due to switching of the assigned range can be suppressed until the number of distribution requests is smoothed.

  Furthermore, the program according to the present invention is a program for causing a computer to execute the distribution request distribution method.

  The distribution request distribution method according to the present invention can be realized by a computer and a program, and can be recorded on a recording medium or provided through a network.

  The present invention can provide a distribution request distribution method, a node, and a program capable of leveling the number of data distribution between nodes while suppressing frequent data movement.

It is a figure explaining the conventional delivery request distribution method. (A) is before the assigned range change, and (b) is after the assigned range change. It is a figure explaining the conventional delivery request distribution method. (A) is before the assigned range change, and (b) is after the assigned range change. It is a figure explaining the distribution request distribution method concerning the present invention. It is a figure explaining the node which concerns on this invention. It is a figure explaining operation | movement of the node which concerns on this invention. It is a figure explaining the distribution request distribution method concerning the present invention. It is a figure explaining the experimental result of the distribution request distribution method which concerns on this invention. It is a figure explaining the experimental result of the distribution request distribution method which concerns on this invention. It is the simulation result which compares the delivery request distribution method which concerns on this invention, and the conventional delivery request distribution method. It is the simulation result which compares the delivery request distribution method which concerns on this invention, and the conventional delivery request distribution method.

  Embodiments of the present invention will be described with reference to the accompanying drawings. Embodiment described below is an example of implementation of this invention, and this invention is not restrict | limited to the following embodiment. In the present specification and drawings, the same reference numerals denote the same components. In the following description, it is assumed that the number of distribution requests and the number of distributions are the same.

FIG. 4 is a diagram illustrating the node 100 according to the present embodiment.
The node 100 is a node that constitutes a distributed hash table and holds data x in a common range of charge H,
A data distribution number investigation module 30 for investigating the number of distribution requests made to other nodes at a certain time;
When a delivery request is received,
Based on the number of distribution requests acquired by the data distribution number survey module 30, a distribution request function f (x) that is a probability density distribution in which the data x is requested to be distributed is determined, and the probability of the number of nodes that hold the data x An assigned range determination module 40 for determining the assigned range H such that the square error between the inclusion functions g (x, H) and f (x), which are density distributions, is minimized in the range of the desired data x;
Is provided.
The node 100 further includes a database 10 and a path control module 20.

FIG. 5 shows an operation example of these modules.
The route control module 20, the database 10, and the data distribution number survey module 30 have the same functions as the existing method. The route control module 20 is a routing module that a general structured overlay should have, and has a function of searching / distributing data and the like from the network.

  The database 10 stores node assignment ranges, route information (IP addresses and IDs of other nodes), distribution request counts of other nodes, and the like. The database 10 is also a necessary function for general structured overlay.

  The data distribution number checking module 30 uses the method proposed in the existing method 1 to check the number of distributions of other nodes on the network at regular time intervals (step S01) and write it in the database 10 (step S03). . Further, the data distribution number investigation module 30 receives the distribution request to itself made to the path control module 20 (step S02) and writes it in the database 10 (step S03). The database 10 notifies the assigned range determination module 40 that the distribution request has been updated (step S04). The assigned range determination module 40 determines the assigned range by using the investigation result of the number of distribution requests to other nodes and itself (Step S05), and updates the assigned range held by the database 10 (Step S06). .

  Subsequently, a method for determining the assigned range will be described. Since a method different from the existing method is performed by the assigned range determining module 40, the assigned range determining module 40 will be described below.

Each node has an ID, the ID of node i (> 0) is X _i , the node that is larger than X _{i and} has the closest ID distance in the ID space is called the successor of node i, and the ID of the node is suc _i And At this time, the assigned range of the node i in the ID space is [X _i , suc _i ).

In the present embodiment, the load due to the increase in the number of distributions is distributed by expanding the assigned ranges of many nodes and overlapping the assigned ranges with respect to nodes with a high number of distribution requests. When the expanded assigned range is H, the assigned range of the node is [X _i , H + X _i ) (where suc _i ≦ H + X _i ). Here, the assigned range H is the same value for all nodes.

  The assigned range H is determined so as to minimize the distance between the probability density distribution of the number of distributions of each data and the probability density distribution indicating how many nodes each data is included in. This is synonymous with determining the optimum number of copies when replicating more data to more nodes as the number of distribution requests increases. An outline of the minimization problem is shown in FIG.

と定義できる。 Here, it is assumed that, as a result of the data distribution number investigation module 30 using the existing method 1 and examining the number of distributions of other nodes at regular intervals, the number of distributions of L data can be acquired. If the number of distribution requests for data x is L _x , the probability density distribution f (x) of the number of distributions of each node is

Can be defined.

と定義する。Ｎは全ノード数である。 Further, a probability density distribution g (x, H) indicating how many nodes the data x is included in is assigned as follows:

It is defined as N is the total number of nodes.

は、ノードｊの担当範囲にデータｘが含まれていれば１となり、そうでなければ０となるカーネル関数となる関数である。本実施形態では

と定義する。 here,

Is a function that is a kernel function that is 1 if the data x is included in the range in charge of the node j, and 0 otherwise. In this embodiment

It is defined as

で表現される両確率密度分布の二乗誤差を最小にするＨを求める。但し、Ｑはデータ配信数調査モジュールにて得られた各データの配信数の集合（具体的には、「所定の時間内に配信されたｘ＝ｊのデータ数」であり、「Ｌ個のデータの配信要求数」）である。 And

H that minimizes the square error of both probability density distributions expressed by However, Q is a set of distribution numbers of each data obtained by the data distribution number survey module (specifically, “the number of data of x = j distributed within a predetermined time”), The number of data delivery requests ").

Range of H depends on the size of the ID space, and generally of the order of 2 ¹⁶⁰ size. Therefore, since it is difficult to calculate H by linear search, the function of the square error is regarded as a convex function, H is obtained by binary search, and node i has its own assigned range as [X _i , H + X _i ). Set and update the database responsibility.

  A distribution request for data managed across a range of responsibility of a plurality of nodes is received by a node randomly selected from the node group and distributed to the request source.

(effect)
The effects of the present embodiment are quantified by comparison experiments with the existing methods 1 and 2. Experimental parameters are as shown in FIG. The number of data for each node is uniform, and the ID of each node and the number of distribution requests for each data depend on the value (q, α) of each Mandelbrot-Zipf distribution (Non-Patent Document 3) shown in Table 2. Shall. Using these parameters, each node smoothes the number of distributions using each method so that its number of distributions becomes 10,000.

  FIG. 7 shows the maximum number of data distributions after smoothing the number of distribution requests using each method. It can be seen that the maximum number of distributions in this embodiment is lower than other methods. FIG. 8 shows the total number of data transfers. In the existing method 2, in order to smooth the maximum number of distributions, it is necessary to transfer about eight times the data of this embodiment. In the existing method 1, data transfer about three times that of this embodiment must be performed. From the above, it can be seen that the present embodiment can smooth the number of data distributions with a small amount of data transfer (replication arrangement).

10: Database 20: Path control module 30: Data distribution number investigation module 40: Responsible range determination module 100: Node

Claims (3)

In a distributed hash table composed of a plurality of nodes each holding data x in a shared range H with a common size, a distribution request distribution method that distributes data distribution requests to each node by changing the range H There,
A data distribution number investigation step in which one node investigates the number of distribution requests made to other nodes at a certain time;
When the one node receives a distribution request,
Based on the number of distribution requests acquired in the data distribution number checking step, a distribution request function f (x), which is a probability density distribution in which data x is requested to be distributed, is determined, and the probability of the number of nodes holding the data x A charge range determination step for determining the charge range H so that the square error between the inclusion functions g (x, H) and f (x), which are density distributions, is minimized in the range of the desired data x;
A distribution request distribution method characterized by: A node that constitutes a distributed hash table and holds data x in the charge range H of a common size,
A data distribution number survey module that investigates the number of distribution requests made to other nodes at a certain time,
When a delivery request is received,
Based on the number of distribution requests acquired by the data distribution number survey module, a distribution request function f (x), which is a probability density distribution in which data x is requested to be distributed, is determined, and the probability of the number of nodes holding the data x A responsible range determination module for determining the responsible range H such that the square error between the inclusion functions g (x, H) and f (x), which are density distributions, is minimized in the range of the desired data x;
A node characterized by comprising:   A program for causing a computer to execute the distribution request distribution method according to claim 1.

Images