JP2008191904A - Distributed data management system and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce delay required for data acquisition in a distributed data management system in a pear-to-pear network. <P>SOLUTION: This data processing prediction means 206 predicts the data processing pattern of the other node based on a history of a data processing request stored in a data processing request history storage means 207, and decides whether to distribute the data to the node in advance. When receiving the instruction of distribution from the data processing predicting means 206, a data prior distribution means 208 distributes the instructed data to the other node. The node distributed in advance can instantaneously acquire the data from a data storage means 201 of its own node when the acquisition request of the data is generated. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a distributed data management system in a peer-to-peer network without a server, and more particularly to a distributed data management system and method to which a distributed hash table technique is applied.

  Distributed hash table technology as a technology for realizing data sharing in a peer-to-peer network where there is no server, any node can join at any time, and any node can leave at any time It has been known. In the peer-to-peer network, a plurality of nodes 200 are connected via the network 100 as shown in FIG. The network 100 is often an IP (Internet Protocol) network, but is not limited to an IP network.

  With respect to the peer-to-peer network as shown in FIG. 4, a Chord system, which is a representative example of the distributed hash table technique, is described in Patent Document 1. A skip net, which is an application example of the distributed hash table technique, is described in Patent Document 2. Hereinafter, the Chord system will be described as a distributed hash table technique.

  In the Chord system, data is stored in association with an ID (identifier). In general, an ID associated with data is a hash value of data or a hash value of a key characterizing the data. As a hash function for calculating the hash value, for example, a SHA-1 (Secure Hash Algorithm 1) hash function is used. In the Chord system, IDs are set in advance for the nodes in the same space as the hash value space, and the distance between the node ID and the data ID is defined to manage the data in a distributed manner.

  That is, a management node for certain data is a node having an ID having the shortest distance to the ID of the data. When registering data, it registers with the management node of the said data, and when acquiring data, it acquires from the management node of the said data. The distance is defined by division modulo the maximum value of the ID space (for example, 2 to the power of 160). In the Chord system, an overlay network is configured by managing a routing table so that a management node can be searched efficiently even in a large-scale peer-to-peer network.

As shown in FIG. 5, the Chord system node 200 includes data storage means 201, data processing means 202, communication processing means 203, route table processing means 204, and route table storage means 205. The operation of the node 200 is as follows.
First, when registering or acquiring data, the routing table processing unit 204 uses the routing table stored in the routing table storage unit 205 to determine whether the node having the ID closest to the data ID is its own node. Judge whether or not.

When the node having the ID closest to the ID of the data is its own node, the data processing unit 202 registers data in the data storage unit 201 or acquires data from the data storage unit 201.
If the node having the ID closest to the data ID is not its own node, the communication processing unit 203 sends another message having an ID as close as possible to the data ID. Forward to. By repeating the same transfer in subsequent nodes, the message reaches the node having the ID closest to the ID of the data.

  The route table in the Chord system stores, for example, the ID and address of a node having an ID that is closest to the power of 2 to the ID of the own node as an entry. It is assumed that the minimum unit of distance is 1, and i that does not exceed the ID space is used for 2 to the power of i. For example, if the maximum value of the ID space is 2 to the 160th power, i satisfies 0 ≦ i ≦ 159. With such a construction method of the routing table, in the Chord system, the number of hops until an arbitrary node arrives at another arbitrary node is O (log (N)). N represents the size of the network, that is, the number of nodes participating in the network, and O () represents an order.

  Actually, the distribution of the number of hops varies, and it is known that the average value of the number of hops is approximately half the maximum value of the number of hops. Also, the bottom of the log matches the number of words in the ID space, which is 2 in the above example. That is, the average value of the number of hops is log (N) / log (2) / 2. Since the number of hops of O (log (N)) increases relatively slowly as N increases, the Chord system is efficient with respect to the size of the network.

JP 2005-354364 A Japanese Patent Application Laid-Open No. 2004-266796

In the distributed data management system using the Chord system and a general distributed hash table technique, the number of hops of O (log (N)) is required for data acquisition, and thus there are the following problems.
The first problem is that when the network exceeds a certain size, the delay due to the number of hops becomes so large that it cannot be ignored. For example, if the number of nodes participating in the network is 1000000 nodes, that is, N = 1000000, the average value of the number of hops is log (100000) / log (2) /2=9.96. If the network delay for one hop is 100 milliseconds, the total delay is nearly one second, which may be unacceptable depending on the application. That is, since the delay is proportional to the number of hops, there is a problem that the delay in data acquisition increases due to an increase in the number of hops in a large-scale network.

  The second problem is that the possibility of passively knowing the existence of data is low. The reason is that data is basically not distributed unless there is an explicit request for data acquisition. A small number of nodes are involved in the transfer during data registration, and the ratio of nodes that can know the existence of data without the key information is limited. This problem becomes more pronounced with larger networks.

An object of the present invention is to provide a distributed data management system and method capable of reducing a delay in data acquisition in a distributed data management system using a distributed hash table technique.
Another object of the present invention is to provide a distributed data management system and a method for predicting future data processing and distributing data in a distributed data management system using a distributed hash table technique, even without a data acquisition request. Is to provide.

The present invention provides a distributed data management system in a peer-to-peer network, in which each node connected to the network communicates with other nodes via the network, data storage means for storing data, and the communication processing means. The data processing means for processing the data stored in the data storage means according to the received data processing request, the data processing request history storage means for storing the history of the data processing request, and the history of the data processing request Predicting the data processing pattern of other nodes based on the data processing prediction means for determining whether or not to deliver data to this node in advance, and when receiving a delivery instruction from this data processing prediction means, Data pre-distribution means for distributing the instructed data to the other nodes.
The data processing prediction means stores the data processing request history in the data processing request history storage means, and predicts future data processing based on the data processing request history stored in the data processing request history storage means. For example, assuming that the distribution of data acquisition is uniform with respect to time, the number of future acquisition requests for data X can be predicted based on the number of acquisition requests per unit time for certain data X. Based on this prediction, if it is determined that the cost (number of messages or message amount) required for a future acquisition request for data X is higher than the cost required for distributing data X in advance, data X Are pre-delivered to all or some nodes. By adopting such a configuration, predicting data processing, and distributing data in advance as necessary, data acquisition delay can be reduced, and the object of the present invention can be achieved. In addition, by predicting data processing and distributing data in advance as necessary, data can be distributed regardless of the data acquisition request, and the other object of the present invention can be achieved.

In addition, one configuration example of the distributed data management system of the present invention further includes a routing table storing unit that stores a routing table, and the communication processing unit based on information on the routing table stored in the routing table storing unit. Route table processing means for determining a transfer destination of the received message.
In one configuration example of the distributed data management system of the present invention, the routing table is constructed using a distributed hash table technique.
Further, in one configuration example of the distributed data management system of the present invention, the data pre-distribution unit distributes data in advance using the information of the routing table stored in the routing table storage unit.
Further, in one configuration example of the distributed data management system of the present invention, the data processing prediction means obtains the data processing pattern from the past to the present based on the history of the data processing request, and from this data processing pattern to the future When the data processing pattern is predicted and it is determined that the cost for data acquisition when not pre-distributed exceeds the cost for data pre-distribution, data pre-distribution is determined.
Further, in one configuration example of the distributed data management system of the present invention, the data processing prediction unit predicts the data processing pattern only for the pre-distribution range with a specific node group as the pre-distribution range, and the data pre- The distribution means distributes data in advance only to the group of nodes.
Further, in one configuration example of the distributed data management system of the present invention, the data processing prediction means assumes that the data acquisition frequency from the past to the present in the other nodes and the future data acquisition frequency have a linear relationship. The future data acquisition frequency is predicted as the data processing pattern.
Further, in one configuration example of the distributed data management system of the present invention, the data processing prediction means sets the time from the data registration time to the present time as T1, the time from the present time to the data expiration date as T2, and the data registration time point. The predicted value of the number of future data acquisitions is K × T2 / T1, where K is the number of data acquisitions from now to the present.

  In the distributed data management method of the present invention, each node connected to the network processes the data stored in the data storage procedure of the own node in response to the data processing request received by the communication processing means of the own node. A data processing procedure, a data processing request history storing procedure for storing the data processing request history, and a data processing pattern of another node are predicted based on the data processing request history, and data is preliminarily stored in this node. A data processing prediction procedure for determining whether to distribute or not, and a data pre-distribution procedure for distributing data determined to be pre-distributed in this data processing prediction procedure to the other nodes are executed.

  According to the present invention, a data processing pattern of another node is predicted based on a history of data processing requests, it is determined whether or not data is to be distributed in advance to this node, Since it is distributed to the nodes, it is possible to reduce the delay in data acquisition. The reason is that by predicting future data processing and distributing the data to other nodes in advance if necessary, the pre-distributed node will automatically receive the data when a request to acquire the data occurs. This is because it can be acquired immediately from the data storage means of the node. In the present invention, data can also be distributed to a node that does not make a data acquisition request. The reason is that data is actively distributed to all or some nodes by predicting future data processing and distributing data in advance as necessary.

  Further, in the present invention, data is distributed in advance using the information of the routing table stored in the routing table storage means, so that it is not necessary for the data pre-distribution means to separately manage information for distribution.

  Further, according to the present invention, the cost of data acquisition when the data processing pattern from the past to the present is obtained based on the history of the data processing request, the future data processing pattern is predicted from the data processing pattern, and the data is not distributed in advance. And the cost for data pre-distribution, the data processing prediction means can determine whether or not to distribute data in advance.

  Further, in the present invention, it is possible to distribute data in advance only to a group of specific nodes by predicting a data processing pattern only for the pre-distribution range with a specific group of nodes as a pre-distribution range. .

  Further, in the present invention, it is assumed that the data acquisition frequency from the past to the present and the future data acquisition frequency in other nodes have a linear relationship, so that the data acquisition frequency from the past to the present to the future data acquisition frequency. Can be predicted as a data processing pattern.

[First Embodiment]
Next, the best mode for carrying out the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing the configuration of a node in the distributed data management system according to the first embodiment of the present invention. Since the configuration of the peer-to-peer network to which the distributed data management system of the present embodiment is applied is as shown in FIG. 4, it is assumed that the network is 100 and the nodes are 200. Although only one node 200 is illustrated in FIG. 1, it goes without saying that a plurality of nodes 200 having the same configuration exist as shown in FIG.

  As shown in FIG. 1, the node 200 includes a data storage unit 201, a data processing unit 202, a communication processing unit 203, a route table processing unit 204, a route table storage unit 205, a data processing prediction unit 206, The data processing request history storage unit 207 and the data pre-distribution unit 208 are configured. Each means of the node 200 generally operates as follows.

The data storage unit 201 stores a part of data stored in the entire distributed hash table, that is, data managed by the own node.
When the data processing unit 202 receives the data processing request from the communication processing unit 203, the data processing unit 202 outputs the data processing request to the data processing prediction unit 206 and executes the data processing in response to the data processing request. For example, the result of the data processing is returned to the requesting node 200 or the client (not shown) through the communication processing means 203.

  When the communication processing unit 203 receives a message from another node 200 or a client via the network 100, if the message is a data processing request, the communication processing unit 203 outputs the request to the data processing unit 202. If it is an update request, the request is output to the routing table processing unit 204, and if any request is necessary, the request is transferred to another node 200.

When the routing table processing unit 204 receives a routing table update request from the communication processing unit 203, the routing table processing unit 204 updates the routing table stored in the routing table storage unit 205. If necessary, the routing table processing unit 204 creates a new one for another node 200. Send a routing table update request to Further, the routing table processing unit 204 determines the necessity and destination of message transfer.
The route table storage unit 205 stores a route table managed by the route table processing unit 204. The routing table is constructed using a distributed hash table technique.

  The data processing prediction unit 206 receives a data processing request from the data processing unit 202 sequentially or periodically, and stores this data processing request as a history in the data processing request history storage unit 207. The data processing prediction unit 206 predicts future data processing based on the data processing request history stored in the data processing request history storage unit 207, and determines that data should be distributed in advance. The pre-distribution means 208 is instructed to pre-distribute the data.

When the data pre-distribution unit 208 receives an instruction for data pre-distribution from the data processing prediction unit 206, the data pre-distribution unit 208 takes out the instructed data from the data storage unit 201 and sends this data to all but the own node via the network 100. Are pre-delivered to the data pre-distribution means 208 of the node 200 or some of the nodes 200.
The node 200 that has received the pre-distributed data stores the data received by the data pre-distribution means 208 in the data storage means 201.

  Reference information for determining whether or not certain data should be distributed in advance includes, for example, the number of acquisitions of the data per unit time (the number of distributions when viewed from the distribution side) and the remaining valid data There are linear prediction acquisition times in time. In addition, when the data is updated data, the acquisition frequency (distribution frequency) of old data before the update may be used as reference information. When the similarity of data can be determined, the data The acquisition frequency of similar data may be used as reference information.

In addition, the pre-delivery range of data, that is, the node of the pre-delivery destination is limited in advance, and it is determined whether or not the data should be pre-delivered using the reference information of the data related to only this pre-delivery range Also good.
The data acquisition frequency and the number of data acquisitions have the same meaning as the reference information. However, when the number of data acquisitions is determined without setting a clear period, the data acquisition frequency is used, and the data acquisition frequency is determined for a specific period. Is the number of data acquisitions.

Next, the overall operation of the present embodiment will be described in detail with reference to the flowcharts of FIGS.
First, the communication processing unit 203 of the node 200 receives a message from another node 200 or a client via the network 100 (step A1). The communication processing unit 203 that has received the message distributes the request according to the type of request included in the message. That is, if the request included in the message is a data processing request, the request is output to the data processing unit 202, and if the request is a routing table update request, the request is output to the routing table processing unit 204 (step A2).

  When receiving a data processing request from the communication processing unit 203, the data processing unit 202 processes the data in response to the data processing request (step A3). There are two types of data processing requests: data registration requests and data acquisition requests.

  When the data processing unit 202 receives a data registration request, the data processing unit 202 registers the data included in the request in the data storage unit 201, and when a reply to the request is necessary (for example, when it is predetermined that a reply should be made) The data processing result indicating that the data registration is completed is returned to the requesting node 200 or the client via the communication processing unit 203 and the network 100. Further, when receiving a data acquisition request, the data processing unit 202 acquires data corresponding to the request from the data storage unit 201, and acquires the acquired data via the communication processing unit 203 and the network 100. Or reply to the client.

Next, the data processing unit 202 outputs the data processing request received from the communication processing unit 203 to the data processing prediction unit 206, and the data processing prediction unit 206 uses the data processing request as a history to store data processing request history storage unit 207. (Step A4).
Subsequently, the data processing prediction unit 206 refers to the history of data processing requests stored in the data processing request history storage unit 207, and obtains data acquisition patterns (data acquisition count or data acquisition) from past to present for certain data. Frequency) from the history, predicting future data acquisition patterns for this data, and determining that the cost of data acquisition (number of messages and message volume) without pre-delivery exceeds the cost of data pre-delivery The data pre-distribution means 208 is instructed to pre-distribute the data (step A5).

When receiving the data pre-distribution instruction, the data pre-distribution means 208 pre-distributes the instructed data to all the nodes 200 or some of the nodes 200 other than the own node via the network 100 (step A6).
In the node 200 that has received the pre-distributed data, the data received by the data pre-distribution means 208 is stored in the data storage means 201 of the own node.

  On the other hand, when receiving the routing table update request from the communication processing unit 203, the routing table processing unit 204 updates the routing table stored in the routing table storage unit 205 in response to the routing table update request (step A7). The routing table update request is generated when a new node 200 is added to the network 100, for example. Subsequently, the route table processing means 204 determines whether or not it is necessary to newly update the route table (step A8). When the routing table processing unit 204 determines that the routing table needs to be updated anew, the communication processing unit 203 transmits a new routing table update request via the network 100 (step A9).

  Next, the routing table processing unit 204 determines whether or not it is necessary to transfer the message received by the communication processing unit 203 in Step A1 (Step A10). As a case where the message needs to be transferred, for example, the message may not be addressed to the own node. When the routing table processing unit 204 determines that the message needs to be transferred, the routing table processing unit 204 determines the transfer destination of the message based on the routing table information stored in the routing table storage unit 205. When the routing table processing unit 204 determines that the transfer is necessary, the communication processing unit 203 transfers the message to the transfer destination node 200 via the network 100 (step A11).

  In step A2 of FIG. 2, a single message received by the communication processing unit 203 may include a plurality of requests. Step A5 and step A8 may be executed by an event other than receiving a message, for example, a regular event.

  Next, the effect of this embodiment will be described. In the present embodiment, the history of data processing requests is stored in the data processing request history storage means 207, and the data processing prediction means 206 uses the history of data processing requests to predict the pattern of future data acquisition requests. If it is determined that the cost (number of messages and message volume) can be reduced by distributing the data in advance, the data is pre-distributed and the subsequent acquisition of the data is immediately completed at the node where the data has been distributed in advance. Therefore, the time required for data acquisition as a whole can be reduced.

  In the distributed data management system using the Chord system and a general distributed hash table technology, the cost for acquiring data once is O (log (N)), and the cost for general data distribution is O (N Therefore, when the number of times of acquisition of certain data is expected to be O (N / log (N)) times in the future, there is no increase in cost due to pre-distribution of data.

[Second Embodiment]
Next, the best mode for carrying out the second invention of the present invention will be described in detail with reference to the drawings. FIG. 3 is a block diagram showing the configuration of a node in the distributed data management system according to the second embodiment of the present invention. The same components as those in FIG. 1 are denoted by the same reference numerals.
This embodiment is different from the configuration of the first embodiment shown in FIG. 1 in that the route table storage unit 205 and the data pre-distribution unit 208a are connected.

In the present embodiment, the data pre-distribution unit 208a determines the data pre-distribution destination using the information of the route table stored in the route table storage unit 205 when the data is pre-distributed.
In the first embodiment, the data pre-distribution unit 208 needs to manage information for pre-distribution, but in this embodiment, the data pre-distribution unit 208a is stored in the routing table storage unit 205. By using the information in the routing table, there is an effect that the data pre-distribution means 208a does not need to separately manage information for distribution.

  In general, in the distributed hash table technique, a network is configured by managing a routing table in order to reach an arbitrary node with O (log (N)) hops. This network can be regarded as a tree rooted at an arbitrary node. There are many cases. Therefore, in the present embodiment, when the data pre-distribution means 208a pre-distributes data, pre-distribution using the tree eliminates the need to prepare a new network structure.

[Third Embodiment]
Next, the operation of the best mode for carrying out the present invention will be described using a specific example. In the present embodiment, the first embodiment will be described more specifically. In this embodiment, a Chord system is assumed as an example of the distributed hash table technique, and an application layer multicast having a binary tree structure is assumed as a technique for pre-distributing data. Other examples of the distributed hash table technique include Kademlia and Pastry. As a technique for data pre-distribution, not only application layer multicast but also IP multicast can be used.

The data storage means 201, data processing means 202, communication processing means 203, route table processing means 204, and route table storage means 205 shown in FIG. 1 are components of the Chord system. The details of the Chord system are disclosed in, for example, Japanese Patent Application Laid-Open No. H10-228707.
In the present embodiment, unlike a normal Chord system, a data processing request is always output to the data processing means 202 for two reasons: obtaining data distributed in advance and recording a history. Like that.

  Hereinafter, the data processing prediction unit 206 excluding the components of the Chord system, the data processing request history storage unit 207, and the data pre-distribution unit 208 will be described in detail. The data processing request history storage means 207 is a database on a memory or file system, and stores, for example, a CSV (Comma Separated Values) format file. One line of the CSV file includes <time>, <acquisition / registration>, <node information>, <key>, <data>, and <expiration date>. However, <data> and <expiration date> are only required for data registration requests. Also, <data> and <expiration date> can be omitted if they overlap with the data storage unit 201. <Node information> is configured by an ID, group information, and the like provided by a distributed hash table (route table) in addition to an IP address.

  When the data processing prediction unit 206 receives a data processing request from the data processing unit 202, the data processing prediction unit 206 adds the data processing request to the CSV file of the data processing request history storage unit 207 sequentially or periodically. In addition, the data processing prediction unit 206 preliminarily sets the data corresponding to the data processing request or all the data stored in the data storage unit 201 in synchronization or asynchronously with the appending to the CSV file. Determine whether or not to deliver.

  In order to determine whether or not data should be distributed in advance, it is necessary to estimate the cost for distribution in advance or dynamically. Here, assuming that the data size is sufficiently small and ignoring the storage cost, the cost is the amount of message transmission / reception. When the data size is fixed, the amount of message transmission / reception is equal to the number of messages, and the number of messages is considered as cost below.

  In the binary tree multicast, all nodes receive a message exactly once, so the cost of pre-distributing data to N nodes is N. By the way, when a distributed hash table is used, it generally costs O (log (N)) for data acquisition, and the Chord system averages log (N) / log (2) / 2 for data acquisition. costly. Assuming that the number of future acquisitions or the number of acquisition nodes for a certain data is M, the expected value of the total cost for this data acquisition is M × log (N) / log (2) / 2.

  The data pre-distribution means 208 determines that the data should be pre-distributed if the total expected value M × log (N) / log (2) / 2 of this cost is greater than the data pre-delivery cost N. In the network used by the data pre-distribution means 208, all the nodes 200 constitute one binary tree. Data is distributed in advance from the node 200 corresponding to the root of the binary tree. A plurality of binary trees may be configured so that the load is not concentrated on the node 200 corresponding to the root.

  In the node 200 that has received the pre-distribution of data, the pre-distributed data is stored in the data storage unit 201. Thereafter, when an acquisition request for the data occurs in the node 200, the data can be acquired from the data storage unit 201, so that the data acquisition is completed immediately.

  Next, a method in which the data pre-distribution unit 208 predicts the future data acquisition frequency (data acquisition frequency) M will be described. There are two methods for predicting M. The first method is a method that assumes that the data acquisition frequency from the past to the present and the future data acquisition frequency have a linear relationship. In this method, for example, if the time from the data registration time to the present is T1, the time from the present time to the data expiration date is T2, and the number of data acquisition from the data registration time to the present is K, the data validity from the present time The number of data acquisitions until the deadline can be predicted as M = K × T2 / T1. For example, if the average value of the number of data acquisitions in unit time T (for example, 1 minute) is L, and T2 is used, the number of data acquisitions from the present to the data expiration date is predicted as M = L × T2 / T. can do.

  In the prediction example of M = K × T2 / T1 or M = L × T2 / T, a time T2 from the present to the expiration date of the data is necessary. If T2 is not defined, for example, T2 is set to T1. You can halve it. Further, for example, when updating certain data, M can be predicted using the past acquisition frequency of the data as the future acquisition frequency. For example, when the similarity of data can be determined, M can be predicted using the past acquisition frequency of data similar to the data as the future acquisition frequency of the data.

  The second method is a method that assumes that the data acquisition frequency from the past to the present and the future data acquisition frequency are in a non-linear relationship. In this method, for example, if the number of data acquisitions from the time of data registration to the present is K, the future number of data acquisitions can be predicted as M = K / 2. Further, for example, assuming that the data acquisition frequency decreases monotonously, the slope J of the decrease in the data acquisition frequency is estimated from the data acquisition pattern from the data registration time to the present. If the current data acquisition frequency is L, the future data acquisition frequency can be predicted as M = L × L / J / 2. Further, for example, assuming that the data acquisition frequency decreases according to the quadratic function, the future data acquisition frequency M may be predicted by estimating the quadratic function by the least square method from the data acquisition pattern from the time of data registration to the present. it can.

  Further, in both cases of the first method and the second method, the data pre-delivery range is limited, and the data acquisition frequency (the number of data acquisitions) in the future using the acquisition pattern of only this pre-delivery range. M may be predicted. In this case, the data pre-distribution means 208 determines that the total expected value M × log (N) / log (2) / 2 for the data acquisition when the pre-distribution cost is not pre-distributed than the data pre-distribution cost of only the pre-delivery range If it is larger, it is determined that the data should be pre-distributed, and the data is pre-distributed in the range.

  As the pre-distribution range, for example, a group of nodes 200 grouped using a subnet of an IP network, an IP multicast group, or a group of nodes 200 grouped using group information provided by a distributed hash table (route table). There are groups.

  When grouping by using the subnet of the IP network, the IP address is regarded as a bit string, and nodes having IP addresses having the same prefix using a certain prefix length are grouped. The data pre-distribution means 208 predicts the data acquisition frequency M for each group, and the expected value M × log (N ′) / log (2) / 2 of the total cost for data acquisition when not pre-distributed is pre-distributed. If it is larger than the range data pre-delivery cost N ′, it is determined that the data should be pre-delivered, and the data is pre-delivered to the group in the pre-delivery range. N ′ is the number of nodes in the group.

  In addition, when performing pre-distribution of data using an IP address prefix, an application layer multicast binary tree is configured to connect nodes having a long IP address prefix match length close to each other.

  Note that the node 200 in the first to third embodiments can be realized by, for example, a computer including a CPU, a storage device, and an interface, and a program that controls these hardware resources. A program for operating such a computer is provided in a state of being recorded on a recording medium such as a flexible disk, a CD-ROM, a DVD-ROM, or a memory card. The CPU writes the read program into the storage device, and executes the processing described in the first to third embodiments according to this program.

  The present invention can be applied to uses such as distributing information from one node to another on a network. Also, the present invention can be applied to uses such as distributing information without specifying a receiving node.

It is a block diagram which shows the structure of the node in the distributed data management system which concerns on the 1st Embodiment of this invention. It is a flowchart which shows operation | movement of the distributed data management system which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structure of the node in the distributed data management system which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows the structural example of a peer to peer network. It is a block diagram which shows the structure of the node in the conventional Chord system.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 ... Network, 200 ... Node, 201 ... Data storage means, 202 ... Data processing means, 203 ... Communication processing means, 204 ... Path table processing means, 205 ... Path table storage means, 206 ... Data processing prediction means, 207 ... Data Processing request history storage means, 208, 208a, data pre-delivery means.

Claims (9)

In a distributed data management system in a peer-to-peer network,
Each node connected to the network
Communication processing means for communicating with other nodes via the network;
Data storage means for storing data;
Data processing means for processing data stored in the data storage means in response to a data processing request received by the communication processing means;
Data processing request history storage means for storing a history of the data processing request;
A data processing prediction unit that predicts a data processing pattern of another node based on the history of the data processing request and determines whether or not to distribute data to this node in advance;
A distributed data management system comprising: a data pre-distribution unit that distributes the instructed data to the other nodes when receiving a distribution instruction from the data processing prediction unit. The distributed data management system according to claim 1, wherein
Furthermore, route table storage means for storing the route table;
A distributed data management system comprising: a routing table processing unit that determines a transfer destination of a message received by the communication processing unit based on routing table information stored in the routing table storage unit. The distributed data management system according to claim 2,
The distributed data management system, wherein the routing table is constructed using a distributed hash table technique. In the distributed data management system according to claim 2 or 3,
The distributed data management system, wherein the data pre-distribution means distributes data in advance using information of a routing table stored in the routing table storage means. The distributed data management system according to any one of claims 1 to 4,
The data processing prediction unit obtains the data processing pattern from the past to the present based on the history of the data processing request, predicts a future data processing pattern from the data processing pattern, and obtains data when not pre-distributed A distributed data management system that determines data pre-distribution when it is determined that the cost of data exceeds the cost of data pre-distribution. The distributed data management system according to any one of claims 1 to 5,
The data processing predicting means predicts the data processing pattern only for the pre-delivery range with a specific group of nodes as the pre-delivery range,
The distributed data management system, wherein the data pre-distribution means distributes data in advance only to the group of nodes. The distributed data management system according to any one of claims 1 to 6,
The data processing prediction means predicts a future data acquisition frequency as the data processing pattern on the assumption that the data acquisition frequency from the past to the present in the other node and the future data acquisition frequency have a linear relationship. Distributed data management system characterized by The distributed data management system according to claim 7,
When the time from the data registration time to the present is T1, the time from the present time to the data expiration date is T2, and the data acquisition count from the data registration time to the present is K, the data processing prediction means A distributed data management system characterized in that the predicted value of the number of times of data acquisition is K × T2 / T1. In a distributed data management method in a peer-to-peer network,
Each node connected to the network
A data processing procedure for processing the data stored in the data storage procedure of the own node in response to the data processing request received by the communication processing means of the own node;
A data processing request history storage procedure for storing a history of the data processing request;
A data processing prediction procedure for predicting a data processing pattern of another node based on the history of the data processing request and determining whether or not to distribute data to this node in advance;
A distributed data management method comprising: executing a data pre-distribution procedure for distributing data determined to be pre-distributed in the data processing prediction procedure to the other nodes.

Images