JP2009289161A - Clustered storage system, node device thereof, and method and program for controlling data read/write - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To freely combine devices storing content, in a clustered storage system which stores content in a plurality of devices in a redundant manner. <P>SOLUTION: In a node device 100 of the clustered storage system which stores content in a plurality of device 110 in a redundant manner, layout information 131 is created which is to be referred in determining the devices to which data are written, and a basket ID is created using a hash value which is calculated using a content key (K) of the desired content. In the layout information 131, the device IDs of the plurality of devices 110 storing the content can be set as device IDs corresponding to the basket ID. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a cluster type storage system.

  Conventionally, there is a cluster type storage system that stores data in a plurality of distributed devices. This cluster type storage system has expandability (scalability) in which throughput, the number of I / Os per unit time, and data storage capacity can be flexibly improved by adding devices.

As this cluster type storage system, there is Dynamo developed by Amazon (see Non-Patent Document 1). This Dynamo is a storage system that uses a distributed hash table (DHT) and manages data (content) write destinations with key-value pairs. The key-value pair is a method of assigning and storing a content key when storing the content. When the system reads the content, the content key is designated and read. In Dynamo, load distribution between node devices (between devices) is performed by equally dividing the DHT space, introducing virtual nodes, and the like. In this Dynamo, a device for storing content is determined by a hash value H (K) calculated by substituting the content key K for the hash function H. Here, in the case where the content is redundantly stored, the content is duplicated and stored in the adjacent device group in the DHT space.
Dynamo, [online], [Search May 9, 2008], Internet, <URL: http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html>

  However, according to the description of Non-Patent Document 1, when the average load of each node device for 30 minutes was measured with Dynamo, the number of node devices whose load was 15% of the average value was 10% to the total number of nodes. Present 20%. There are various causes for such a load imbalance. For example, concentrated access to content of a specific key by an application can be mentioned. As a specific example, content C1 is stored in three devices D1, D2, and D3 adjacent in the DHT space, and another content C2 is stored in three devices D2, D3, and D4 adjacent in the DHT space. Think of a system. In the case of this system, when an application that frequently accesses the contents C1 and C2 appears, the load on the devices D2 and D3 becomes particularly high, and there is a problem that the load is unbalanced between devices in the system (between node devices). . Here, in Dynamo, the combination of redundant devices is determined to be adjacent devices in the DHT space. Therefore, in order to eliminate the uneven load, it is not possible to change the combination of devices for storing contents redundantly to an arbitrary combination.

  SUMMARY OF THE INVENTION An object of the present invention is to solve the above-mentioned problems and to enable any combination of devices as storage destinations of contents in a cluster type storage system for storing contents redundantly in a plurality of devices. And

In order to solve the above-mentioned problem, the invention according to claim 1 is provided with a plurality of node devices that read and write data, and a cluster type storage that stores the data redundantly in a plurality of devices accommodated in the node device The node device in the system,
For each basket ID that is identification information obtained based on a hash value of a content key that is identification information of the data, arrangement information indicating identification information of a plurality of the devices that store the data, and identification of the device For each piece of information, a write / read request or read of the data, including a storage unit that stores address information indicating identification information of a node device that accommodates the device, and a content key of the data to be read or written From the arrangement information, using an input unit that receives an input of a request, a hash calculation unit that calculates the basket ID corresponding to the content key based on a hash value of the content key, and the calculated basket ID as a key, The identification information of the device corresponding to the basket ID is searched and before this search Referring to the identification information of the device and the address information, the identification information of the node device that accommodates the device corresponding to the searched identification information is specified, and the node device of the identified identification information is transferred to the device A routing unit for transmitting a data read request or a write request corresponding to the content key, and a data receiving unit for receiving a result of the data read request or write request from the node device.

  According to an eighth aspect of the present invention, there is provided a cluster type storage system comprising a plurality of node devices that read and write data, wherein the data is stored redundantly in a plurality of devices accommodated in the node device. For each basket ID that is identification information obtained based on the hash value of the content key that is information, for each of the arrangement information indicating the identification information of the plurality of devices storing the data, and for each identification information of the device, The node device including a storage unit that stores address information indicating identification information of a node device that accommodates the device includes a content key of the data to be read or written, A step of accepting an input of a read request, and a hash value of the content key A step of calculating the basket ID corresponding to the content key on the basis, a step of searching for identification information of the device corresponding to the basket ID from the arrangement information using the calculated basket ID as a key, A step of identifying identification information of a node device that accommodates a device corresponding to the retrieved identification information with reference to the retrieved identification information of the device and the address information, and to the node device of the identified identification information Transmitting a data read request or write request corresponding to the content key to the device, and receiving a result of the data read request or write request from the node device. Data read and write control method.

  By doing so, the node device of the cluster type storage system (hereinafter abbreviated as “system”) becomes the data write destination (or read destination) using the arrangement information that clearly identifies the identification information of the device to be made redundant. Determine the device. Therefore, a system administrator or the like can freely set a device as a data write destination (or read destination) determined by each node device by setting this arrangement information.

  According to a second aspect of the present invention, the node device according to the first aspect includes the content key of the data and identification information of the plurality of devices that are storage destinations of the data via the input unit. When receiving the input of the setting information of information, the hash calculation unit calculates the basket ID corresponding to the content key based on the hash value of the content key included in the setting information, the node device, An arrangement information management unit that creates the arrangement information in which the calculated basket ID is associated with the identification information of the plurality of devices included in the setting information is provided.

  By doing so, a system administrator or the like can freely set a device as a data write destination (or read destination) determined by each node device.

  According to a third aspect of the present invention, in the node device according to the second aspect, in order to change the device that is the storage destination of the data, the content key of the data to be changed via the input unit, When receiving input of change information including identification information of a device that is a new storage destination of data, the arrangement information management unit corresponds to the content key of the data to be changed calculated by the hash calculation unit The identification information of the device corresponding to the basket ID is searched from the arrangement information using the basket ID as a key, and the identification information of the searched device in the arrangement information is stored in the new storage destination device. Change to identification information, when the routing unit searches for the identification information of the device, referring to the changed arrangement information, Wherein the retrieving the identity of the serial device.

  According to a ninth aspect of the present invention, in the data read / write control method according to the eighth aspect, since the node device changes a device that is a storage destination of the data, the content key of the data to be changed And calculating the basket ID corresponding to the content key of the data to be changed when the input of the change information including the identification information of the device as the new storage destination of the data is received, and the calculated Using the basket ID as a key, the step of searching the device identification information corresponding to the basket ID from the placement information, and the device information that becomes the new storage destination of the device identification information in the placement information When the device identification information is searched for, Referring to further the arrangement information, characterized by searching for identification information of the device.

  In this way, for example, in order to eliminate the uneven processing load in the system, the data stored in the node device with a high processing load by the administrator or the like is transferred to the node device with a relatively low processing load. If the migration is performed, if the migration destination device is set as the storage destination of the data in the arrangement information, the node device is allowed to execute read processing and write processing on the migration destination device after data migration. Can do.

  According to a fourth aspect of the present invention, in the node device according to any one of the first to third aspects, when the data read request or write request is received from another node device, A data transmission unit that executes the read or write process on the data stored in a device included in the node apparatus, and transmits the execution result to the node apparatus that is a transmission source of the data read request or write request; It is characterized by providing.

  In this way, even when a data read request or write request is received from another node device, it is possible to execute processing according to this request and return the execution result to the other node device.

  According to a fifth aspect of the present invention, when the arrangement information management unit of the node device according to any one of the second to fourth aspects changes the arrangement information in the storage unit, the cluster type Sending the changed placement information to other node devices in the storage system, and receiving the changed placement information from the other node devices, storing the changed placement information in the storage unit, The routing unit searches for the identification information of the device with reference to the changed arrangement information when searching for the identification information of the device.

  In this way, when the placement information is changed in any node device of the system, each node device can determine the data read destination or write destination based on the changed placement information.

  According to a sixth aspect of the present invention, there is provided a cluster type storage system comprising a plurality of node devices according to any one of the first to fifth aspects.

  By doing so, a cluster storage system can be realized by the node device according to any one of claims 1 to 5.

  According to a seventh aspect of the invention, the cluster type storage system according to the sixth aspect further includes a monitoring device that monitors a processing load in each of the node devices in the cluster type storage system, When detecting a bias in processing load between the node devices, the node device uses the placement information for a basket ID corresponding to identification information of a device accommodated in the node device having a relatively high processing load. The device identification information corresponding to the basket ID is changed to device identification information accommodated in the node device having a relatively low processing load.

  By doing in this way, when the processing load is uneven between the node devices of the system, the node device can automatically change the arrangement information. Therefore, it is possible to reduce the trouble of changing the arrangement information when the processing load is uneven between the node devices of the system.

  The invention described in claim 10 is a program for causing the node device, which is a computer, to execute the data read and write control method according to claim 8 or claim 9.

  According to such a program, it is possible to cause a general computer to execute the data read and write control method according to claim 8 or claim 9.

  According to the present invention, in a cluster type storage system (system) in which contents are stored redundantly in a plurality of devices, an administrator or the like can arbitrarily set a combination of devices that are contents storage (storage) destinations. Therefore, when load imbalance occurs between devices, it is easy to change the device that is the storage destination of the content in order to reduce the load imbalance. In addition, the system administrator can arbitrarily set a combination of devices as storage destinations of the content according to the system operating environment, so that the system availability can be improved.

  Hereinafter, the best mode for carrying out the present invention (hereinafter referred to as an embodiment) will be described.

≪Overview≫
First, an overview of the cluster type storage system of this embodiment will be described with reference to FIG. FIG. 1 is a diagram showing a configuration example of a cluster type storage system according to the present embodiment.

  As shown in FIG. 1, a cluster storage system (hereinafter abbreviated as “system”) includes a plurality of node devices 100 (100A, 100B, 100C, 100D). Here, a case where contents corresponding to the same content key (content identification information) are redundantly stored in three devices 110 (110C, 110E, 110G) will be described as an example. Here, it is assumed that the three devices 110 (110C, 110E, and 110G) are the devices 110 in the node apparatuses 100B, 100C, and 100D, respectively.

  The client device 20 is a terminal device that includes an application for accessing the system. When each node device 100 accepts access from the client device 20, the node device 100 reads or writes content from / to a predetermined node device 100 based on a data (content) read request or write request from the client device 20.

  In such a cluster-type storage system, each node device 100 has location information 131 indicating the storage destination of the content and address information indicating identification information such as an IP address of the node device 100 in which the device 110 is accommodated. 132. When the placement information 131 is updated, the node device 100 transmits the updated placement information 131 to each of the other node devices 100. Then, each node device 100 determines a device 110 that is a read destination or a write destination of the content based on the updated arrangement information 131.

  For example, when the node device 100A receives a read / write request (content read request or write request) designating the content key (K) from the client device 20, the node device 100A assigns the content key (K) to the hash function H and stores the basket. An ID (eg, “1”) is calculated. Then, the node device 100A searches the device ID (for example, “110C, 110E, 110G”) corresponding to the basket ID from the arrangement information 131 using the calculated basket ID (for example, “1”) as a key. To do. Further, the node device 100A refers to the address information 132 and specifies the node ID (such as the IP address of the node device 100) of the node device 100 that accommodates the device 110 corresponding to the searched device ID. Then, the content read / write request is transmitted to each of the devices 110 of the identified node device 100. When the node device 100A receives the execution result for the read / write request from the node devices 100B, 100C, and 100D, the node device 100A returns the execution result to the client device 20.

  Here, the arrangement information 131 included in each node device 100 is information indicating one or more device IDs (identification information of the device 110) corresponding to the basket ID for each basket ID as illustrated in FIG. . This basket ID is a value obtained by substituting the content key (K) into the hash function H, H (K). The reason why the basket ID is determined using such a hash function is to prevent the load from being biased between baskets in the system.

  Note that the device ID group corresponding to the basket ID in the arrangement information 131 of the present embodiment can be set by an instruction input via the input / output unit 11. That is, in the conventional cluster type storage system, when content is stored redundantly in a plurality of devices, the combination of the devices can be made only with devices adjacent in the DHT space. However, according to the node apparatus 100 of the present system, when content is stored redundantly in a plurality of devices, the combination of the devices can be arbitrarily set by an administrator or the like. Therefore, it becomes easy to transfer data between devices in the system.

≪Configuration≫
The configuration of the node device 100 will be described with reference to FIG. As illustrated in FIG. 1, the node device 100 includes one or more devices 110 and a content management unit 10. The device 110 stores content for each content key (K) and basket ID described above. The device 110 is realized by an HDD (Hard Disk Drive) or the like, for example. When the content management unit 10 receives a content read request or write request (hereinafter, abbreviated as “read / write request” as appropriate) from the client device 20, the content management unit 10 refers to the arrangement information 131 and stores the content. 110 is searched. A read / write process (read process or write process) of the content is executed on the searched device 110. Further, the content management unit 10 updates or creates the placement information 131 based on a setting instruction from the client device 20 or the like, and exchanges the placement information 131 with another node device 100.

  Such a content management unit 10 includes an input / output unit 11, a processing unit 12, and a storage unit 13.

  The input / output unit 11 includes a communication interface for performing communication with other node devices 100 and client devices 20. The processing unit 12 is realized by a program execution process by a CPU (Central Processing Unit) included in the node device 100. Further, the storage unit 13 includes a storage medium such as a random access memory (RAM), a read only memory (ROM), a hard disk drive (HDD), and a flash memory. The storage unit 13 stores a program for realizing the function of the node device 100.

  The input / output unit (input unit and output unit) 11 receives an input of a content read / write request including a content key, or receives an input of a setting instruction for the arrangement information 131. In addition, content is transmitted / received to / from other node devices 100, and results of read / write processing are transmitted / received.

  The processing unit 12 executes read / write processing on the device 110 of the node device 100 and creates the arrangement information 131. The processing unit 12 includes a hash calculation unit 121, an arrangement information management unit 122, an address information management unit 123, a routing unit 124, a data reception unit 125, and a data transmission unit 126. .

  The hash calculator 121 assigns the content key (K) to the hash function H and calculates the hash value H (K).

  The arrangement information management unit 122 newly creates the arrangement information 131 based on the setting input from the input / output unit 11 or changes the contents of the existing arrangement information 131. For example, when the arrangement information 131 is newly created, the arrangement information management unit 122 receives a setting input including a content key of content and device IDs of a plurality of devices 110 that store the content in a redundant manner. The hash calculator 121 calculates a basket ID based on the hash value of the content key. Then, the arrangement information management unit 122 creates the arrangement information 131 by associating the basket ID with the device ID included in the setting input. The arrangement information management unit 122 changes the contents of the existing arrangement information 131 as follows. First, when the arrangement information management unit 122 receives input of change information including the content key of the content to be changed and the device ID of the device 110 that is the new storage destination of the content, the hash calculation unit 121 The basket ID corresponding to the content key of this content is calculated. Then, the arrangement information management unit 122 searches the arrangement information 131 for a device ID corresponding to the basket ID using the basket ID as a key. Then, the device 110 corresponding to this device ID is rewritten with the device ID of the device 110 that becomes a new storage destination. Further, the arrangement information management unit 122 transmits the arrangement information 131 created by its own node apparatus 100 to the other node apparatus 100 and receives the arrangement information 131 from the other node apparatus 100. That is, the arrangement information 131 is exchanged between the node devices 100. The created or updated placement information 131 and the placement information 131 received from another node device 100 are stored in the storage unit 13.

  The address information management unit 123 updates the contents of the address information 132 based on the input from the input / output unit 11.

  When the routing unit 124 receives an input of a content read / write request including the content key (K) of the content to be processed from the client device 20 via the input / output unit 11, the hash calculation unit 121 causes the hash value of the content key to be received. Calculate H (K). Then, a basket ID corresponding to the content key (K) is obtained from the hash value H (K). Here, a case where the hash value H (K) is used as it is as a basket ID will be described as an example.

  The routing unit 124 searches the arrangement information 131 for a device ID corresponding to this basket ID. Then, the routing unit 124 searches the address information 132 for the node ID of the node device 100 that accommodates the device 110 corresponding to this device ID. Then, the routing unit 124 transmits a read / write request for content stored in the device 110 corresponding to the device ID to the node device 100 corresponding to the searched node ID. When the arrangement information 131 in the storage unit 13 is changed, the routing unit 122 searches for the device ID with reference to the changed arrangement information 131.

  Further, the read request transmitted by the routing unit 124 includes the device ID, basket ID, content key (K), and the like of the device 110 from which the content is read. In addition to the device ID, basket ID, and content key (K) of the device 110 that is the content write destination, the write request includes the data that is the write content and the time that is the time when the write request is received from the client device 20. Including stamps. This time stamp is, for example, “20070501100015 (May 1, 2007 10:00:15)” or the like.

  The data receiving unit 125 receives a response to a content read / write request from another node device 100. The received response is stored in the RAM of the storage unit 13 or the like.

  When the data transmission unit 126 receives a content read / write request from another node device 100, the data transmission unit 126 executes a read process or a write process of the content stored in the device 110. Then, the processing result is returned to the node device 100 which is the transmission source of this read / write request.

  The protocol used for transmitting the content read / write request and the request result (including the response result and content) uses, for example, HTTP (HyperText Transfer Protocol) or a data communication protocol corresponding to HTTP.

  The storage unit 13 stores arrangement information 131 and address information 132.

  As described above, the arrangement information 131 is information indicating the device ID of the device 110 that stores the content corresponding to the basket ID for each basket ID. For example, in the system illustrated in FIG. 1, the content corresponding to the basket ID “1” is stored redundantly in the devices 110C, 110E, and 110G, and the content corresponding to the basket ID “2” is stored in the devices 110D, 110F, In the case of storing 110H in a redundant manner, “110C, 110E, 110G” is set as a device ID corresponding to the basket ID “1” of the arrangement information 131. Further, “110D, 110F, 110H” is set as a device ID corresponding to the basket ID “2” of the arrangement information 131.

  The address information 132 is information indicating the node ID of the node device 100 that accommodates the device 110 corresponding to the device ID for each device ID. This node ID may be described by a combination of an IP address and a port number.

  In FIG. 1, the content management unit 10 and the device 110 have been described as being included in the same node device 100, but they may be realized as separate devices.

≪Processing procedure≫
Next, the processing procedure of the node device 100 will be described using FIG. 2 with reference to FIG. FIG. 2 is a diagram illustrating a procedure in which the node device of FIG. 1 transmits a content read / write request to another node device. Here, an example will be described in which the node device 100A that has received a content read / write request from the client device 20 transmits the content read / write request to the devices 110C, 110E, and 110G that store the content.

  First, the routing unit 124 (see FIG. 1) of the node device 100A receives a content read request or write request (read / write request) from the client device 20 (S101). Note that the read request includes the content key (K) of the content to be read. The write request includes the content key (K) of the content to be written and data indicating the write content.

  The hash calculator 121 (see FIG. 1) of the node device 100A calculates a basket ID based on the received content key (K) (S102). Then, the routing unit 124 refers to the calculated basket ID and the arrangement information 131, and searches for the device IDs of the devices 110C, 110E, and 110G that are the write destinations of the contents (S103). Next, the routing unit 124 uses the searched device ID as a key, and from the address information 132, the node device 100 (node devices 100B, 100C, 100D) that accommodates the device 110 (device 110C, 110E, 110G) corresponding to this device ID. ) Address (node ID) is searched (S104).

  Then, based on the retrieved address, the routing unit 124 makes a request to read or write the content (read / write) to the node devices 100B, 100C, and 100D that accommodate the devices 110C, 110E, and 110G that are the content write destinations. Request) is transmitted (S105). The read request transmitted here includes the device ID, basket ID, and content key (K) as described above. The write request includes a device ID, a basket ID, a content key (K), data indicating the write content, and a time stamp indicating the date and time when the write request is received.

  Next, referring to FIG. 1, using FIG. 3, a processing procedure when the node device 100A that has transmitted a content read / write request by the procedure shown in FIG. 2 returns a response to the read / write request to the client device 20 will be described. explain. FIG. 3A is a diagram illustrating a procedure in which the node device in FIG. 1 transmits a response to the content read / write request to the client device, and FIG. 3B conceptually illustrates the process of S202 in FIG. FIG.

  In FIG. 2, the node devices 100B, 100C, and 100D that have received the content read request or write request (read / write request) from the node device 100A have this content read request or write request (see FIG. 1) (see FIG. 1). A response to the read / write request is transmitted to the node device 100A (S201).

  Then, the data receiving unit 125 of the node device 100A receives a response to this read request or write request (read / write request) from the node devices 100B, 100C, and 100D. Here, when the routing unit 124 determines that the number of responses received by the data receiving unit 125 has satisfied a predetermined quorum (S202), the received response is returned to the client device 20. As shown in FIG. 3A, the response to the read request and the response to the write request indicate the results of the processing based on the request (success or failure of the processing). If the read process is successful, the response includes the read content.

  By doing so, for example, even when the number of node devices 100 storing the content is large, a response can be returned to the client device 20 without waiting for a response from all the node devices 100.

  Here, the quorum satisfaction determination process in S202 will be described using a specific example with reference to FIG. Here, an example will be described in which the number of node devices 100 that transmitted a read request or a write request (read / write request) is “4” and the quorum is “2”.

  For example, in the case of a write request, a write request is transmitted from the node device 100 to another node device 100, and when the number of results OK (success) received for this request reaches “2”, the node device 100 at this time. Returns OK to the client device 20 (see Examples 1 and 2). That is, when the result NG is included in the received response, the response of the result NG is not counted, and when the number of received responses of the result OK reaches “2”, the node device 100 determines that the client device 20 OK is returned to (see Example 2). Then, a response to the write request received so far is transmitted to the client device 20. As described above, even if the node device 100 receives a response after the number of received responses of the result OK reaches “2”, the response is ignored. Note that the quorum for this write request may be the number of redundant devices 110 itself, or may be smaller than the number of redundant devices 110.

  In the case of a read request, when a read request is transmitted from the node device 100 to another node device 100, the number of received contents having the latest time stamp (for example, “Time = 3”) reaches “2”. The node device 100 returns OK to the client device 20 (see Examples 3 and 4). That is, (1) if the received response contains a result NG, and (2) the content whose time stamp value is not the latest (for example, “Time = 2”) is included. If this occurs (see Example 4), this response is not counted. Then, when the number of received contents having the latest time stamp reaches “2”, the node device 100 returns OK to the client device 20. Then, the content received so far is transmitted to the client device 20. Here, the node device 100 does not return this response to the client device 20 for the response received by the node device 100 after the number of received responses of the result OK reaches “2”. Note that if the content reception process is completed before the node device 100 receives the latest number of received contents reaches “2”, the received content may be returned to the client device 20.

≪Specific example≫
Next, rewriting of the arrangement information 131 when load concentration occurs in the node device 100 of the cluster type storage system will be described using a specific example with reference to FIG. FIG. 4A is a diagram exemplifying the relative proportions of the devices accommodated in each node device and the load in each node device in the cluster type storage system of the present embodiment. FIG. 4B illustrates the arrangement information before rewriting and the arrangement information after rewriting.

  Here, a case where the data centers A, B, and C each include a node device 100 (100A to 100L) that accommodates devices as shown in FIG. 4 in the cluster type storage system will be described as an example. The contents of the placement information 131 of each node device 100 at this time are as shown in the placement information 131A of FIG. The numerical value appended to the left of each node device 100 indicates the relative ratio of the load in each node device 100 of this cluster type storage system. In such a cluster type storage system, the loads of the node devices 100C, 100G, and 100K are relatively high (relative load ratio “2”), and the loads of the node devices 100D, 100H, and 100L are relatively low (relative ratio). “0.67”).

  In such a state, in order to eliminate the load imbalance between the node devices 100, the load of the node devices 100C, 100G, and 100K in FIG. , 100L load should be 3/2 times that of the prior art. For this purpose, for example, the content write destinations corresponding to the basket IDs “12, 13” may be changed to the devices 041, 141, 241, and the devices 042, 142, 242, respectively. With such a change, the arrangement information management unit 122 creates the arrangement information 131B based on an instruction input from the administrator or the like. This arrangement information 131B is obtained by rewriting the content write destination of the basket ID “12, 13” in the arrangement information 131A from the node devices 100C, 100G, 100K to the node devices 100C, 100G, 100K. Then, the rewritten placement information 131B is propagated to each node device 100, and the read / write processing with respect to the content corresponding to the basket ID is performed on the node device 100 indicated by the placement information 131B.

  In this way, when load is concentrated on a specific node device 100 (or device 110) in a system that stores the contents redundantly in each node device 100, the arrangement information 131 is set so as to eliminate the concentration of the load. Can be rewritten.

  In addition to this case, the device ID corresponding to each basket (basket ID) in the arrangement information 131 can be rewritten by the administrator or the like. Therefore, it is possible to freely select a combination of devices 110 for storing contents in a redundant manner on the DHT space in accordance with the operation environment of the system.

  A monitoring device that monitors the processing load in each node device 100 may be provided in the system. When this monitoring device detects a bias in processing load between the node devices 100, the value of this processing load is transmitted to the node device 100. Then, the node device 100 refers to the value of the processing load, and reduces the number of baskets corresponding to the node device 100 having a relatively high processing load in the arrangement information 131, and the node device 100 having a relatively low processing load. You may make it increase the number of the baskets corresponding to. For example, for the basket ID corresponding to the device 110 of the node device 100 having a relatively high processing load in the arrangement information 131, the node device 100 assigns the device 110 corresponding to this basket ID to the node device 100 having a relatively low processing load. The device 110 may be changed.

The basket ID used for the arrangement information 131 is the hash value H (K) of the content key (K), but is not limited to this. For example, the hash value H ′ (K) may be obtained by the hash function H ′ that can obtain a sufficiently larger number of hash values than the number of baskets that the administrator or the like wants to prepare, and the mask value may be used as the basket ID. Therefore, if 2 ¹⁶ baskets are required, the hash value CRC32 for obtaining 2 ³² types of hash values is used to obtain the hash value of the content key (K), and the mask value of the lower 16 bits is used as the basket ID. Good.

  Various methods are conceivable for storing (storing) content in the device 110. For example, (1) one database may be placed in the device 110, and the basket ID, the content key (K), and the content may be stored in association with the database, or (2) the basket ID may be stored in the device 110. A database may be placed for each, and the content key (K) and the content may be stored in association with each database. Further, (3) the contents may be stored in the file system using the basket ID and the content key (K) as a directory name or a file name.

1 is a diagram illustrating a configuration example of a cluster type storage system according to the present embodiment. It is the figure which showed the procedure in which the node apparatus of FIG. 1 transmits the reading / writing request | requirement of a content to another node apparatus. (A) is the figure which showed the procedure in which the node apparatus of FIG. 1 transmits the response with respect to the read / write request | requirement of a content to a client apparatus, (b) showed the process of S202 in (a) notionally. FIG. (A) is the figure which illustrated the relative ratio of the device which each node apparatus accommodates, and the load in each node apparatus in the cluster type storage system of this Embodiment, (b) is the arrangement information before rewriting, It is the figure which illustrated the arrangement information after rewriting.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Content management part 11 Input / output part 12 Processing part 13 Storage part 20 Client apparatus 100 (100A, 100B, 100C, 100D) Node apparatus 110 (110C, 110D, 110E, 110F, 110G, 110H) Device 121 Hash calculation part 122 Arrangement Information management unit 123 Address information management unit 124 Routing unit 125 Data reception unit 126 Data transmission unit 131 (131A, 131B) Arrangement information 132 Address information

Claims (10)

The node device in a cluster type storage system comprising a plurality of node devices for reading and writing data, and storing the data redundantly in a plurality of devices accommodated in the node device,
For each basket ID that is identification information obtained based on a hash value of a content key that is identification information of the data, arrangement information indicating identification information of a plurality of the devices that store the data, and identification of the device For each information, a storage unit that stores address information indicating identification information of a node device that accommodates the device;
An input unit that receives a request to read or write the data including a content key of the data to be read or written;
A hash calculator that calculates the basket ID corresponding to the content key based on the hash value of the content key;
Using the calculated basket ID as a key, the device identification information corresponding to the basket ID is retrieved from the arrangement information, and the retrieved device identification information and the address information are referred to. The identification information of the node device that accommodates the device corresponding to the searched identification information is specified, and the data read request or write request corresponding to the content key to the device is transmitted to the node device of the specified identification information. A routing section;
A node device, comprising: a data receiving unit that receives a result of the data read request or write request from the node device. When receiving the setting information of the arrangement information including the content key of the data and the identification information of the plurality of devices serving as storage destinations of the data via the input unit,
The hash calculation unit calculates the basket ID corresponding to the content key based on a hash value of the content key included in the setting information,
The node device is
2. The arrangement information management unit according to claim 1, further comprising an arrangement information management unit that creates the arrangement information in which the calculated basket ID is associated with identification information of the plurality of devices included in the setting information. Node device. Input of change information including the content key of the data to be changed and the identification information of the device to be the new storage destination of the data via the input unit in order to change the device as the data storage destination When you accept
The arrangement information management unit uses the basket ID corresponding to the content key of the data to be changed calculated by the hash calculation unit as a key, and from the arrangement information, identifies the device identification information corresponding to the basket ID. Search and change the identification information of the searched device in the arrangement information to the identification information of the device as the new storage destination,
The node device according to claim 2, wherein the routing unit searches for the identification information of the device with reference to the changed arrangement information when searching for the identification information of the device.   When the data read request or write request is received from another node device, the read or write processing is executed on the data stored in the device included in the node device, and the execution result is obtained. 4. The node device according to claim 1, further comprising: a data transmission unit configured to transmit to a node device that is a transmission source of the data read request or write request. 5. When the placement information in the storage unit is changed, the placement information management unit transmits the changed placement information to another node device in the cluster storage system, and the change is made from the other node device. When the later arrangement information is received, the changed arrangement information is stored in the storage unit,
The routing unit searches the device identification information with reference to the arrangement information stored in the storage unit including the changed arrangement information when searching for the device identification information. The node device according to any one of claims 2 to 4.   A cluster type storage system comprising a plurality of node devices according to any one of claims 1 to 5. A monitoring device for monitoring a processing load in each of the node devices in the cluster type storage system;
In the monitoring device, when detecting a bias in processing load between the node devices,
For the basket ID corresponding to the identification information of the device accommodated in the node device having a relatively high processing load in the arrangement information, the node device uses the processing load to identify the device identification information corresponding to the basket ID. 7. The storage system according to claim 6, wherein the storage system is changed to identification information of a device accommodated in the node device having a relatively low value. In a cluster-type storage system that includes a plurality of node devices that read and write data, and stores the data redundantly in a plurality of devices accommodated in the node device, a hash value of a content key that is identification information of the data For each basket ID which is identification information obtained based on the arrangement information indicating the identification information of the plurality of devices storing the data, and identification of the node device which accommodates the device for each identification information of the device The node device including a storage unit that stores address information indicating information,
Receiving an input of the data read / write request or read request including a content key of the data to be read or written;
Calculating the basket ID corresponding to the content key based on the hash value of the content key;
Searching the device identification information corresponding to the basket ID from the arrangement information using the calculated basket ID as a key;
Identifying the identification information of the node device accommodating the device corresponding to the searched identification information with reference to the searched identification information of the device and the address information;
Transmitting a data read request or a write request corresponding to the content key to the device to the identified identification information node device;
And a step of receiving a result of the data read request or write request from the node device. The node device is
In order to change the device that is the storage destination of the data, when receiving the input of the change information including the content key of the data to be changed and the identification information of the device that is the new storage destination of the data,
Calculating a basket ID corresponding to the content key of the data to be changed;
Searching the device identification information corresponding to the basket ID from the arrangement information using the calculated basket ID as a key;
Executing the step of changing the searched device identification information in the arrangement information to the new storage destination device identification information;
9. The data read and write control method according to claim 8, wherein when the device identification information is searched, the device identification information is searched with reference to the changed arrangement information.   The program for making the said node apparatus which is a computer perform the data read-out and write-in control method of Claim 8 or Claim 9.

Images