JP5470148B2 - Node device and computer program - Google Patents

Description

  The present invention relates to file distribution management among nodes distributed on a network, and more particularly to a node device and a computer program of a P2P (Peer-to-Peer) type distributed file system.

A computer handles data stored by a user as a series of data chunks called files. This file is recorded in a storage device such as a hard disk drive. A portion that manages where data in a file is recorded in which location of the storage device is called a file system and operates as a part of an OS (operating system).
The distributed file system is a system that provides a user with file operations such as reading and writing of a file, apparently considering a plurality of storage servers connected via a computer network as one storage device.

  In the distributed file system, it is necessary to manage to which storage server the stored data is stored. Management methods can be broadly divided into a centralized type and a distributed type. Here, a distributed file management method, particularly a P2P (Peer-to-Peer) type file management method will be described. The P2P type file management method has a strong feature against failure because there is no server that centrally manages the file name and its storage address. The P2P type file management method is classified into a non-structural type and a structural type. A file management technique based on the structural type P2P using DHT (Distributed Hash Table) will be described below.

  In P2P using DHT, an overlay network is configured by a plurality of nodes. Here, the node is a generic term for servers that participate in the overlay network. FIG. 20 is a diagram illustrating the principle of a P2P overlay network using DHT. In the example shown in the figure, a circular overlay network is formed. A node participating in the overlay network generates a node ID by using a predetermined hash function such as SHA (Secure Hash Algorithm) -1, and identifies each node by this node ID in the overlay network. A circular overlay network (solid-line circle in FIG. 20) shows a state in which node IDs are arranged clockwise in order of increasing numerical values of IDs. Further, a file ID is assigned to a file held by each node. The file ID is generated using a hash function based on the file name, for example. In the overlay network, a node having a node ID close to the file ID value in the clockwise direction manages the file. Here, the file management is to store the file ID and the address of the node that holds the file.

  The node that owns the file registers the association between the file ID and its own node address in the node that should manage this file ID. The user of the file accesses the file by finding a node having a node ID close to the file ID from the file ID of the file to be used on the overlay network and knowing the address of the node that holds the file to be used. be able to.

  Here, the file ID is a key, the node address is a value corresponding to the key, and P2P by DHT is used as a database for acquiring a node address for storing a file from the file ID. This P2P by DHT can be used as a general-purpose key-value type database for retrieving a value from a key.

  In order to search a node that manages the file ID from the file ID (note that it is not a node that stores the file), each node needs to be able to search for a node participating in the overlay network. Therefore, each node holds a node table in which a correspondence between a node ID and a node address is registered, and searches for a node close to a desired file ID. When a management node having a desired file ID cannot be found from the held node table, a node close to the file ID is finally searched from participating nodes by transferring (routing) it to another node. . In P2P using DHT, various methods have been proposed according to a node table holding method and a search method (search request routing method) (for example, see Non-Patent Document 1). Here, OneHop, which is a method in which all nodes store a node table including all participating nodes, will be described (for example, see Non-Patent Document 2).

  In OneHop, all nodes hold a complete table of all participating nodes, so there is a drawback that the size of the node table of each node becomes large. However, since routing at the time of search is not required, high-speed search can be realized. In addition, with P2P, which manages the node table in a distributed manner, it is difficult to grasp all the nodes participating in the overlay network. However, since OneHop can easily grasp all the participating nodes, node state management is required. This is an effective method for various applications.

  In OneHop, all nodes hold a complete node table by notifying all nodes in the overlay network of node joins and leaves. Therefore, as shown in FIG. 21, the overlay network is equally divided into k pieces, and a slice leader is determined in each region. Further, the area divided into k pieces is equally divided into s pieces, and a unit leader is determined for each of the divided areas. The node that detects the joining or leaving of the node first notifies the slice leader of its own region (step S1). The slice leader that has received the notification notifies the slice leader in another area (step S2). The slice leader of each area that has received the notification notifies the unit leader in its own area. As described above, OneHop notifies all nodes in the overlay network of node joining and leaving information by a hierarchical structure.

  As described above, in the P2P type file management technology using DHT as well as OneHop, the node table is used for searching the file ID. Therefore, even when a node joins or leaves, each node holds it. It is necessary to keep the node table in the correct state. However, inconsistencies may occur in the node table due to the failure of node join / leave notifications to arrive correctly. In response, each node generally performs a process called stabilization. In Stabilization, each node periodically transmits a survival confirmation packet for detecting whether or not each node exists to each node recorded in the node table held by the node. As a result, the joining or leaving of the node is detected, and the node table held by each node is maintained.

  The overlay network is a virtual network determined by the node ID regardless of the physical network. Therefore, even if it is an adjacent node on the overlay network, it may be a node physically located at a long distance. As a result, many packets flow on the physical network. OneHop expansion method has been proposed as a method for reducing the amount of communication between nodes at long distances (see, for example, Patent Document 1 and Non-Patent Document 3).

  In the OneHop expansion method, the position (locality) of the node on the physical network is considered separately from the search table that is a node table for searching the node that manages the file ID from the file ID as described above. Thus, a management table which is a node table in which nodes are arranged is held. Then, by using the management table, notification of node joining / leaving and Stabilization communication are performed, thereby reducing the amount of communication between nodes at a long distance.

JP 2009-230686 A

Kazuyuki Shudo, “Technology for Scale Out”, Information Processing, pp.1080-1085, Vol.50, No.11, 2009 A. Gupta, B. et al., “One Hop Lookups for Peer-to-Peer Overlays,” 9th Workshop on Hot Topics in Operating Systems (HotOS-IX), 2003. Kaneko et al., “OneHop-P2P extension method considering node locality and management fairness”, RL-006, FIT2008, 2008

  In the distributed file system, in addition to managing the storage location of the file, the access function for users to connect to the distributed file system, the capacity to equalize the used capacity of each storage node that holds the file (storage) Various functions such as a load balancing function and a management function for grasping the status of nodes participating in the overlay network are required. In that case, it is necessary to add or delete functions depending on the situation, such as increasing or decreasing the number of nodes having access functions in order to distribute the access load according to the increase or decrease of users.

  On the other hand, as described above, in the P2P type file management technology using DHT, when a node joins or leaves, the range of file IDs managed by adjacent nodes of the joined or detached node changes. There is a problem that a node managing a desired file ID cannot be found while the node table of the node is old (inconsistent state). Also, when the file ID management node is changed, the file ID cannot be found until the node information (such as the server address of the file storage destination) is correctly registered in the new management node. There is.

Inconsistencies in the node table can be corrected automatically by stabilization, but it takes time until the correction is made, so that the search for the file (file ID) will be hindered while the correction is being made. .
In other words, in a distributed file system composed of storage nodes and functional nodes having individual functions other than file storage, file IDs are managed when functional nodes are frequently joined and removed from the overlay network as necessary. There is a problem that the period when the file ID cannot be searched increases even if the node is frequently changed and the storage node itself storing the file does not participate or leave.

  In order to cope with a search failure due to node join / leave, file ID registration not only to the management node but also to neighboring nodes, and the node table between neighboring nodes at the time of node join / leave Have been proposed, but in any case, a processing load and a communication load between the nodes are generated for the processing.

  The present invention has been made in view of such circumstances, and its purpose is to allow a node having a function other than file storage to search for a file without imposing a load on other nodes and the network. It is an object of the present invention to provide a distributed file system node device and a computer program that can participate and leave without affecting.

[1] One aspect of the present invention is a node device that participates via a network in a distributed file system that distributes and manages files, and a storage node device that distributes and stores files or a function that provides an additional function A management node table that associates node IDs and addresses of node devices that constitute the distributed file system, and a search node table that associates node IDs and addresses of the storage node devices. A storage unit for storing, and notification information of participation or withdrawal of each node device in the distributed file system, and the address selected based on the node ID stored in the management node table as the destination The notification information processing unit that sends the notification information and the storage node device participated When the notification information processing unit receives the notification information indicating that, the node ID and address of the storage node device are written to the management node table and the search node table based on the notification information, When the notification information processing unit receives notification information indicating that a functional node device has participated, the node ID and address of the functional node device are written only in the management node table based on the notification information, When the notification information processing unit receives notification information indicating that the storage node device has left, the management node table and the address corresponding to the node ID of the storage node device based on the notification information The notification information indicating that the functional node device has been removed is deleted from the search node table. When the information processing unit receives, the table processing unit for deleting the address corresponding to the node ID of the functional node device based on the notification information from the management node table, the specified file ID, And a table search unit that acquires an address of the storage node device that manages the file ID based on the node ID stored in the search node table.
According to the present invention, in a distributed file system composed of a plurality of node devices connected via a network, each node device holds a node table for file search and a node table for node device management, For storage node devices that store files, node information is registered in the search node table and management node table, and for node devices having additional functions, node information is registered only in the management node table.
As a result, even when the distributed file system is in operation, it is possible to add or delete nodes having individual functions other than file storage without affecting file management. Further, since there is no need to synchronize the search node table, no load is applied to the network or the node device.

[2] One aspect of the present invention is the above-described node device, further including a stabilization unit that confirms whether the node device at the address registered in the management node table has left, The table processing unit leaves the management node table and the search node table when the stabilization unit detects that the node device at the address registered in the management node table has left. And the notification information processing unit detects the address selected based on the node ID stored in the management node table as the destination, and the stabilization unit detects The notification information indicating that the node device has left is transmitted.
According to the present invention, a node device that has left the distributed file system is detected, the information of the node device that has left is deleted from the search node table and the management node table, and the node device is also transferred to other node devices. Notify device disconnection.
As a result, when the node device leaves the distributed file system, the search node table and the management node table can be updated without notifying other node devices of the departure.

[3] One aspect of the present invention is the above-described node device, which stores a file storage unit that stores a file, a file ID, and an address of the storage node device that stores the file in association with each other. A key table storage unit; a key table operation unit that reads out and outputs the address of the storage node device corresponding to the specified file ID from the key table storage unit; and an operation instruction that specifies a file name; And a file operation unit for performing an operation on the file in the file storage unit specified by a name.
According to this invention, the node device stores the file to be managed in the distributed file system and manages the address of the storage node device in which the file is stored. When the file ID is received, the address of the storage node apparatus storing the file with the file ID is returned. In addition, a file operation instructed by another node device is performed on the file stored in itself.
As a result, the node device functions as a storage node device, manages the location of the file to be managed in the distributed file system, and enables file operations on files stored in itself from other node devices.

[4] One aspect of the present invention is the node device described above, wherein the key table storage unit stores a status ID and status information in association with each other, and stores a status ID for specifying the status information of the own node device. A status information registration unit that sends the status ID and a request for registration of the status information to an address acquired by the table search unit corresponding to the status ID as a destination, the key table operation When receiving a status ID and status information registration request, the unit writes the status ID requested for registration and the status information in association with each other and writes them in the key table storage unit, and requests status information specifying the status ID. Received, the scan corresponding to the received status ID is received. The status information is read from the key table storage unit and output, and the table search unit acquires the address of the storage node device using the status ID instead of the file ID.
According to the present invention, the node device distributes and manages the status information of each node device, and outputs the managed status information when there is an inquiry.
As a result, the node device functioning as a storage node device functions as a capacity load distribution node device that distributes and manages the status information of each storage node device in the distributed file system and equalizes the used disk capacity of the storage node device. The status information can be notified to the node device and the node device functioning as a management node device that provides the status information of each node device to the user.

[5] One aspect of the present invention is the node device described above, which receives a file operation instruction specifying a file name, passes a file ID generated from the received file name to the table search unit, and The file ID is transmitted to the address acquired by the table search unit corresponding to the ID, the address of the storage node device storing the file corresponding to the transmitted file ID is received, and the received address And a file processing unit for transmitting a file operation instruction designating the file name as a destination.
According to the present invention, a file operation instruction is received from a client device, the location of the file operation target file is searched using the search node table, and the file operation instruction is output to the storage node device obtained as a result of the search To do.
As a result, the node device functions as an access node device and enables the client device to perform file operations on files managed by the distributed file system.

[6] One aspect of the present invention is the above-described node device, which passes a status ID for specifying status information of each of the storage node devices in the distributed file system to the table search unit, and corresponds to the status ID. The status ID is transmitted to the address acquired by the table search unit as a destination, the status information acquisition unit requesting status information, and the status information of the storage node device acquired in response to the request of the status information acquisition unit Based on the storage node device of the migration source and destination of the file, a capacity equalization calculation unit for determining the migration capacity, and the file from the storage node device of the file migration source determined by the capacity equalization calculation unit Based on the migration capacity to the destination storage node device And a file moving unit that moves the file, wherein the table search unit obtains the address of the storage node device using the status ID instead of the file ID.
According to the present invention, the status information of each storage node device in the distributed file system is acquired, and based on the acquired status information, the file capacity between the storage node devices is set so that the used disk capacity in each storage node device approaches evenly. Move.
As a result, the node device functioning as a capacity load distribution node device does not require any user operation, and can be adjusted so that the disk usage rates of the storage node devices in the distributed file system approach evenly. For example, if a storage node device that has not been registered with a file is added to the distributed file system, the storage node device with a high disk usage rate can be changed to a newly joined storage node device without the administrator explicitly instructing to move the file. You can move files.

[7] One aspect of the present invention is the node device described above, wherein a status ID that specifies status information of the node device is passed to the table search unit, and the table search unit acquires the status ID corresponding to the status ID. The table search unit further includes: a status information acquisition unit that transmits the status ID with the address as a destination and acquires status information; and an information presentation unit that outputs the status information acquired by the status information acquisition unit Uses the status ID instead of the file ID to obtain the address of the storage node device.
According to the present invention, the search node table is used to search for the storage node device storing the status information of each storage node device, and the status information read from the storage node device obtained as a result of this search is presented. To do.
Thereby, the node device functions as a management node device, and can present status information of the storage node device instructed by the user.

[8] One aspect of the present invention is the node device described above, and outputs join or leave of a node device when it is detected that a node ID has been added to or deleted from the management node table. An information presentation unit is further provided.
According to this invention, the node device outputs the participation of the node device when the node ID is registered in the management node table, and outputs the detachment of the node device when the node ID is deleted.
Accordingly, the node device functions as a management node device, and can present to the user that the node device has joined or left the distributed file system.

[9] One aspect of the present invention provides a storage node apparatus or an additional function for distributing and storing a computer used as a node apparatus that participates in a distributed file system that distributes and manages files via a network. A management node table that associates node IDs and addresses of node devices that constitute the distributed file system, and a search node table that associates node IDs and addresses of the storage node devices And a storage unit that stores the notification information of participation or withdrawal of each node device from the distributed file system, and the address selected based on the node ID stored in the management node table A notification information processing unit for transmitting the notification information as When the notification information processing unit receives notification information indicating that the node device has joined, the management node table and the search node are used to determine the node ID and address of the storage node device based on the notification information. And when the notification information processing unit receives notification information indicating that the functional node device has participated, the node ID and address of the functional node device are assigned to the management node based on the notification information. When the notification information processing unit receives notification information that is written only in the table and indicates that the storage node device is detached, the node ID of the storage node device and the corresponding address are managed based on the notification information. Deleted from the node table for search and the node table for search and A table processing unit that deletes the node ID of the functional node device and the corresponding address from the management node table based on the notification information when the notification information processing unit receives And a table search unit that acquires an address of the storage node device that manages the file ID based on the file ID and the node ID stored in the search node table. It is a computer program.

  According to the present invention, in a distributed file system composed of a plurality of node devices connected via a network, each node device has a search node table for file search, and a management node table for node device management. For node devices that store files and store files, register node information in the search node table and management node table, and for node devices that do not store files and have individual functions, in the management node table By registering only node information, node devices having individual functions can be added or deleted without affecting file management even when the distributed file system is in operation. In addition, since it is not necessary to synchronize the search node table, a high load is not applied to the network and the node device.

1 is a configuration diagram of a distributed file system according to an embodiment of the present invention. FIG. It is a block diagram which shows the internal structure of the node apparatus by the embodiment. It is a block diagram which shows the detailed structure of the node table management part by the embodiment. It is a figure explaining the example of the management node table and search node table by the embodiment. It is a figure which shows the structure of the request packet and response packet by the embodiment. It is a figure which shows the content of the specific data of the request packet and response packet by the embodiment. 5 is a flowchart of NOTIFY reception processing according to the embodiment. 3 is a block diagram illustrating a configuration of a unique function processing unit of the storage node device according to the embodiment. FIG. It is a block diagram which shows the example of the key table by the embodiment. It is a list of interface functions provided in the low-level interface unit according to the embodiment. It is a flowchart of operation | movement of the file information registration part by the embodiment. It is a flowchart of operation | movement of the status information registration part by the embodiment. It is a block diagram which shows the structure of the specific function process part of the access node apparatus by the embodiment. It is a list of interface functions provided in the high-level interface unit according to the embodiment. It is a block diagram which shows the structure of the intrinsic | native function process part of the capacity load distribution node apparatus by the embodiment. It is a flowchart of operation | movement of the status information acquisition part by the embodiment. It is a flowchart of operation | movement of the capacity equalization calculation part by the embodiment. It is a flowchart of operation | movement of the capacity equalization calculation part by the embodiment. It is a block diagram which shows the structure of the intrinsic | native function process part of the management node apparatus by the embodiment. It is a figure explaining the distribution management of the content in a P2P system. It is a figure explaining the notification method of the information in OneHop.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[1. Overall description]
FIG. 1 is a configuration diagram of a distributed file system according to an embodiment of the present invention. In the figure, the distributed file system is configured by connecting a storage node device 100, an access node device 200, a capacity load distribution node device 300, and a management node device 400 via a communication network 1. The communication network 1 is an IP (Internet Protocol) communication network composed of various network devices such as communication devices such as routers and communication lines such as cables. In the distributed file system shown in the figure, storage node devices 100a, 100b, and 100c are shown as a plurality of storage node devices 100. There may be a plurality of access node devices 200, capacity load distribution node devices 300, and management node devices 400.

  The storage node device 100 stores files to be managed in the distributed file system. Each of the access node device 200, the capacity load distribution node device 300, and the management node device 400 provides individual functions other than file storage. Hereinafter, the storage node device 100, the access node device 200, the capacity load distribution node device 300, and the management node device 400 are collectively referred to as “node devices”, and the access node device 200, the capacity load distribution node device 300, and The management node device 400 is collectively referred to as “function node device”. The functions provided by these function node devices are called additional functions. The additional function is an additional function other than the function of storing a file provided to the distributed file system.

  The node device of the present embodiment is a device that executes a node, and the node corresponds to a process in a computer. In some cases, one computer device includes only one node device, and when one computer device includes two or more node devices, a plurality of nodes having different functions are executed in one computer device. There is also.

  A user can operate a file in the distributed file system via the client device 2 connected to the access node device 200. One computer apparatus may include only the client apparatus 2, and one computer apparatus may include a node apparatus such as the access node apparatus 200 and the client apparatus 2.

  The distributed file system according to the present embodiment implements an overlay network with node devices connected via the communication network 1. Each node device participating in the overlay network stores, as node tables, a search node table used for file search and a management node table used for managing all node devices participating in the overlay network. The storage node device 100 is registered in both the search node table and the management node table, and the function node device is registered only in the management node table. The node device uses only the search node table and searches for the node device that manages the file ID as in P2P using DHT (Distributed Hash Table). As a result, even when the function node device joins or leaves, it is possible to prevent the file ID search from being delayed.

[2. Node device configuration]
FIG. 2 is a block diagram showing the configuration of the node device, and shows only the functional blocks related to the present invention. As shown in the figure, each node device of a storage node device 100, an access node device 200, a capacity load distribution node device 300, and a management node device 400 includes a node table management unit 10, a unique function processing unit 20, a network interface. The unit 30 and the storage unit 40 are provided. All the node devices have a common configuration except for the unique function processing unit 20.

  The storage unit 40 is realized by a hard disk device, a semiconductor memory, or the like, and stores a management node table 41 and a search node table 42. The management node table 41 shows correspondence between node IDs and node addresses of all node devices participating in the overlay network. The search node table 42 indicates a correspondence between the node ID and the node address of the storage node device 100.

  The network interface unit 30 provides a data transmission / reception interface via the communication network 1. The network interface unit 30 outputs data received from the communication network 1 to each unit, and transmits data via the communication network 1 according to instructions from each unit. The node table management unit 10 exchanges node device participation and withdrawal notification messages with other node devices, and updates the management node table 41 and the search node table 42 in the storage unit 40. The unique function processing unit 20 uses the management node table 41 and the search node table 42 in the storage unit 40 to store the storage node device 100, the access node device 200, the capacity load distribution node device 300, the management node device 400, and the like. The function corresponding to the type of the node device is executed.

[2.1 Internal configuration of node table management unit 10]
FIG. 3 is a block diagram showing the internal configuration of the node table management unit 10 of the node device shown in FIG. 2, and shows only functional blocks related to the present invention. As shown in the figure, the node table management unit 10 includes a notification processing unit 12 (notification information processing unit), a stabilization processing unit 13, and a table processing unit 14.
The notification processing unit 12 exchanges notification messages with the notification processing unit 12 of other nodes via the network interface unit 30 as notification information indicating participation and withdrawal of the node device. In order to correctly maintain the management node table 41 and the search node table 42 in the storage unit 40, the stabilization processing unit 13 performs existence confirmation with other node devices via the network interface unit 30. The table processing unit 14 receives a notification of joining or leaving the node device from the notification processing unit 12 or the stabilization processing unit 13 and rewrites the management node table 41 and the search node table 42 in the storage unit 40.

[2.2 Table structure]
FIG. 4 is a diagram illustrating a setting example of the management node table 41 and the search node table 42 stored in the storage unit 40. As shown in the figure, the management node table 41 includes a key value indicating the node IDs of all the node devices participating in the overlay network, and data indicating the node addresses of the node devices specified by the node IDs. Are registered in association with each other.

On the other hand, in the search node table 42, a key value indicating the node ID of the storage node device 100 participating in the overlay network corresponds to data indicating the node address of the storage node device 100 specified by the node ID. It is registered with it.
In the management node table 41 and the search node table 42, the node IDs are arranged in ascending order.

[2.3 Communication packets between node devices]
FIG. 5 is a diagram illustrating a structure of a communication packet (notification information) exchanged between the notification processing unit 12 and the stabilization processing unit 13 with another node device. As shown in the figure, the communication packet includes a command type or a response type, a transmission destination address, a transmission source address, and unique data. The communication packet of this embodiment includes a request packet and a response packet that is a response to the request packet. Request packet command types include “JOIN”, “PING”, and “NOTIFY”, and response packet response types include “OK” and “NG”.

FIG. 6 is a diagram showing the contents of the unique data of the communication packet shown in FIG. As shown in the figure, the content of the unique data varies depending on the type of command.
In the case of the JOIN command and the PING command, the node ID, the function type, and the node address are set to the unique data. In the case of the NOTIFY command, the host ID, the function type, the node address, the operation type, and the notification mode are set to the unique data. Is set. In the case of a response to the JOIN command, the management node table is set as unique data. No unique data is set in the response to the PING command and NOTIFY command.
The function type indicates, for example, “storage node”, “access node”, “capacity load distribution node”, “management node”, etc., and the operation type is “join (JOIN)” or “leave (LEAVE)”. Indicates one of the following.
Each command will be described below.

[2.3.1 JOIN command]
The JOIN command is used when the node device participates in the overlay network. In the JOIN command request packet (hereinafter referred to as “request packet (JOIN)”), the node ID, function type, and node address of the node device participating in the overlay network are stored as unique data. Further, the node ID is generated by a predetermined hash function. For example, the hash value of the node address (combination of the IP address and port number of the node device) is calculated using the hash function of SHA-1, and the calculated hash value is used as the node ID.
When the JOIN command is successful, the management node table 41 stored in the node device that has received the request packet (JOIN) has a response type “OK” response packet (hereinafter referred to as “response packet (OK)”) to the JOIN command. The data is stored in the unique data and sent back to the source node device of the request packet (JOIN).

[2.3.2 PING command]
The PING command is used to check the existence of the node device. In the PING command request packet (hereinafter referred to as “request packet (PING)”), the node ID, function type, and node address of the node device that is the source of the PING command are stored as unique data. The node device transmits the request packet (PING) to the destination node device and receives the response packet (OK), thereby confirming the existence of the node device subject to the existence confirmation. In addition, when the node device that has transmitted the request packet (PING) does not exist in the node table held by the node device that has received the request packet (PING), the node ID and the node of the source node device are stored in the node table. Add an address.

[2.3.2 NOTIFY command]
The NOTIFY command is used to notify all node devices on the overlay network that the node device has joined or left. The NOTIFY command request packet (hereinafter referred to as “request packet (NOTIFY)”) stores the node ID, function type and node address, operation type, and notification mode of the node device that has joined or withdrawn. The notification mode indicates a notification stage in OneHop and corresponds to each step described in FIG. That is, the notification mode “1” is a notification from the reporting source to the slice leader (Slice Leader) (step S1 in FIG. 21), and the notification mode “2” is a notification from the slice leader to another slice leader. (Step S2 in FIG. 21), the notification mode “3” represents notification from the slice leader to the unit leader (Step S3 in FIG. 21). The notification mode “4” represents notification from the unit leader to its subordinate nodes.

[2.4 Operations of Node Table Management Unit 10]
Next, the operation of each unit of the node table management unit 10 will be described.

[2.4.1 Operation of notification processing unit 12]
The notification processing unit 12 performs transmission and reception processing of the JOIN command and NOTIFY command.

[2.4.1.1 JOIN command transmission processing]
When a node device newly joins an overlay network, the notification processing unit 12 of the node device transmits a request packet (JOIN) to another node device that has already joined the overlay network. The method for acquiring the node address of the other node device to which the participation is notified is arbitrary. For example, the user may input it, and the search packet is transmitted on the communication network 1 and is already participating in the overlay network. You may acquire by receiving the response corresponding to a search packet from a node apparatus. Upon receiving the response packet (OK) of the transmitted request packet (JOIN), the notification processing unit 12 outputs the management node table stored in the unique data of the received response packet (OK) to the table processing unit 14. Instructs the node table to be updated.

[2.4.1.2 JOIN command reception processing]
When the notification processing unit 12 receives a request packet (JOIN) from another node device newly participating in the overlay network, the notification processing unit 12 outputs the unique data of the received request packet (JOIN) to the table processing unit 14 as node information, Instruct to update the table. Further, when the notification processing unit 12 reads the management node table 41 from the storage unit 40, the notification processing unit 12 stores the read management node table 41 in the unique data of the response packet (OK) of the JOIN command, and requests packet (JOIN) To the transmission source node device. Further, the notification processing unit 12 generates a request packet (NOTIFY) in which “participation (JOIN)” is set as the operation type and “1 (notification from the report source to the slice leader)” is set as the notification mode. Transmit to the leader node device.
The notification processing unit 12 of the node device that has received the request packet (NOTIFY) transmitted by the above process further transmits a NOTIFY packet according to the notification mode of the received packet.

[2.4.1.3 NOTIFY command reception processing]
FIG. 7 is a flowchart of the NOTIFY command reception process in the notification processing unit 12.
When receiving the request packet (NOTIFY) from another node device, the notification processing unit 12 outputs the unique data of the received request packet (NOTIFY) to the table processing unit 14 as node information, and instructs to update the node table ( Step S12-1). Details of the operation of the table processing unit 14 at this time will be described later.

  When the notification processing unit 12 determines that the mode of the received request packet (NOTIFY) is “1 (notification from the report source to the slice leader)” (step S12-2: YES), the request packet (NOTIFY) ) And the notification mode is rewritten to “2 (notification from slice reader to other slice reader)”, and a request packet (NOTIFY) in which the rewritten unique data is set is transmitted to all slice readers. (Step S12-3).

  When the notification processing unit 12 determines that the mode of the received request packet (NOTIFY) is “2” (step S12-2: NO, S12-4: YES), the notification processing unit 12 obtains unique data from the request packet (NOTIFY). Read and rewrite the notification mode to “3 (notification from the slice leader to the unit leader)”, and change the request packet (NOTIFY) in which the rewritten unique data is set to the unit (area) to which the node device belongs. Transmit to the reader (step S12-5).

  When the notification processing unit 12 determines that the mode of the received request packet (NOTIFY) is “3” (step S12-2: NO, S12-4: NO, step S12-6: YES), the request packet The unique data is read from (NOTIFY), the notification mode is rewritten to “4”, and the request packet (NOTIFY) in which the rewritten unique data is set is transmitted to all the node devices in the unit (area) to which the own node device belongs. (Step S12-7).

[2.4.1.4 Determination method of slice leader and unit leader]
The notification processing unit 12 divides the overlay network configured by the node devices registered in the management node table 41 into k regions, and selects one of the node devices in each divided region as a slice leader. To do. The notification processing unit 12 further divides the k divided area into s pieces, and sets one of the node devices in the area as a unit leader. The selection method of the slice leader and unit leader in each area is arbitrary. For example, there is a method of selecting a node device having the smallest node ID in each area as a slice leader or unit leader. For example, when dividing with k = 256 (2 ⁸ ) and s = 16 (2 ⁴ ), the upper 8 bits of the node ID are the slice number, and the 4 bits from the upper 9 bits to the 12th bit are the unit numbers. Since it can be divided, there is a method of determining a node having the minimum node ID in each area as a slice leader or a unit leader.

[2.4.2 Operation of stabilization processing unit 13]
The stabilization processing unit 13 maintains the overlay network structure by confirming the existence of the node device.

[2.4.2.1 PING command transmission processing]
The stabilization processing unit 13 refers to the management node table 41 stored in the storage unit 40 and periodically sends a PING command request packet (PING) to the node devices registered in the management node table 41 in order. To send.

  Specifically, when the survival confirmation process is started every predetermined time, the stabilization processing unit 13 sequentially reads out the node addresses of the node devices registered from the management node table 41, and the read node address A request packet (PING) is transmitted using as a transmission destination address. When the response to the transmitted request packet (PING) is received, the stabilization processing unit 13 transmits the request packet (PING) to the next node device to be checked for survival.

  On the other hand, when there is no response to the transmitted request packet (PING), the stabilization processing unit 13 sets “Leave” as the operation type on the assumption that the node device that is the transmission destination of the request packet (PING) has left. Then, a request packet (NOTIFY) in which “1 (notification source to slice reader)” is set as the notification mode is transmitted to the slice reader. As a result, the node devices are notified of the detachment of all the node devices constituting the overlay network. Thereafter, a request packet (PING) is transmitted to the next node device to be checked for survival.

[2.4.2.2 PING command reception processing]
When the stabilization processing unit 13 receives a request packet (PING) from another node device, the stabilization processing unit 13 outputs the information stored in the specific data in the received request packet (PING) to the table processing unit 14 as node information. The node table is instructed to be updated. Thereafter, the stabilization processing unit 13 returns a response packet (OK) to the transmission source of the request packet (PING).

[2.4.3 Operation of table processing unit 14]
The table processing unit 14 changes the management node table 41 and the search node table 42 stored in the storage unit 40.
This embodiment is characterized in that all node devices participating in the overlay network are registered in the management node table, and only the storage node device 100 is registered in the search node table. In order to achieve this, when the table processing unit 14 receives node information whose operation type is “join (JOIN)” from the notification processing unit 12 or the stabilization processing unit 13, the function type included in the node information Is a “storage node”, a combination of the node ID and the node address included in the node information is written in the management node table 41 and the search node table 42, and the function type is “access node”, “capacity” In the case of a functional node such as “load distribution node” or “management node”, the combination of the node ID and the node address included in the node information is written only in the management node table 41. At this time, the table processing unit 14 performs writing so that the node ID values are in ascending order in the table.

Further, when the table processing unit 14 receives node information whose operation type is “LEAVE” from the notification processing unit 12 or the stabilization processing unit 13, the table processing unit 14 sets the node ID and node address set in the node information. The set is deleted from the management node table 41 and the search node table 42. However, when the function type included in the node information indicates a function node, since the combination of the node ID and the node address to be deleted is not registered in the search node table 42, the deletion is actually performed. Absent.
When the table processing unit 14 receives the management node table from the notification processing unit 12, the table processing unit 14 writes the received management node table in the storage unit 40.

[2.5 Effects of the node device of this embodiment]
With the above operation, even if the function node device frequently participates or leaves the overlay network, the search node table 42 does not change, and thus the file ID search operation is not affected. Further, since the search node table 42 can be generated based on information for updating the management node table, the communication amount between the nodes is the same as that of the conventional OneHop.

  In the present embodiment, the description has been given by arranging the nodes in the management node table 41 in ascending order of the node IDs, but the present invention is not limited to this. For example, as described in Non-Patent Document 3, it is possible to reduce the amount of packets between physically separated nodes by using a local node ID including information on the physical position of the node. is there. Further, as described in Non-Patent Document 3, by generating a virtual node ID and registering it in the search node table 42 based on the performance of the storage node device, the processing load is proportional to the performance of the storage node device. It is possible.

[3. Unique function of each node device]
Next, operations of the unique functions of the storage node device 100, the access node device 200, the capacity load distribution node device 300, and the management node device 400 will be described. The operation of the unique function of each node device is determined by the operation of the unique function processing unit 20 shown in FIG.

[3.1 Unique Function of Storage Node Device 100]
[3.1.1 Configuration of Unique Function Processing Unit]
FIG. 8 is a diagram showing a configuration of the unique function processing unit 120 of the storage node device 100, and corresponds to the unique function processing unit 20 shown in FIG. The unique function processing unit 120 of the storage node device 100 shown in the figure includes a low-level interface unit 101, a key table operation unit 102, a file operation unit 103, a file information registration unit 104, a status information registration unit 105, a table search unit 106, A key table storage unit 107 and a file storage unit 108 are provided. The key table storage unit 107 and the file storage unit 108 are realized by a hard disk device, a semiconductor memory, or the like.

The key table storage unit 107 stores a key table indicating the association between keys and values. In the key table, key-value values in a range managed by the storage node device 100 in a key-value type database configured by P2P (Peer-to-Peer) based on DHT (Distributed Hash Table) are stored. Remember. In the distributed file system of this embodiment, since a key-value type database based on P2P is used for file ID management, a file ID for specifying a file as a key is set in the key table, and the file is stored as a value. The node address of the storage node device 100 is set. Various status information of each node device is also set in the key table. In this case, the storage node device 100 and a status ID that identifies the type of status information are set as a key, and status information is set as a value. The status information includes storage capacity such as a disk capacity that is the maximum storage capacity of the file storage device 108, a used disk capacity of the file storage device 108, a CPU load of the storage node device 100, and the number of files stored in the file storage unit 108. Various pieces of information related to the node device 100.
The file storage unit 108 stores a file to be operated in the distributed file system.

  The low level interface unit 101 is recorded in the key table stored in the key table storage unit 107 of the own storage node device 100 and the file storage unit 108 of the own storage node device 100 via the network interface unit 30. Provides an interface for manipulating existing files. The key table operation unit 102 executes processing corresponding to the key table operation interface provided by the low-level interface unit 101, and performs registration and search of keys with respect to the key table stored in the key table storage unit 107. The file operation unit 103 executes processing corresponding to the file operation interface provided by the low-level interface unit 101, and reads, writes, and deletes files recorded in the file storage unit 108, and obtains a list of files. To do.

  The file information registration unit 104 stores the file ID of the file stored in the file storage unit 108 and the node address of the own storage node device 100, and a key table held by the storage node device 100 that manages this file ID. Register with. The status information registration unit 105 periodically registers the status information of the own storage node device 100 in a key table held by the storage node device 100 that manages the status information. The table search unit 106 refers to the search node table 42 stored in the storage unit 40, specifies a node ID close to the specified ID in the clockwise direction, and associates it with the specified node ID. Read and output the node address.

FIG. 9 is a diagram illustrating a key table stored in the key table storage unit 107. As shown in the figure, the key table is composed of records in which a key and a value are associated with each other. Here, the key is a file ID that uniquely identifies a file, or a status ID that identifies a combination of a status information name and a node device.
As the file ID, for example, a hash value generated from a file name using a predetermined hash function can be used. When the key is a file ID, the node address of the storage node device 100 that stores the file specified by the file ID is set as a corresponding value.
On the other hand, for the status ID, for example, a hash value generated from data obtained by combining the status information name and the node address using a predetermined hash function can be used. The status information name is a name indicating the type of status information such as disk capacity, used disk capacity, CPU load, and the like. When the key is a status ID, status information corresponding to the type of the node device and status information specified by the status ID is set as a value.

  FIG. 10 shows an interface provided by the low-level interface unit 101. As shown in the figure, the low-level interface unit 101 provides get and put interfaces for key table operations, low_read, low_write, low_delete for file operations, and get_list interfaces for file list acquisition. The arguments “in” and “out” indicate the data communication direction, “in” indicates a value set by the caller of the interface, and “out” indicates a value returned to the caller.

get is an interface that reads a value (value) corresponding to the argument key from the key table stored in the key table storage unit 107 and outputs it to the caller as an argument value.
put is an interface for writing a set of an argument key and an argument value to the key table stored in the key table storage unit 107.

low_read is an interface that reads data from a file stored in the file storage unit 108. The argument fname indicates the file name of the data read target file. The argument offset indicates the head position from which data is read, and the number of bytes from the head of the file is set. The argument size indicates the byte size of the data to be read. The argument data indicates the read data.
low_write is an interface for writing data to a file stored in the file storage unit 108. The argument fname indicates the file name of the data write target file. The argument offset indicates the head position where data is written, and the number of bytes from the head of the file is set. The argument size indicates the byte size of data to be written, and the argument data indicates data to be written to the file.

low_delete is an interface for deleting a file stored in the file storage unit 108. The argument fname indicates the file name of the file to be deleted.
get_list is an interface that provides a list of files stored in the file storage unit 108. The argument flist indicates a list in which file names and sizes of files stored in the file storage unit 108 are associated with each other.

[3.1.2 Operation of Key Table Operation Unit 102]
The key table operation unit 102 performs processing for the get and put interfaces of the low level interface unit 101. When the put interface of the low-level interface unit 101 is called, the key table operation unit 102 stores a key pair stored in the key table storage unit 107 as a key (value) specified as an argument key and an argument value. Register in the table. On the other hand, when the get interface of the low-level interface unit 101 is called, the key table operation unit 102 retrieves the value (value) for the argument key from the key table stored in the key table storage unit 107 and reads it, and get Return to the node device that called the interface.

[3.1.3 Operation of file operation unit 103]
The file operation unit 103 performs processing for the low_read, low_write, low_delete, and get_list interfaces of the low-level interface unit 101. When the low_write interface of the low-level interface unit 101 is called, the file operation unit 103 adds the number of bytes indicated by the argument offset from the beginning of the file to the file with the file name of the argument fname stored in the file storage unit 108. Write the data of the number of bytes indicated by the argument size, starting from the advanced position. If the specified file does not exist, create a new file and write the data.

  On the other hand, when the low_read interface of the low-level interface unit 101 is called, the file operation unit 103 sets the number of bytes indicated by the argument offset from the beginning of the file whose argument fname is stored in the file storage unit 108. Using the proceeding position as the start position, the byte data indicated by the argument size is read and set as the argument data, and returned to the node device that called the low_read interface.

When the low_delete interface of the low-level interface unit 101 is called, the file operation unit 103 deletes the file whose argument fname is the file name from the file storage unit 108.
When the get_list interface of the low-level interface unit 101 is called, the file operation unit 103 reads the file names and sizes of all the files stored in the file storage unit 108, and displays a list of the read file names and sizes. Return to the calling node device.

[3.1.4 Operation of file information registration unit 104]
The file information registration unit 104 uses the file ID of the file stored in the file storage unit 108 as a key, and sets the node address of the storage node device 100 as a value (value). The set is registered in the key-value type database configured by P2P.

  FIG. 11 shows a flowchart of the operation of the file information registration unit 104. The flowchart shown in the figure is a registration operation for one file stored in the file storage unit 108, and the file information registration unit 104 repeatedly performs this operation for the number of files stored in the file storage unit 108. The file information registration unit 104 is activated periodically or when a new file is written in the file storage unit 108.

  First, the file information registration unit 104 generates a file ID from the file name of the file stored in the file storage unit 108 (step S104-1). As a file ID generation method, for example, there is a method of generating a file ID from a file name using a SHA-1 hash function. Next, the node device to which the file ID is to be registered is searched from the generated file ID (step S104-2). The table search unit 106 performs the search. The table search unit 106 searches the search node table 42 stored in the storage unit 40 for a node ID close to the designated file ID in the clockwise direction, and returns the node address. Next, the file information registration unit 104 requests the registration destination node device specified by the node address output as the search result from the table search unit 106 to execute the put interface (step S104-3). The generated file ID is set in the argument key of the put interface, and the node address of the own storage node device 100 is set in the argument value. As a result, a set of (file ID, node address) is registered in the storage node device 100 which is the management node of the generated file ID.

[3.1.5 Operation of Status Information Registration Unit 105]
The status information registration unit 105 periodically registers the status information of the storage node device 100 in a key-value type database based on P2P. The status information registration unit 105 registers these pieces of information using a predetermined key so that various status information of the storage node device 100 can be acquired from other node devices.

FIG. 12 shows a flowchart of the operation of the status information registration unit 105.
When the status information registration unit 105 is activated, a status information name indicating the type of status information is input together with the status information. Status information such as disk capacity, disk usage capacity, and CPU load can be acquired by, for example, a resource management function unit included in the storage node device 100 in which the node device is mounted. The status information registration unit 105 first generates a key (S105-1). The status ID is a hash value generated by a hash function such as SHA-1 from a combination of a predetermined status information name indicating the type of status information and the node address of the own storage node device 100. For example, “DISKCAPA_node address” for the disk capacity and “CPU_node address” for the CPU load are input as the hash function, and the hash value is calculated.

  Since the status information registration unit 105 registers the status information in the P2P key-value database using the status ID created by the hash function, the table search unit 106 searches for a node device that manages the status ID ( Step S105-2). That is, the table search unit 106 searches for a node ID close to the designated status ID in the clockwise direction from the search node table 42 stored in the storage unit 40, and returns the node address. The status information registration unit 105 requests the registration destination storage node device 100 of the node address output as the search result from the table search unit 106 to execute the put interface, and registers the status information (step S105-3). The generated status ID is set in the argument key of the put interface, and status information is set in the argument value.

[3.1.6 Effects of Storage Node Device 100]
As described above, a P2P distributed file system using DHT can be constructed by the storage node device 100 having the unique function processing unit 120.

[3.2 Unique Function of Access Node Device 200]
[3.2.1 Configuration of Unique Function Processing Unit]
FIG. 13 is a diagram illustrating a configuration of the unique function processing unit 220 of the access node device 200, and corresponds to the unique function processing unit 20 illustrated in FIG.
The unique function processing unit 220 of the access node apparatus 200 includes a high-level interface unit 201, a file processing unit 202, and a table search unit 203.

  The high-level interface unit 201 provides an interface for manipulating files stored in the storage node device 100 in the distributed file system. The client apparatus 2 can operate files in the distributed file system by using this interface via the communication network 1. The file processing unit 202 operates the operation target file according to the interface received by the high level interface unit 201. The table search unit 203 has the same function as the table search unit 106 of the storage node device 100, and refers to the search node table 42 stored in the storage unit 40, and rotates the specified file ID clockwise. A near node ID is specified, and a node address associated with the specified node ID is read and output.

FIG. 14 shows an interface provided by the high-level interface unit 201. As shown in the figure, the high-level interface unit 201 provides hi_read and hi_write interfaces.
hi_read is an interface for reading data from a file having a specified file name. The argument fname indicates the file name of the data read target file. The argument offset indicates the head position from which data is read, and the number of bytes from the head of the file is set. The argument size indicates the byte size of the data to be read. The argument data indicates the read data.
hi_write is an interface for writing data to a file having a specified file name. The argument fname indicates the file name of the data write target file. The argument offset indicates the start position for writing data, and the number of bytes from the start of the file is set. The argument size indicates the byte size of data to be written, and the argument data indicates data to be written to the file.

[3.2.2 Operation of Unique Function Processing Unit 220]
In the access node device 200, when the client device 2 requests execution of the hi_read interface or the hi_write interface of the high level interface unit 201, the file processing unit 202 performs actual processing on the interface requested by the high level interface unit 201. I do.

  When the hi_read interface of the high-level interface unit 201 is called by the client device 2, the file processing unit 202 generates a file ID from the file name fname specified as the argument fname. For example, a predetermined hash function such as SHA-1 is used for generating the file ID. Next, the file processing unit 202 outputs a file ID to the table search unit 203 and requests a node address. The table search unit 203 outputs the received file ID to the table search unit 203, and searches for the node address of the storage node device 100 that manages the file ID. When the table search unit 203 searches the search node table 42 stored in the storage unit 40 by the received file ID and acquires the node address, the table search unit 203 outputs the node address to the file processing unit 202.

  The file processing unit 202 requests the low-level interface unit 101 of the storage node apparatus 100 that manages the file ID to execute the get interface, using the node address acquired by the table search unit 203 as a result of the search. A file ID is set in the argument key of the get interface. The storage node device 100 requested to execute the get interface reads a value corresponding to the argument key from the key table stored in the key table storage unit 107 and returns it as an argument value. The argument value indicates the node address of the storage node device 100 that stores the file specified by the file ID.

  Using the node address indicated by the argument value of the get interface, the file processing unit 202 requests the low-level interface unit 101 of the storage node device 100 storing the operation target file to execute the low_read interface, and Is read out. The arguments fname, offset, and size of the hi_read interface are set in the arguments fname, offset, and size of the low_read interface. The storage node device 100 that has received the execution of the low_read interface returns an argument data of the execution result. The file processing unit 202 of the access node apparatus 200 outputs the argument data returned from the storage node apparatus 100 to the client apparatus 2 that is the execution request source of the hi_read interface.

  Also, when the hi_write interface of the high level interface unit 201 is called, the same processing as when the hi_read interface is called is performed, and the low level interface unit 101 of the storage node device 100 storing the file is processed. , Instructs execution of the low_write interface and performs file write processing. The arguments fname, offset, size, and data of the hi_write interface are set in the arguments fname, offset, size, and data of the low_write interface. The storage node device 100 requested to execute the low_write interface executes the low_write interface.

[3.2.3 Effects of access node apparatus 200]
The access node device 200 having the unique function processing unit 220 described above allows the client device 2 to operate files stored in the distributed file system of the present embodiment. The access node device 200 can provide access to the distributed file system to the plurality of client devices 2 via the high-level interface unit 201.

  By using the access node device 200 according to the present embodiment, when the number of client devices 2 increases, it becomes possible to add the access node device 200 to the distributed file system and distribute the access load to the distributed file system. Further, when the number of client devices 2 decreases, the access node device 200 can be detached and a useless access node can be deleted. In the present embodiment, the addition and removal of the access node device 200 does not delay the file management of the distributed file system. Therefore, there is no problem in joining and leaving the access node device 200 at any timing in the distributed file system in operation. Further, the access node device 200 and the client device 2 can be operated on the same computer. In this case, high-speed file operation without access from other client devices 2 is possible.

  In the present embodiment, the high-level interface unit 201 provided by the access node apparatus 200 has been described only for basic interfaces such as the hi_read and hi_write interfaces. However, the file processing unit 202 includes Kaneko et al., “Structural P2P”. As described in “Distributed Directory Management Method in Distributed File System Using”, L-033, FIT2009, 2009 (Reference Document 1), the distributed file system of this embodiment can be obtained by using the distributed management method using directory files. It is also possible to provide a file system having a directory structure. For example, i-node files, directory files, and data files with unique file names in the distributed file system are distributed and stored in the file storage device 108 of the storage node device 100. The directory file includes management information regarding the correspondence between the subordinate directory name and the file name of the directory file, and the correspondence between the subordinate user use file name and the data file name. The i-node file includes a file name of a directory file or a data file. The data file is the file body. The access node device 200 receives a file operation request based on the directory name and the user use file name from the client device 2. When the access node device 200 reads the i-node file and the directory file from the storage node device 100 in order from the i-node file corresponding to the root directory, the access node device 200 acquires the data file name corresponding to the directory name and the user-used file name. The file operation requested by the client apparatus 2 is requested to the storage node apparatus 100 that holds the data file body having the data file name. Note that the reading of the i-node file and the directory file and the file operation of the data file are performed in the same manner as in the above-described embodiment based on the file ID generated from the file names of the i-node file, the directory file, and the data file.

[3.3 Unique Function of Capacity Load Balancing Node Device 300]
[3.3.1 Configuration of unique function processor]
The capacity load distribution node device 300 is a functional node device that makes the usage rate of the file storage unit 108 included in the storage node device 100 in the distributed file system evenly close.
FIG. 15 is a diagram illustrating a configuration of the unique function processing unit 320 of the capacity load balancing node apparatus 300, and corresponds to the unique function processing unit 20 illustrated in FIG. The unique function processing unit 320 of the capacity load distribution node device 300 includes a status information acquisition unit 301, a capacity equalization calculation unit 302, a file movement unit 303, and a table search unit 304.

  The status information acquisition unit 301 acquires status information of each storage node device 100 in the distributed file system. The capacity equalization calculation unit 302 is based on the status information acquired by the status information acquisition unit 301, and in order to equalize the usage rate of the file storage unit 108 of each storage node device 100, the storage node that is the target of file migration The apparatus 100 and the file capacity to be moved are determined. The file migration unit 303 instructs file migration according to the file migration source and destination storage node devices 100 determined by the capacity equalization calculation unit 302 and the file capacity to be migrated. The table search unit 304 sequentially reads the node address of the storage node device 100 from the search node table 42 stored in the storage unit 40 and outputs the node address to the status information acquisition unit 301. The table search unit 304 has a function similar to that of the table search unit 106 of the storage node device 100. The ID specified from the status information acquisition unit 301 or the file migration unit 303 with reference to the search node table 42 is provided. A node ID close to clockwise is identified, and a node address associated with the identified node ID is read and output.

[3.3.2 Operation of status information acquisition unit 301]
FIG. 16 shows a flowchart of the operation of the status information acquisition unit 301. The capacity load distribution node device 300 activates the usage rate equalization processing periodically at a predetermined time or at a time instructed by the user. Thereby, the process of FIG. 16 is started.

  In the figure, when the status information acquisition unit 301 sets n to 0 as an initial value, the storage node device 100 registered in the search node table 42 by the table search unit 304 (hereinafter referred to as “storage node”). The node address of “device (n)” is acquired (step S301-1). Next, the status information acquisition unit 301 generates a status ID for each of the disk capacity and the used disk capacity from the status information name and the node address acquired by the table search unit 304 (step S301-2). The method of generating the status ID is the same as the method described in the operation of the status information registration unit 105 of the storage node device 100. For example, in the case of disk capacity, the hash value of “DISKCAPA_node address” is used as the hash function. Generate by. Next, the generated status ID is output to the table search unit 304, and the node address of the node device managing the status ID is requested. The table search unit 304 searches the search node table 42 stored in the storage unit 40 by the status ID to acquire a node address, and the status information acquisition unit 301 obtains the node address of the search result from the table search unit 304. Obtain (step S301-3).

  The status information acquisition unit 301 uses the node address acquired in step S301-3 to request the low-level interface unit 101 of the storage node device 100 to execute the get interface, and the disk capacity of the storage node device (n). And status information of each used disk capacity is acquired (step S301-4). The status ID generated in step S301-2 is set in the argument key of the get interface. If the storage node device (n) is not the last node device registered in the search node table 42 (step S301-5: NO), 1 is added to n (step S301-6), and then step S301. Repeat the operation from -1. In step S301-5, if the storage node device (n) is the last node device in the search node table 42 (step S301-5: YES), the operation of the status information acquisition unit 301 is terminated, and the capacity Execution of the operation of the equalization calculation unit 302 is started (step S301-7).

[3.3.3 Operations of Capacity Equalization Calculation Unit 302]
The capacity equalization calculation unit 302 determines whether the usage rate of all the storage node devices 100 is constant from the maximum storage capacity of the storage nodes acquired by the status information acquisition unit 301 and the storage capacity in use. Calculate the file transfer capacity at.

[3.3.3.1 Capacity equalization algorithm]
First, the capacity equalization calculation method according to the present embodiment will be described.
The number of nodes N is the number of storage node devices 100 (N is an integer of 1 or more), the disk capacity C _i is the disk capacity of the storage node device (i), and the used disk capacity M _i is the used disk capacity of the storage node device (i). Then (i is an integer of 0 or more and (N−1) or less), the average storage usage rate r _avg in all the storage node devices 100 is calculated by the following equation (1).

When M ^′ _i is the used disk capacity of the storage node device (i) after capacity equalization, the following expression (2) is established.

Assuming that the conversion matrix from the current disk usage capacity M _i of the storage node device (i) to the disk usage capacity M ^′ _i after capacity equalization of the storage node device (i) is A, the following equation (3) ) Holds.

The component a _ij (i, j is an integer not _smaller than 0 and not larger than (N−1)) of the transformation matrix A is _calculated from the storage node device (j) to the storage node device with respect to the currently used disk capacity of the storage node device (j). (I) shows the ratio of the disk capacity to be moved. Therefore, capacity equalization is to obtain the transformation matrix A under the condition indicated by the following equation (4).

  This is because the i column component indicates the proportion of how the used disk capacity of the storage node device (i) is distributed, and the sum of the column components obtained by adding up the distribution proportions is 1. .

  Since the transformation matrix A is not fixed to one, in this embodiment, the transformation matrix A is obtained by the following (procedure 1) to (procedure 3).

(Procedure 1) Determination of diagonal component:
The diagonal component a _ii of the transformation matrix A and other related components are determined according to:

  (A) is a case where the current used disk capacity is equal to or greater than the target used capacity, and it is not necessary to receive a file from another storage node device 100. Further, (b) is a case where the current used disk capacity is smaller than the target used capacity, and it is not necessary to move the file to another storage node apparatus 100, and the sum of the column components is 1. It is.

  Here, description will be given by taking five storage node devices (0) to (4) in the state shown in Table 1 below as an example.

As shown in Formula (5), the average usage rate r _{avg in} this example is as follows.

  From this, the following equation (6) is obtained as M ′ = AM. That is, in (procedure 1), (a) is applied to the storage node device (i) of the node number i = 0, 1, 3 and the storage node device (i) of the node number i = 2, 4 is applied. (B) applies. "-" Is an unset component.

(Procedure 2) Determination of unset components:
An unset component to be processed is defined as a _IJ . At this time, the capacity of the storage node 100 (I) after moving the file can be expressed as (7) below.

Then, the component a _IJ is calculated under the following conditions so that the capacity of the storage node device (I) after the file movement shown in (7) is maximized.

Condition 1 indicates that the total ratio of disk capacities moved from the storage node device (J) is 1 or less. The condition 2 is the amount of used disk space storage node device after file migration (I) have shown to be a less used disk space M _^'I after the capacity equalization.
Furthermore, by determining the component a _IJ as described above, other components are also determined when the following conditions are satisfied.

  The above is obtained sequentially for all unset components.

When an unset component is calculated using the example of the above equation (6), the result is as follows.
First, the processing target is an unset component a ₂₀ in the 0th column. From condition 1 and condition 2, component a ₂₀ = 0.511 is calculated, and component a ₄₀ = 0 is obtained by (c). Therefore, Expression (6) becomes the following Expression (8).

Next, the processing target is an unset component a ₂₁ in the first column. From condition 1 and condition 2, component a ₂₁ = 0.45 is calculated, and component a ₄₁ = 0 is obtained by (c). Therefore, Expression (8) becomes the following Expression (9).

Next, the processing target is an unset component a ₂₃ in the third column. From condition 1 and condition 2, component a ₂₃ = 0.095 is calculated, and the following equation (10) is obtained.

Next, the processing target is an unset component a ₄₃ in the third column. From condition 1 and condition 2, component a ₄₃ = 0.355 is calculated, and the following expression (11) is obtained.

(Procedure 3) Determination of movement source, movement destination and movement capacity:
A component other than the diagonal component of the transformation matrix A obtained by the above (procedure 2) and a component other than 0 is a portion representing the movement of the disk capacity. That is, for the component a _ij , the source node device is the storage node device (j), the destination node device is the storage node device (i), and the migration capacity is a _ij × M _j .
In the case of the above example, the movement amount between the storage node devices 100 can be obtained as shown in Table 2.

[3.3.3.2 Processing flow]
17 and 18 show a flowchart of the capacity equalization calculation unit 302.
In FIG. 17, first, the capacity equalization calculation unit 302 initializes all components a _ij (i, j are integers of 0 or more and (N−1) or less) of the N-row N-column conversion matrix A (step S302). -1) Using the disk capacity C _i and the used disk capacity M _i of all the N storage node devices (i) acquired by the status information acquisition unit 301, the average usage rate of all the storage node devices 100 according to equation (2) r _avg is calculated (step S302-2). Subsequently, the capacity equalization calculation unit 302 uses the average usage rate r _avg calculated in step S302-2 and the used disk capacity M _{i of the} storage node device 100 (i), and calculates all the storage nodes according to Expression (3). For the device (i), the used disk capacity M ^′ _i of the storage node device (i) after capacity equalization is calculated (step S302-3).

Subsequently, when the capacity equalization calculation unit 302 initializes by substituting 0 for i (step S302-4), the current used disk capacity M _i of the storage node device (i) is the used disk after capacity equalization. It is determined whether the capacity is equal to or greater than the capacity M ^′ _i (step S302-5). If YES is determined in the step S302-5, the process (a) of the (procedure 1) of the capacity equalization algorithm is performed. That is, the capacity equalization calculation unit 302 sets (M ^′ _i / M _i ) in the diagonal component a _ii (step S302-6), and sets the storage node device (i) from all other storage node devices (j). ) Is set to 0 for the component a _ij corresponding to the movement to () (step S302-7). On the other hand, if NO is determined in step S302-5, the capacity equalization calculation unit 302 sets 1 to the diagonal component a _ii (step S302-8), and all other storages from the storage node device (i). The component a _ji corresponding to the movement to the node device (j) is set to 0 (step S302-9).

After the process of step S302-7 or step S302-9, the capacity equalization calculation unit 302 adds 1 to the current value of i (step S302-10), and is i greater than (N-1)? Is determined (step S302-11). When it is determined NO in step S302-11, the capacity equalization calculation unit 302 repeats the process from step S302-5, and when it is determined YES in step S302-11, the process of FIG. 18 is performed.

In FIG. 18, the capacity equalization calculation unit 302 performs the process (procedure 2) of the capacity equalization algorithm. That is, the capacity equalization calculation unit 302 selects the component a _IJ for which the current value is not set (step S302-12). The component a _IJ can be arbitrarily selected from the components for which the current value is not set. Here, the component having the smallest I and J is selected.

When the capacity equalization calculation unit 302 calculates the value of the component a _IJ so as to satisfy (condition 1) and (condition 2) in (procedure 2) of the capacity equalization algorithm (step S302-13), the capacity equalization It is determined whether the condition (c) or (d) of (Procedure 2) of the algorithm is satisfied.

When the capacity equalization calculation unit 302 determines that the total ratio of the disk capacity moved from the storage node device (J) is 1, and satisfies the condition of (c) (step S302-14: YES), the value is still There is set to 0 component _{a kJ} not set (step S302-15). If the capacity equalization calculation unit 302 determines NO in step S302-14, the used disk capacity of the storage node device (I) after the file movement is equal to the used disk capacity M ^′ _i after the capacity equalization. Is determined (step S302-16). When the capacity equalization calculation unit 302 determines that step S302-16 is YES and satisfies the condition of (d), the capacity equalization calculation unit 302 sets 0 to the component a _Ik that has not yet been set (step S302-17). .

After the processing of step S302-15 and step S302-17, or when NO is determined in step S302-16, the capacity equalization calculation unit 302 determines whether there is a component for which no value is set. (Step S302-18). When it is determined that there is a component for which no value is set (step S302-18: YES), the processing from step S302-12 is repeated.

When it is determined that there is no component for which no value is set (step S302-18: NO), the capacity equalization calculation unit 302 executes (procedure 3) of the capacity equalization algorithm. That is, the capacity equalization calculation unit 302 uses the non-zero component a _ij (i ≠ j, and i and j are integers of 0 or more and (N−1) or less) from the storage node device (i). The migration capacity a _ij × M _j to the node device (j) is calculated (step S302-19), and the file migration unit 303 is instructed to migrate the file (step S302-20).

[3.3.3 Operation of file moving unit 303]
The file moving unit 303 performs a file moving process between the storage node devices 100 obtained by the capacity equalization calculating unit 302. However, if all the file movement processing between the storage node devices 100 obtained in the capacity equalization calculation unit 302 is performed, not only the load is applied to the communication network 1, but also the load increases for many storage node devices 100. End up. Also, access to the file may be restricted during the move. In view of this, the file migration process is performed only for migration between storage node devices 100 that meet a predetermined condition, for example, the migration capacity is greater than or equal to a predetermined value, or the disk usage rate of the storage node device 100 that is the file migration source is greater than or equal to a predetermined value. Is targeted. In the following description, it is assumed that only file movement between storage node devices 100 having the largest migration capacity is performed. Further, the load on the communication network 1 is suppressed by performing the file movement at a time when the traffic is low.

First, the file migration unit 303 determines that the file migration from the storage node device (i) to the storage node device (j) having the largest migration capacity is performed based on the migration capacity calculated by the capacity equalization calculation unit 302. To do.
The file moving unit 303 outputs the node ID of the storage node device (i) to the table searching unit 304 and requests a node address. The table searching unit 304 uses the search node table 42 stored in the storage unit 40. The node address corresponding to the node ID is read out. The file moving unit 303 requests execution of get_list using the node address read by the table searching unit 304 as a destination.

  The storage node device 100 reads a list of file names and sizes of files stored in the file storage unit 108, sets the argument list, and returns the list to the capacity load balancing node device 300. The file migration unit 303 of the capacity load distribution node device 300 selects a migration target file based on the list acquired from the storage node device 100 so that the total size is close to the migration capacity. For example, select files in order from the largest file, and when the size movement capacity of the selected file is exceeded for the first time, or before the time when it exceeds, the selected file is determined to be moved, or all combinations of files are selected. It is possible to generate and select a combination of files whose total file size is the closest to the transfer capacity, and determine a transfer target.

  The file moving unit 303 outputs execution of low_read of each file to be moved to the storage node device (i) in order to read the file determined as the target to be moved. The file name of the file to be moved is set in the argument fname of low_read, 0 is set in the argument offset, and the size of the file to be moved is set in the argument size. The storage node device (i) reads the file body of the specified file name from the file storage unit 108, sets it as an argument data, and returns it to the capacity load distribution node device 300.

  When the file migration unit 303 reads the file to be migrated from the storage node device (i), the file migration unit 303 outputs execution of low_write of each file to be migrated to the storage node device (j) in order to write each read file to the migration destination. To do. The file name of the file to be moved is set in the argument fname of low_read, 0 is set in the argument offset, the size of the file to be moved is set in the argument size, and the file body is set in the argument data. The storage node device (j) writes the file to be moved to the file storage unit 108. When the storage node device (j) writes the file, it performs the processing shown in FIG.

  Subsequently, the file moving unit 303 outputs execution of low_delete for each file to be moved stored in the storage node device (i). The file name of the file to be moved is set in the argument fname of low_delete. The storage node device (i) deletes the file having the file name specified by the argument fname from the file storage unit 108.

[3.3.4 Effect of Capacity Load Balancing Node Device 300]
As described above, the capacity load distribution node device 300 can add a function for equalizing the used capacity of the storage node device in the distributed file system according to the present embodiment. Then, by regularly transferring files only between the storage node devices 100 having a large migration capacity, such as once a day, the used capacity can be gradually approached evenly so as not to place a load on the entire distributed file system. it can.
In addition, since the addition and deletion of the distributed file system of the capacity load distribution node device 300 does not delay the search processing of the distributed file system, the addition and deletion of the capacity load distribution function is performed at any time during the operation of the distributed file system. The number of capacity load distribution node devices 300 is not limited.

  In the present embodiment, the description has been given of the case where the capacities of all the storage node devices 100 in the distributed file system are equalized. However, the present invention is not limited to this. For example, when the management node table 41 is created using the local node ID considering the physical position of the storage node device described in Non-Patent Document 3, the processing of the capacity load balancing node device 300 is performed locally. It is also possible to perform capacity load distribution by limiting to the storage node device 100 in a range limited by the ID.

In addition, the file moving unit 303 of the capacity load balancing node device 300 acquires a list of file names and update dates / times of files stored in the file storage unit 108 from each storage node device 100, and updates / dates before a predetermined date / time. It is also possible to move these files to the storage node device 100 dedicated to archiving in accordance with the file movement procedure between the storage node devices 100 described above. As a result, it is possible to move the old file only to the storage node device 100 dedicated to the archive.
Furthermore, the processing of the capacity load distribution function can be operated only at a specific time such as midnight when there are few accesses of the distributed file system.

[3.4 Unique Function of Management Node Device 400]
[3.4.1 Configuration of unique function processor]
The management node device 400 is a functional node for displaying the status of each node device in the distributed file system.

  FIG. 19 is a diagram illustrating a configuration of the unique function processing unit 420 of the management node device 400, and corresponds to the unique function processing unit 20 illustrated in FIG. The unique function processing unit 420 of the management node device 400 includes a status selection unit 401, a status information acquisition unit 402, an information presentation unit 403, a node notification unit 404, and a table search unit 405.

  The status selection unit 401 accepts a status information confirmation target. The status information acquisition unit 402 acquires status information via the network interface unit 30 from the storage node device 100 that manages the status information of the node device that is the confirmation target received by the status selection unit 401, and the information presentation unit 403 is notified. The node notification unit 404 monitors the management node table 41 of the storage unit 40 and notifies the information presentation unit 403 of information on the node device that has joined or left when the node device joins or leaves. The information presentation unit 403 presents the status information notified from the status information acquisition unit 402 or the participation or withdrawal of the node device notified from the node notification unit 404 to the user by displaying on the display. The table search unit 405 has a function similar to that of the table search unit 106 of the storage node device 100, and the node ID close to the specified status ID in the clockwise direction from the search node table 42 stored in the storage unit 40. And return its node address.

[3.4.2 Operation of management node apparatus 400]
First, the status selection unit 401 receives, from a user who uses the management node device 400, the selection of a node device that is a confirmation target of status information and the input of the type of status information to be confirmed, and the status information acquisition unit 402 receives these information. Notify information.

  The status information acquisition unit 402 generates a status ID as a key from the node address of the node device notified from the status selection unit 401 and the type of status information by the same method as the status information registration unit 105. The status information acquisition unit 402 outputs the status ID to the table search unit 405, and acquires the node address of the storage node device 100 that manages the status ID. The status information acquisition unit 402 requests the storage node device 100 of the acquired node address to execute a get interface, and acquires status information. The generated status ID is set in the argument key of the get interface. The status information acquisition unit 402 notifies the information presentation unit 403 of the status information acquired from the storage node device 100, and the information presentation unit 403 presents the status information notified from the status information acquisition unit 402. The information presentation unit 403 may also present the node device designated as the confirmation target and the type of status information. The information presenting unit 403 can use any presenting method such as, for example, displaying data by text or displaying graphs for each node.

On the other hand, the node notification unit 404 monitors the management node table 41 of the storage unit 40, and when the node ID and node address are added to the management node table 41 by the table processing unit 14, the node ID and node address are The information presentation unit 403 is notified of the node information that has been added as a trigger. The information presenting unit 403 presents the node information notified from the node notifying unit 404 and the participation.
Similarly, when the node ID and the node address are deleted from the management node table 41 by the table processing unit 14, the node notification unit 404 notifies the information presentation unit 403 of the node information that triggered the deletion. The information presenting unit 403 presents the node information notified from the node notifying unit 404 and the fact that it is a detachment.

[3.4.3 Effects of management node device 400]
As described above, the management node device 400 can provide the user with the participation / leaving status of all the node devices participating in the distributed file system of this embodiment and the status information of each node device. it can. Since the participation and withdrawal of the management node device 400 according to the present embodiment does not delay the file management of the distributed file system, the management node device 400 can join or leave at any time during the operation of the distributed file system. There is no limit to the number.
In the present embodiment, the presentation of node information has been described for all node devices participating in the distributed file system, but the present invention is not limited to this. For example, as proposed in Non-Patent Document 3, a local node ID including information on the physical position of a node device is used in the management node table of this embodiment, so that the management node device 400 is specified. It is also possible to display the status information limited to the local ID, that is, the node device within a specific physical range.

[4. Effects of this embodiment]
As described above, according to the present embodiment, in a P2P type distributed file system using DHT, each node device participating in the overlay network has a search node table that is a file search node table, and node device management. The node ID and node address of the storage node device that holds the management node table that stores the file and stores the file are registered in the search node table and the management node table. The function node device that provides the function is registered only in the management node table. Therefore, the search node table is not changed even if the function node device joins or leaves, so that the search for the file ID is not delayed and the search for the operation target file is not affected. Further, it is not necessary to synchronize the search node table.

In this way, in the P2P type distributed file system, the addition and removal of functional node devices having various functions other than file storage can be performed without affecting the file search, so the distributed file system is stopped. It is possible to extend the functions without having to do so.
In addition, the access load can be distributed by joining or leaving the access node device according to the present embodiment in response to the increase or decrease of users of the distributed file system.
In addition, the capacity load distribution node device according to the present embodiment can equalize the usage amount of the disk required for the storage node device to store the file, or adjust the usage amount intended by the administrator.
In addition, the management node device according to the present embodiment can grasp the state of the node device participating in the overlay network and present it to the user.

[5. Others]
The storage node device 100, the access node device 200, the capacity load distribution node device 300, the management node device 400, and the device that implements the client device 2 described above have a computer system therein. The operation processes of the storage node device 100, the access node device 200, the capacity load distribution node device 300, the management node device 400, and the client device 2 are stored in a computer-readable recording medium in the form of a program. The above processing is performed by the computer system reading and executing the program. The computer system here includes a CPU, various memories, an OS, and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage unit such as a hard disk built in the computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case is also used to hold a program for a certain period of time. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

DESCRIPTION OF SYMBOLS 1 ... Communication network 2 ... Client apparatus 10 ... Node table management part 12 ... Notification processing part (notification information processing part)
13 ... Stabilization processing unit 14 ... Table processing units 20, 120, 220, 320, 420 ... Unique function processing unit 30 ... Network interface unit 40 ... Storage unit 41 ... Management node table 42 ... Search node table 100 ... Storage node Device 101 ... Low-level interface unit 102 ... Key table operation unit 103 ... File operation unit 104 ... File information registration unit 105 ... Status information registration unit 106 ... Table search unit 107 ... Key table storage unit 108 ... File storage unit 200 ... Access node Device 201 ... High-level interface unit 202 ... File processing unit 203 ... Table search unit 300 ... Capacity load distribution node device 301 ... Status information acquisition unit 302 ... Capacity equalization calculation unit 303 ... File transfer unit 304 ... Table search unit 400 ... Management node Location 401 ... status selection section 402 ... status information acquisition unit 403 ... information presentation unit 404 ... node notification unit 405 ... table search unit

Claims (9)

A node device that participates via a network in a distributed file system that distributes and manages files,
A storage node device that distributes and stores files, or a function node device that provides additional functions, a management node table that associates node IDs and addresses of node devices that constitute the distributed file system, and the storage node A storage unit that stores a search node table that associates node IDs and addresses of devices;
Notification of receiving notification information of participation or withdrawal of each node device from the distributed file system, and transmitting the notification information to the address selected based on the node ID stored in the management node table An information processing unit;
When the notification information processing unit receives notification information indicating that a storage node device has participated, the management node table and the search node are used to determine the node ID and address of the storage node device based on the notification information. Write to node table,
When the notification information processing unit receives notification information indicating that a functional node device has participated, the node ID and address of the functional node device are written only in the management node table based on the notification information,
When the notification information processing unit receives notification information indicating that the storage node device has left, the management node table and the address corresponding to the node ID of the storage node device based on the notification information Delete from the search node table,
If the notification information processing unit receives notification information indicating that the functional node device has left, the node ID of the functional node device and the corresponding address are deleted from the management node table based on the notification information A table processing unit to
A table search unit that acquires an address of the storage node device that manages the file ID based on the specified file ID and the node ID stored in the search node table;
A node device comprising: A stabilizing unit for confirming whether or not the node device at the address registered in the management node table is disconnected;
The table processing unit leaves the management node table and the search node table when the stabilization unit detects that the node device at the address registered in the management node table has left. Delete the node ID and address of the node device
The notification information processing unit, with the address selected based on the node ID stored in the management node table as a destination, notification information indicating that the node device detected by the stabilization unit has detached Send,
The node device according to claim 1, wherein: A file storage unit for storing files;
A key table storage unit that stores a file ID and an address of the storage node device storing the file in association with each other;
A key table operation unit that reads out and outputs the address of the storage node device corresponding to the specified file ID from the key table storage unit;
A file operation unit that receives an operation instruction specifying a file name and performs an operation on the file in the file storage unit specified by the file name;
The node device according to claim 1, further comprising: The key table storage unit stores a status ID and status information in association with each other,
A status ID for specifying the status information of the own node device is passed to the table search unit, and the registration request for the status ID and the status information is transmitted to the address acquired by the table search unit corresponding to the status ID. It further includes a status information registration unit,
When receiving a registration request for status ID and status information, the key table operation unit writes the status ID requested for registration and the status information in association with each other in the key table storage unit, and designates the status ID. When a request for status information is received, the status information corresponding to the received status ID is read from the key table storage unit and output,
The table search unit acquires the address of the storage node device using the status ID instead of the file ID.
The node device according to claim 3. A file operation instruction designating a file name is received, a file ID generated from the received file name is passed to the table search unit, and the file obtained by the table search unit corresponding to the file ID is used as the destination. A file processing unit that transmits an ID, receives an address of a storage node device storing a file corresponding to the transmitted file ID, and transmits a file operation instruction specifying the file name with the received address as a destination Further comprising
The node device according to claim 1, wherein the node device is a node device. A status ID for identifying status information of each storage node device in the distributed file system is passed to the table search unit, and the status ID is transmitted to the address acquired by the table search unit corresponding to the status ID. A status information acquisition unit that requests status information;
Based on the status information of the storage node device acquired in response to the request of the status information acquisition unit, a capacity equalization calculation unit that determines the storage node device of the migration source and migration destination of the file, and the migration capacity;
A file mover for moving a file based on the move capacity from the storage node device of the file move source determined by the capacity equalization calculation unit to the storage node device of the file move destination,
The table search unit acquires the address of the storage node device using the status ID instead of the file ID.
The node device according to claim 1, wherein the node device is a node device. A status for acquiring status information by passing a status ID for identifying status information of the node device to the table search unit, transmitting the status ID to the address acquired by the table search unit corresponding to the status ID An information acquisition unit;
An information presentation unit that outputs the status information acquired by the status information acquisition unit,
The table search unit acquires the address of the storage node device using the status ID instead of the file ID.
The node device according to claim 1, wherein the node device is a node device. An information presenting unit that outputs participation or withdrawal of a node device when it is detected that a node ID has been added or deleted from the management node table;
The node device according to claim 1, wherein the node device is a node device. A computer used as a node device that participates via a network in a distributed file system that distributes and manages files.
A storage node device that distributes and stores files, or a function node device that provides additional functions, a management node table that associates node IDs and addresses of node devices that constitute the distributed file system, and the storage node A storage unit that stores a search node table that associates node IDs and addresses of devices;
Notification of receiving notification information of participation or withdrawal of each node device from the distributed file system, and transmitting the notification information to the address selected based on the node ID stored in the management node table An information processing unit;
When the notification information processing unit receives notification information indicating that a storage node device has participated, the management node table and the search node are used to determine the node ID and address of the storage node device based on the notification information. Write to node table,
When the notification information processing unit receives notification information indicating that a functional node device has participated, the node ID and address of the functional node device are written only in the management node table based on the notification information,
When the notification information processing unit receives notification information indicating that the storage node device has left, the management node table and the address corresponding to the node ID of the storage node device based on the notification information Delete from the search node table,
If the notification information processing unit receives notification information indicating that the functional node device has left, the node ID of the functional node device and the corresponding address are deleted from the management node table based on the notification information A table processing unit to
A table search unit that acquires an address of the storage node device that manages the file ID based on the specified file ID and the node ID stored in the search node table;
A computer program that functions as a computer program.

Images