JP2011186616A - Distributed file system, and server, method, and program for remote file processing - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce a network load accompanied by file access via a network. <P>SOLUTION: When referring to a remote file, a first server transmits a reference request of the remote file to a plurality of servers including a second server. The second server transmits the remote file requested by the reference request to the first server. On receiving the remote file from the second server, the first server registers the second server into a first file information storage as storage target server information by relating to the identification information of the received remote file. When referring to the remote file again, the first server refers to the first file information storage, specifies that the storage destination server information related to the remote file identification information to is for the second server, and transmits a reference request including the remote file identification information to the specified second server. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a distributed file system, a remote file processing server, a method, and a program, and more particularly to a distributed file system, a remote file processing server, a method and a program for processing a remote file stored in an external server via a network. Regarding the program.

  In recent years, NAS (Network Attached Storage) has been used to share files among a plurality of servers via a network. FIG. 10 is a block diagram showing a network file system 900 which is an example of a computer system using NAS. The network file system 900 includes a NAS 91, a server 92, a server 93, and servers 94a,... 94n, which are connected via the network 1.

  The NAS 91 is a file server dedicated machine that can be directly connected to the network 1 to allow another server to use an HDD (Hard Disk Drive) 910. Here, it is assumed that the HDD 910 stores a remote file 911 which is an arbitrary file. The servers 92, 93, 94a,... 94n are general-purpose computer systems, which are so-called server devices. As for the server 92, the server 93, the server 94a,... 94n, an arbitrary OS (Operating System) (not shown) operates. In particular, in the server 92, the application 920 operates. Further, the server 93 runs an application 930. For example, the application 920 performs processing for generating an arbitrary file, and stores the generated file as a remote file 911 in the HDD 910 via the network 1. Then, the application 930 acquires the remote file 911 from the HDD 910 via the network 1 and performs a predetermined process using the acquired remote file 911. Thereby, the file can be shared by the NAS 91 between the server 92 and the server 93. Similarly, files can be shared with the servers 92 and 93 by the NAS 91 for the servers 94a,.

  Patent Document 1 discloses a technique related to a method of caching and sharing a file in a computer system in a computer network including one or a plurality of network computers. The method according to Patent Literature 1 is a peer list including a list of candidate peer servers selected from one or a plurality of network computers, and constructing a peer list that does not include a predetermined number of candidate peer servers or more. Querying candidate peer servers in the peer list for operations, files or portions of files, and from peer servers in the peer list that the peer server has a file or portion of a file available for download An operation of receiving a message and an operation of downloading a file or a part of a file from a peer server.

Special table 2009-506455 gazette

  For file sharing using the NAS described above, for example, when the same file is repeatedly referred to, communication with the NAS 91 is required each time, and the amount of data flowing on the network 1 increases, so the load on the network 1 increases. There is a problem.

  Also, in Patent Document 1, when there is a possibility that a file to be referred to is updated, it is necessary to query a candidate peer server in the peer list every time, and there is a problem that the load on the network also increases. .

  The present invention has been made to solve such problems, and provides a distributed file system, a remote file processing server, a method, and a program for reducing the load on the network associated with file access via the network. The purpose is to do.

  A distributed file system according to a first aspect of the present invention includes a first file information storage unit that stores storage destination server information that identifies a server in which a file is stored, and is a remote file stored in an external server A first server that processes the remote file, and a second server that stores the remote file, and the first server includes a plurality of servers including the second server when referring to the remote file. The remote server transmits a remote file reference request to the first server, and the second server transmits the remote file requested by the reference request to the first server, and the first server transmits the second file to the second server. When the remote file is received from the server, the second server is the storage server information, and the received remote file identification information and When storing and registering in the first file information storage unit and referring to the remote file again, the storage destination server associated with the remote file identification information by referring to the first file information storage unit The information is specified as the second server, and a reference request including the identification information of the remote file is transmitted to the specified second server.

  A remote file processing server according to a second aspect of the present invention includes a file information storage unit that stores identification information of a file and storage destination server information that identifies the server in which the file is stored, When referring to a remote file stored in a server, the determination unit determines whether or not the remote file identification information exists in the file information storage unit, and the determination unit determines whether the remote file identification information is When it is determined that it exists, the file information storage unit is referred to, and the remote file reference request is transmitted to the server specified from the storage destination server information associated with the remote file identification information, When it is determined that the identification information of the remote file does not exist, the remote server is notified to a plurality of servers. Comprising a file request portion for transmitting a reference request for the file, upon receiving the remote file as a response from the server that received the reference request, the file processing unit that performs processing based on the remote file the received and.

  A remote file processing method according to a third aspect of the present invention includes a first file information storage unit that stores storage destination server information that identifies a server in which a file is stored, and is stored in an external server. A first server that processes a file; and a second server that stores the remote file. When the first server refers to the remote file, the first server includes a plurality of second servers. The remote file reference request is transmitted to the server, the second server transmits the remote file requested by the reference request to the first server, and the first server transmits the remote file When the remote file is received from the second server, the second server is used as the storage destination server information and the received remote file is received. When the remote file is registered in the first file information storage unit in association with the file identification information, and the remote file is referenced again, the first file information storage unit is referred to and associated with the remote file identification information. The storage destination server information is specified as the second server, and a reference request including the remote file identification information is transmitted to the specified second server.

  When a remote file processing program according to the fourth aspect of the present invention refers to a remote file stored in an external server, the storage destination server that identifies the file identification information and the server in which the file is stored A determination process for determining whether or not the remote file identification information exists in the file information storage unit that stores the information in association with each other, and when the determination process determines that the remote file identification information exists , Referring to the file information storage unit, sending a remote file reference request to the server specified from the storage destination server information associated with the remote file identification information, and the remote file identification information If it is determined that the file does not exist, the remote file must be referenced to multiple servers. A remote file process including: a file request process for transmitting a file process; and a file process for performing a process based on the received remote file when the remote file is received as a response from the server that has received the reference request. Let it run.

  According to the present invention, it is possible to provide a distributed file system, a remote file processing server, a method, and a program for reducing the load on the network associated with file access via the network.

It is a block diagram which shows the structure of the distributed file system concerning Embodiment 1 of this invention. It is a sequence diagram which shows the flow of the remote file acquisition process concerning Embodiment 1 of this invention. It is a block diagram which shows the structure of the network file system concerning Embodiment 2 of this invention. It is a figure which shows the example of the file information concerning Embodiment 2 of this invention. It is a block diagram which shows the structure of the server concerning Embodiment 2 of this invention. It is a block diagram which shows the hardware constitutions of the server concerning Embodiment 2 of this invention. It is a flowchart figure which shows the flow of the file information registration process concerning Embodiment 2 of this invention. It is a flowchart figure which shows the flow of the file request | requirement process concerning Embodiment 2 of this invention. It is a flowchart figure which shows the flow of the file transmission process concerning Embodiment 2 of this invention. It is a figure for demonstrating generation | occurrence | production of a subject.

  Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings. In the drawings, the same elements are denoted by the same reference numerals, and redundant description will be omitted as necessary for the sake of clarity.

<Embodiment 1 of the Invention>
FIG. 1 is a block diagram showing a configuration of a distributed file system 100 according to the first exemplary embodiment of the present invention. The distributed file system 100 includes a server 2, a server 3, and servers 4 a,... 4 n, which are connected via the network 1.

  The server 2 is a general-purpose computer system and includes at least a control unit 21 that processes a remote file stored in an external server and a file information storage unit 22. The file information storage unit 22 is a storage device that stores file identification information 221 and storage destination server information 222. The file information storage unit 22 is preferably a non-volatile storage device such as an HDD or a flash memory. The file identification information 221 is information for identifying a file. The file identification information 221 is, for example, a file name or a file ID. The storage destination server information 222 is information for specifying the server in which the file is stored. The storage destination server information 222 is information indicating the destination of the server where the file is stored. The storage destination server information 222 may be, for example, a host name, an IP (Internet Protocol) address, or the like. The storage destination server information 222 may be information indicating the local host when indicating the server 2 itself. The storage destination server information 222 may be information for uniquely identifying a server connected to the network 1 at least in the distributed file system 100.

  The server 3 is a general-purpose computer system and includes at least an HDD 31. The HDD 31 is a storage device that stores the remote file 311. The remote file 311 is an arbitrary file, for example, a file generated in the server 3. The HDD 31 is an example of a storage device provided in the server 3. The server 3 includes a control unit (not shown) that acquires the remote file 311 from the HDD 31 and returns it to the request source in response to a reference request for the remote file 311 from an external server.

  The servers 4a,..., 4n are computer systems equivalent to the server 2 or the server 3. The network 1 is a communication network such as an intranet or a LAN.

  Next, an overview of remote file acquisition processing in the distributed file system 100 will be described. First, when referring to the remote file 311, the server 2 transmits a reference request for the remote file 311 to a plurality of servers including the server 3. Next, the server 3 transmits the remote file 311 requested by the reference request to the server 2. When the server 2 receives the remote file 311 from the server 3, the server 2 sets the server 3 as storage destination server information 222 and registers it in the file information storage unit 22 in association with the file identification information 221 of the received remote file 311. Thereafter, when referring to the remote file 311 again, the server 2 refers to the file information storage unit 22 and specifies that the storage destination server information 222 associated with the file identification information 221 of the remote file 311 is the server 3. Then, a reference request including the file identification information 221 of the remote file 311 is transmitted to the specified server 3.

  FIG. 2 is a sequence diagram showing the flow of the remote file acquisition process according to the first embodiment of the present invention. First, the control unit 21 of the server 2 refers to the remote file 311 (S11). Then, the control unit 21 transmits a reference request for the remote file 311 to a plurality of servers connected to the network 1. That is, the control unit 21 transmits a reference request (S12, S12a,... S12n) of the remote file 311 to the servers 3, 4a,.

  Next, the server 3 receives the reference request in step S <b> 12 and confirms that the remote file 311 requested by the reference request is stored in the HDD 31. Then, the server 3 acquires the remote file 311 from the HDD 31, and returns the remote file 311 to the server 2 that is the request source (S13). At this time, each of the servers 4a,..., 4n does not store the remote file 311 requested by the reference request received in steps S12a,. May be.

  Then, the control unit 21 receives the remote file 311 from the server 3. As a result, the control unit 21 can perform processing on the remote file 311. At this time, the control unit 21 sets the server 3 as the storage destination server information 222 and registers it in the file information storage unit 22 in association with the remote file 311 and the file identification information 221 (S14).

  Thereafter, the remote unit 311 is referred to in the control unit 21 of the server 2 (S15). Then, the control unit 21 refers to the storage destination server information 222 associated with the file identification information 221 of the remote file 311 with respect to the file information storage unit 22 (S16). The control unit 21 receives a response of the storage destination server information 222 from the file information storage unit 22 (S17). At this time, the control unit 21 specifies that the storage destination server is the server 3 based on the received storage destination server information 222. Then, the control unit 21 transmits a reference request including the file identification information 221 of the remote file 311 to the specified server 3 (S18).

  As described above, in Embodiment 1 of the present invention, when the server 2 refers to a remote file that is not stored by itself, it is initially unknown where the remote file is stored. A reference request is transmitted to a plurality of servers connected to the network 1. As a result, when a remote file exists in the distributed file system 100, it is possible to reliably acquire the requested remote file. Then, when processing the received remote file, the server 2 also registers the remote file identification information and the storage destination server information in association with each other in the file information storage unit 22. As a result, when referring to the same remote file again, the server 2 refers to the file information storage unit 22 and specifies that the storage destination server is the server 3, thereby determining the destination of the reference request. One place can be provided. Therefore, even when multiple file accesses occur to the same remote file, direct communication with the storage destination server can be made after the second time. Therefore, compared to the case where each file is centrally managed as in the case of NAS, it is possible to reduce the load on the network associated with file access via the network.

<Embodiment 2 of the Invention>
The network file system 200 according to the second embodiment of the invention is one example of the distributed file system 100 according to the first embodiment of the invention. FIG. 3 is a block diagram showing the configuration of the network file system 200 according to the second embodiment of the present invention. The network file system 200 includes a server 2a, a server 3a, and servers 5a,... 5n, which are connected via the network 1, respectively.

  The server 2a is a general-purpose computer system, and includes an application 210, a monitoring daemon 220, an HDD 230, and a file information storage unit 240. The application 210 is software that generates, references, and updates an arbitrary file. The HDD 230 is a storage device that stores the generated file 231 generated by the application 210. The HDD 230 is an example of a storage device provided in the server 2a.

  The file information storage unit 240 is a storage device that stores the file path name 241, the hash value 242, the storage destination server information 243, and the update date / time information 244 in association with each other. In other words, the file information storage unit 240 is a database that manages information about the location of a file as cache information. The file information storage unit 240 is preferably a nonvolatile storage device such as an HDD or a flash memory. The file path name 241 is information indicating the file name and the directory path where the file is stored. That is, the file path name 241 is information uniquely determined within a predetermined file system. The hash value 242 is a hash value calculated from the file path name 241 by a predetermined hash function. The storage destination server information 243 is information equivalent to the storage destination server information 222 described above. The update date / time information 244 is information indicating the date and time when the associated file was last updated at least. The update date / time information 244 may include an update date / time other than the last, for example, a creation date / time. FIG. 4 is a diagram illustrating an example of data stored in the file information storage unit 240 according to the second embodiment of the present invention.

  The monitoring daemon 220 monitors the HDD 230, and when the file access from the application 210 to the HDD 230 occurs and the file access requests a file stored in the external server, It is software that makes reference requests. In addition, the monitoring daemon 220 manages information on the location of the file acquired by the reference request by the file information storage unit 240.

  The server 3a is a computer system equivalent to the server 2a, and includes an application 310, a monitoring daemon 320, an HDD 330, a file information storage unit 340, a generated file 331, a file path name 341, a hash value 342, storage destination server information 343, and an update date and time. The information 344 is an equivalent component corresponding to each component in the server 2a. Therefore, detailed description is omitted.

  The servers 5a,..., 5n are computer systems equivalent to the servers 2a and 3a. The network 1 is a communication network such as an intranet or a LAN.

  Next, an overview of remote file acquisition processing in the network file system 200 will be described. Note that in the network file system 200, description of processing equivalent to that of the distributed file system 100 is omitted.

  When the application 210 of the server 2a refers to the generated file 331 stored in the server 3a, that is, the remote file, the monitoring daemon 220 determines whether or not the remote file identification information exists in the file information storage unit 240. To do. If it is determined that the file exists, the monitoring daemon 220 transmits a reference request including the remote file identification information to the server 3a specified from the storage destination server information 243 associated with the remote file identification information. . If the monitoring daemon 220 determines that the file does not exist, the monitoring daemon 220 transmits a reference request including the identification information of the remote file to a plurality of servers including the server 3a. Accordingly, when the location of the generated file 331 is managed by the file information storage unit 240, the reference request is directly transmitted to the storage destination server, so that the load on the network can be reduced. Even if the location of the generated file 331 is not managed by the file information storage unit 240, it is possible to reliably refer to the remote file by sending a reference request to a plurality of servers connected to the network 1. can do.

  Further, when the application 310 of the server 3 a generates the generation file 331, the monitoring daemon 320 sets itself as storage destination server information 343 and registers it in the file information storage unit 340 in association with the identification information of the generation file 331. Then, when referring to the generated file 331, the monitoring daemon 220 of the server 2a transmits a reference request including the identification information of the generated file 331. At this time, when the monitoring daemon 320 of the server 3a receives a reference request from the server 2a, the monitoring daemon 320 refers to the file information storage unit 340, and identifies itself as the storage destination server information 343 in the identification information of the associated file. Then, it is determined whether or not the identification information included in the reference request exists. If it is determined that the identification information exists, the monitoring daemon 320 transmits the generated file 331 to the server 2a. As a result, it is possible to cope with file access including file identification information from an external server. In particular, since the file information storage unit 240 and the file information storage unit 340 have the same information management method regarding the location of the file, the file can be efficiently processed.

  Further, when the application 310 of the server 3a generates the generation file 331, the monitoring daemon 320 calculates the hash value 342 from the file name of the generated generation file 331, sets itself as the storage destination server information 343, and the hash value 342 Are registered in the file information storage unit 340. Then, the monitoring daemon 220 of the server 2a calculates a hash value 242 from the file name of the generated file 331, and transmits a reference request including the hash value 242. When the monitoring daemon 320 receives a reference request from the server 2 a, the monitoring daemon 320 refers to the file information storage unit 340 and includes itself in the hash value 342 associated with the storage destination server information 343 in the reference request. It is determined whether or not the hash value hash value 242 exists. If the monitoring daemon 320 determines that the hash value 242 exists, the monitoring daemon 320 transmits the generated file 331 to the server 2a. Thereby, the identification information of the file which is data flowing in the network 1 can be made into small data called a hash value, and the load on the network can be further reduced. In addition, a file uniquely determined by a hash value can be specified.

  Further, the servers 2a and 3a may generate a hash value including the name of the directory where the remote file is stored. As a result, the file can be specified more accurately based on the file identification information.

  When the application 310 of the server 3a updates the generated file 331, the update date / time information 344 of the application 310 is further registered in the file information storage unit 340 in association with the hash value 342 of the generated file 331. When the monitoring daemon 320 of the server 3a receives the reference request from the server 2a and determines that the identification information exists, the update date / time information 344 associated with the identification information included in the reference request from the file information storage unit 340. And the generated file 331 including the acquired update date / time information 344 is transmitted to the server 2a. Thereafter, when receiving the generated file 331 from the server 3a, the monitoring daemon 220 of the server 2a further registers the update date / time information 344 in the file information storage unit 240 in association with the identification information of the received generated file 331. Thereby, the update date / time information of the remote file can be managed each time the server 2a is referred to.

  Further, when referring to the generated file 331, the monitoring daemon 220 of the server 2a may transmit a reference request including the identification information of the generated file 331 to a plurality of servers including the server 3a by broadcasting. In that case, the monitoring daemon 320 of the server 3a receives a reference request from the server 2a by broadcast, and performs a response after performing the above-described processing. Thereby, even if the location of the remote file is unknown, the reference request can be efficiently transmitted to the server that stores the remote file.

  FIG. 5 is a block diagram showing a configuration of the server 2a according to the second embodiment of the present invention. In addition, since the structure of the server 3a is equivalent to the server 2a, illustration and description are abbreviate | omitted. The server 2a includes a file information storage unit 240, a hash calculation unit 251, a determination unit 252, a file request unit 253, a file processing unit 254, a file information registration unit 255, and a file transmission unit 256.

  Here, the server 2 a needs to include at least a file information storage unit 240, a determination unit 252, a file request unit 253, and a file processing unit 254. In this case, the file information storage unit 240 stores the file path name 241 that is the file identification information and the storage destination server information 243 in association with each other. Here, first, when referring to a remote file stored in an external server, the determination unit 252 determines whether or not identification information of the remote file exists in the file information storage unit 240. Next, when the determination unit 252 determines that the remote file identification information exists, the file request unit 253 refers to the file information storage unit 240 and stores the storage destination server associated with the remote file identification information. The remote file reference request is transmitted to the server specified from the information 243. Further, when it is determined that the remote file identification information does not exist, the file request unit 253 transmits the remote file reference request to a plurality of servers. When the file processing unit 254 receives a remote file as a response from the server that has received the reference request, the file processing unit 254 performs processing based on the received remote file. Thereby, when the location of the remote file is managed by the file information storage unit 240, the reference request is directly transmitted to the storage destination server, so that the load on the network can be reduced. Further, even when the location of the remote file is not managed by the file information storage unit 240, the remote file is reliably referred to by transmitting a reference request to a plurality of servers connected to the network 1. be able to.

  Furthermore, the server 2a preferably includes a file information registration unit 255. In this case, when receiving a remote file, the determination unit 252 determines whether or not the received remote file identification information exists in the file information storage unit 240. Then, when the determination unit 252 determines that the received remote file identification information does not exist, the file information registration unit 255 sets the transmission source server of the remote file as the storage server information 243, and receives the received remote file. The file information is registered in the file information storage unit 240 in association with the file identification information. As a result, when referring to the remote file received this time again, it is possible to specify the storage destination server, and it is not necessary to send a reference request to multiple servers, so an unnecessary reference request is sent. Can reduce the load on the network.

  Further, when the file processing unit 256 generates a file, the file information registration unit 255 may register itself in the file information storage unit 240 in association with the identification information of the generated file. Thereby, the file 231 generated by the application 210 can also be managed together with the remote file stored in the external server.

  The server 2a preferably includes a file transmission unit 256. In this case, when the determination unit 252 receives a reference request from an external server, the determination unit 252 refers to the file information storage unit 240 and includes the reference information in the identification information of the file associated with the storage server information 243 as a reference. It is determined whether or not the remote file identification information requested by the request exists. Then, when the determination unit 252 determines that the requested remote file identification information exists, the file transmission unit 256 transmits a file corresponding to the requested remote file identification information to an external server. Thereby, it is possible to efficiently process a file access from an application of an external server.

  The server 2a preferably further includes a hash value calculation unit 251 that calculates a hash value from the file name of the file. In this case, the file information storage unit 240 stores the hash value 242 calculated by the hash value calculation unit 251 as file identification information. Thereby, the file to be referred to can be specified with smaller data.

  Furthermore, it is desirable that the hash value calculation unit 251 calculates the hash value 242 including the directory name in which the file is stored. As a result, the file can be specified more accurately based on the file identification information.

  The file information storage unit 240 may store the file update date / time information 244 in association with the identification information of the file. As a result, the state of the file can be managed accurately.

  In addition, the file request unit 253 may transmit the remote file reference request to a plurality of servers by broadcasting. Thereby, even if the location of the remote file is unknown, the reference request can be efficiently transmitted to the server that stores the remote file.

  FIG. 6 is a block diagram of a hardware configuration of the server according to the second embodiment of the present invention. Since the hardware configuration of the server 3a is the same as that of the server 2a, illustration and description are omitted.

  The server 2a includes a CPU (Central Processing Unit) 201, a RAM (Random Access Memory) 202, a ROM (Read Only Memory) 203, a communication unit 204, and an HDD 230. The HDD 230 is a nonvolatile storage device, and stores a generation file 231, an OS 232, an application program 233, and a monitoring daemon program 234. The HDD 230 may further include a file information storage unit 240.

  The application program 233 is a computer program in which the application 210 is installed. Further, the application program 233 may include processing of the file processing unit 254. The monitoring daemon program 234 is a computer program in which a hash calculation unit 251, a determination unit 252, a file request unit 253, a file information registration unit 255, and a file transmission unit 256 are installed.

  The CPU 201 controls various processes in the server 2a, access to the RAM 202, ROM 203, communication unit 204, and HDD 230, and the like. The communication unit 204 communicates with the servers 3a, 5a,... 5n via the network 1.

  In the server 2a, the CPU 201 reads and executes the OS 232, the application program 233, and the monitoring daemon program 234 stored in the RAM 202, the ROM 203, or the HDD 230. Thereby, the server 2a functions as the application 210 and the monitoring daemon 220, and can perform remote file acquisition processing and processing on various files.

  FIG. 7 is a flowchart showing the flow of file information registration processing according to the second embodiment of the present invention. Here, first, the application 210 generates the generation file 231 and stores it in the HDD 230. At this time, the monitoring daemon 220 detects the generated file 231 stored in the HDD 230. Then, the monitoring daemon 220 calculates a hash value 242 from the file path name 241 of the generated file 231 (S21).

  Next, the monitoring daemon 220 determines whether or not the calculated hash value 242 is unregistered in the file information storage unit 240 (S22). At this time, the monitoring daemon 220 may determine whether or not the file path name 241 is unregistered in the file information storage unit 240.

  When it is determined in step S22 that the hash value 242 is not registered, the monitoring daemon 220 associates the file path name 241, the hash value 242, the storage destination server information 243, and the update date / time information 244 in the file information storage unit 240. Store (S23). Thus, the file information storage unit 240 can manage information regarding the location of the newly created file.

  If it is determined in step S22 that the hash value 242 is not registered, the monitoring daemon 220 updates the update date / time information 244 in association with the calculated hash value 242 (S24). For example, when the application 210 updates the generated file 231, the monitoring daemon 220 detects the generated file 231 that is updated in the HDD 230. In this case, the process of step S24 is performed.

  Note that when a remote file is received from an external server, the monitoring daemon 220 similarly receives the received remote file and executes the processes of steps S21 to S24.

  FIG. 8 is a flowchart showing a flow of a file request process according to the second embodiment of the present invention. First, a file reference request is generated in the application 210. For example, the application 210 may refer to a file generated by itself or refer to a file whose file path is designated. Then, the application 210 performs file access to the HDD 230. At this time, the monitoring daemon 220 detects file access.

  Then, the monitoring daemon 220 determines whether or not the reference object related to the detected file access is a remote file (S31). Specifically, the monitoring daemon 220 determines whether or not the file path for the reference target exists in the file system of the generated file 231. Alternatively, when the file access from the application 210 to the HDD 230 results in an error, that is, when a message indicating that the reference target file does not exist is detected from the HDD 230, the monitoring daemon 220 determines that the reference target is a remote file. You may judge.

  If it is determined in step S31 that the reference target is not a remote file, the monitoring daemon 220 acquires the target file from the HDD 230 (S37). Alternatively, the application 210 may acquire the target file from the HDD 230. In any case, after that, the application 210 processes the acquired file.

  If it is determined in step S31 that the reference target is a remote file, the monitoring daemon 220 calculates a hash value 242 from the file path name 241 of the remote file (S32). Then, the monitoring daemon 220 determines whether or not the calculated hash value 242 is unregistered in the file information storage unit 240 (S33). At this time, the monitoring daemon 220 may determine whether or not the file path name 241 is unregistered in the file information storage unit 240.

  If it is determined in step S33 that the hash value 242 is unregistered, the monitoring daemon 220 transmits a remote file reference request by broadcasting (S34). At this time, the monitoring daemon 220 transmits the reference request including the hash value 242. Specifically, the monitoring daemon 220 includes the hash value 242 in the transmission packet. Thereby, the reference request is transmitted to the servers 3a, 5a,.

  If it is determined in step S33 that the hash value 242 is not registered, the monitoring daemon 220 acquires storage destination server information 243 from the file information storage unit 240 (S35). That is, the monitoring daemon 220 can specify the destination of the server in which the remote file to be referenced is stored based on the acquired storage destination server information 243. Then, the monitoring daemon 220 transmits a remote file reference request to the identified server (S36). Thereby, a reference request can be transmitted only to a specific server, and a remote file to be referenced can be acquired more quickly.

  FIG. 9 is a flowchart showing the flow of the file transmission process according to the second embodiment of the present invention. First, the monitoring daemon 320 receives a reference request from the server 2a (S41). Here, it is assumed that the server 3a has received a broadcast or direct reference request from the server 2a. Next, the monitoring daemon 320 extracts the hash value of the file included in the reference request (S42).

  Then, the monitoring daemon 320 determines whether or not the extracted hash value is unregistered in the file information storage unit 340 (S43). Specifically, the monitoring daemon 320 refers to the file information storage unit 340 and compares the extracted hash value with the hash value 342 stored in the file information storage unit 340. If it is determined in step S43 that the hash value has not been registered, the monitoring daemon 320 responds to the requesting server 2a that the file does not exist (S44). In this case, the monitoring daemon 220 receives a response from the server 3a and notifies the application 210 to that effect.

  If it is determined in step S43 that the hash value is not yet registered, the monitoring daemon 320 acquires the file path name 341 and the update date / time information 344 associated with the hash value (S45). Then, the monitoring daemon 320 acquires the generated file 331 from the HDD 330 based on the acquired file path name 341 (S46). Thereafter, the monitoring daemon 320 transmits the acquired generation file 331 to the requesting server 2a (S47). At this time, the monitoring daemon 320 transmits the update date and time information 344 to the server 2a.

  Thereafter, the monitoring daemon 220 receives a response from the server 3 a and inputs the generated file 331 included in the response to the application 210. As a result, the application 210 can refer to the generation file 331 and perform processing on the generation file 331.

  In addition, the monitoring daemon 220 executes the processes of steps S21 to S24 in FIG. 7 for the generated file 331. In particular, when the monitoring daemon 220 transmits a reference request by broadcasting, information regarding the location of the received remote file is newly registered in the file information storage unit 240. Thereafter, when the application 210 accesses the generated file 331 again, it is determined in step S31 in FIG. 8 that the reference target is a remote file, and the monitoring daemon 220 executes the processing from step S32 onward. At this time, since it is determined in step S33 that the hash value 242 has not been registered, the monitoring daemon 220 executes the processes of steps S35 and S36. Therefore, the monitoring daemon 220 does not need to transmit unnecessary reference requests on the network 1 by broadcasting, and can reduce the load on the network 1.

  As described above, according to the second embodiment of the present invention, a file can be distributed and stored using HDDs of a plurality of servers without using NAS, and the files can be shared among the servers. That is, Embodiment 2 of the present invention can be said to be a network file system 200 to which a distributed local cache is added.

  Further, since the file is stored as a cache in the HDD on each server and the cache is managed by the file information storage units 240 and 340, the file that is the input / output result of each application in each server is passed through the network file system. Reference and writing can be performed at high speed.

<Other embodiments of the invention>
As described above, according to the first and second embodiments of the present invention, a file that is an input / output result of a program executed on a server connected via a network can be referred and written at high speed and the load on the network can be reduced. To do.

  In the network file system 900 shown in FIG. 10, for example, when the same file is referred to many times or when the output file is reread, it is necessary to communicate with the NAS 91 every time the file is output. is there. In this case, the amount of data flowing on the network 1 increases, the load on the network 1 increases, and the data writing or reading speed decreases. Along with this, there is a problem that the execution speed of the program itself executed on each server is also affected. The NAS 91 also has a problem that when a large number of files are placed in one folder, the response when referring to the folder is delayed.

  Therefore, in the first and second embodiments of the present invention, HDDs of a certain size are connected to individual servers that perform calculations using various applications, and the output results of applications in each server are placed in each HDD as a cache. Is. In addition, each server is provided with file information storage units 22, 240, 340, etc. that manage the location of the output result as a cache database, so that the output result can be referred from another server without using NAS 91. That is, file sharing can be performed and the load on the network can be reduced. Therefore, a program that performs file access via a network can be executed at high speed.

  Furthermore, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present invention described above. For example, in the above-described embodiment, the present invention has been described as a hardware configuration, but the present invention is not limited to this. The present invention can also realize arbitrary processing by causing a CPU (Central Processing Unit) to execute a computer program. In this case, the computer program can be provided by being recorded on a recording medium, or can be provided by being transmitted via the Internet or another transmission medium. Examples of the recording medium include a flexible disk, a hard disk, a magnetic disk, a magneto-optical disk, a CD-ROM (Compact Disc Read Only Memory), a DVD (Digital Versatile Disc), and a BD (Blu-ray (registered trademark) Disc). ROM (Read Only Memory) cartridge, battery-backed RAM (Random Access Memory) memory cartridge, flash memory cartridge, nonvolatile RAM cartridge, and the like. The communication medium includes a wired communication medium such as a telephone line, a wireless communication medium such as a microwave line, and the like.

  A part or all of the above embodiments can be described as in the following supplementary notes, but is not limited thereto.

(Supplementary Note 1) A first server that includes a first file information storage unit that stores storage server information for specifying a server in which a file is stored, and that processes a remote file stored in an external server;
A second server for storing the remote file,
When the first server refers to the remote file, the first server transmits a request for referring to the remote file to a plurality of servers including the second server,
The second server sends the remote file requested by the reference request to the first server;
The first server is
When the remote file is received from the second server, the second server is used as the storage destination server information, and is registered in the first file information storage unit in association with the received remote file identification information. ,
When referring to the remote file again, the storage server information associated with the identification information of the remote file is identified as the second server by referring to the first file information storage unit. A distributed file system, wherein a reference request including identification information of the remote file is transmitted to the second server.

  (Supplementary Note 2) When the first server refers to the remote file, it determines whether or not the remote file identification information exists in the first file information storage unit, and determines that it exists When a reference request including the remote file identification information is transmitted to the second server identified from the storage destination server information associated with the remote file identification information, and it is determined that the remote file does not exist The distributed file system according to appendix 1, wherein a reference request including identification information of the remote file is transmitted to a plurality of servers including the second server.

(Appendix 3) The second server is
A second file information storage unit for storing storage destination server information indicating a server in which the file is stored;
When the remote file is generated, the self is set as the storage destination server information, and is registered in the second file information storage unit in association with the identification information of the remote file,
When referring to the remote file, the first server transmits the reference request including identification information of the remote file,
The second server is
When the reference request is received from the first server, the reference request is referred to the second file information storage unit, and the reference request is included in the identification information of the file associated as the storage destination server information. To determine whether the identification information contained in exists,
The distributed file system according to appendix 1 or 2, wherein the remote file is transmitted to the first server when it is determined that the identification information exists.

(Supplementary Note 4) When the second server generates the remote file, the second server calculates a hash value from the file name of the generated remote file, sets itself as the storage destination server information, and associates the hash value with the hash value. Register to the second file information storage unit,
The first server calculates a hash value from the file name of the remote file, transmits the reference request including the hash value,
The second server is
When the reference request is received from the first server, the second file information storage unit is referred to and is included in the reference request in the hash value associated with the storage destination server information. Determine whether there is a hash value to be
The distributed file system according to appendix 3, wherein the remote file is transmitted to the first server when it is determined that the hash value exists.

  (Supplementary note 5) The distributed file system according to supplementary note 4, wherein the first server and the second server generate the hash value including a directory name in which the remote file is stored.

(Appendix 6) The second server is
When the remote file is updated, the update date and time information of the remote file is further registered in the second file information storage unit in association with the identification information of the remote file,
When the reference request is received from the first server and it is determined that the identification information exists, the update date / time information associated with the identification information included in the reference request is obtained from the second file information storage unit. And sending the remote file including the acquired update date and time information to the first server,
When the first server receives the remote file from the second server, the first server further registers the update date / time information in the first file information storage unit in association with identification information of the received remote file. 6. The distributed file system according to any one of appendices 3 to 5, characterized in that:

(Supplementary note 7) When the first server refers to the remote file, it transmits a reference request including the identification information of the remote file to a plurality of servers including the second server by broadcast,
The distributed file system according to any one of appendices 1 to 6, wherein the second server receives the reference request from the first server by broadcast.

(Additional remark 8) The file information storage part which associates and stores the identification information of a file, and the storage destination server information which specifies the server where the said file is stored,
A determination unit that determines whether or not the remote file identification information exists in the file information storage unit when referring to a remote file stored in an external server;
When the determination unit determines that the identification information of the remote file exists, the file information storage unit is referred to and the server identified from the storage destination server information associated with the identification information of the remote file A file request unit that transmits a remote file reference request to a plurality of servers when it is determined that the remote file identification information does not exist;
A file processing unit that performs processing based on the received remote file when the remote file is received as a response from the server that has received the reference request;
A remote file processing server comprising:

(Additional remark 9) When the said determination part receives the said remote file, it determines whether the identification information of the said received remote file exists in the said file information storage part,
When the determination unit determines that there is no identification information of the received remote file, the server that is the transmission source of the remote file is set as the storage server information, and is associated with the received identification information of the remote file. 9. The remote file processing server according to appendix 8, further comprising a file information registration unit for registering in the file information storage unit.

  (Supplementary Note 10) When the file processing unit generates a file, the file information registration unit sets itself as the storage server information and registers it in the file information storage unit in association with identification information of the generated file. The remote file processing server according to appendix 9, characterized by:

(Supplementary Note 11) When the determination unit receives the reference request from an external server, the determination unit refers to the file information storage unit, and identifies itself as the storage server information in the identification information of the file. Determine whether the identification information of the remote file requested by the reference request exists,
And a file transmission unit configured to transmit a file corresponding to the requested remote file identification information to the external server when the determination unit determines that the requested remote file identification information exists. 11. The remote file processing server according to any one of appendices 8 to 10, characterized in that:

(Additional remark 12) It further has the hash value calculation part which calculates a hash value from the file name of a file,
The remote file processing server according to any one of appendices 8 to 11, wherein the file information storage unit stores the hash value calculated by the hash value calculation unit as identification information of the file.

  (Supplementary note 13) The remote file processing server according to supplementary note 12, wherein the hash value calculation unit calculates the hash value including a directory name in which the file is stored.

  (Supplementary note 14) The remote file processing server according to any one of supplementary notes 8 to 13, wherein the file information storage unit stores update date information of the file in association with identification information of the file.

  (Supplementary note 15) The remote file processing server according to any one of supplementary notes 8 to 14, wherein the file request unit transmits a reference request for the remote file to the plurality of servers by broadcast.

(Supplementary Note 16) A first server that includes a first file information storage unit that stores storage server information that identifies a server in which a file is stored, and that processes a remote file stored in an external server;
A second server for storing the remote file,
In the first server, when referring to the remote file, a request to reference the remote file is transmitted to a plurality of servers including the second server,
In the second server, the remote file requested by the reference request is transmitted to the first server,
In the first server,
When the remote file is received from the second server, the second server is used as the storage destination server information, and is registered in the first file information storage unit in association with the received remote file identification information. ,
When referring to the remote file again, refer to the first file information storage unit, specify that the storage destination server information associated with the identification information of the remote file is the second server,
A remote file processing method, comprising: transmitting a reference request including identification information of the remote file to the identified second server.

(Supplementary Note 17) In the first server,
When referring to the remote file, it is determined whether the identification information of the remote file exists in the first file information storage unit,
If it is determined that it exists, a reference request including the remote file identification information is transmitted to the second server specified from the storage destination server information associated with the remote file identification information;
18. The remote file processing method according to appendix 16, wherein a reference request including identification information of the remote file is transmitted to a plurality of servers including the second server when it is determined that the file does not exist.

(Supplementary Note 18) The second server further includes a second file information storage unit that stores storage destination server information indicating a server in which a file is stored,
In the second server, when the remote file is generated, the self is used as the storage destination server information, is registered in the second file information storage unit in association with the identification information of the remote file,
In the first server, when referring to the remote file, the reference request including the identification information of the remote file is transmitted,
In the second server,
When the reference request is received from the first server, the reference request is referred to the second file information storage unit, and the reference request is included in the identification information of the file associated as the storage destination server information. To determine whether the identification information contained in exists,
The remote file processing method according to appendix 16 or 17, wherein when it is determined that the identification information exists, the remote file is transmitted to the first server.

(Supplementary note 19) In the second server, when the remote file is generated, a hash value is calculated from the file name of the generated remote file, self is used as the storage destination server information, and the hash value is associated with the hash value. Register to the second file information storage unit,
In the first server, a hash value is calculated from the file name of the remote file, the reference request including the hash value is transmitted,
In the second server,
When the reference request is received from the first server, the second file information storage unit is referred to and is included in the reference request in the hash value associated with the storage destination server information. Determine whether there is a hash value to be
The remote file processing method according to appendix 18, wherein the remote file is transmitted to the first server when it is determined that the hash value exists.

  (Supplementary note 20) The remote file processing method according to supplementary note 19, wherein the hash value is generated including a directory name in which the remote file is stored in the first server and the second server. .

(Supplementary note 21) In the second server,
When the remote file is updated, the update date and time information of the remote file is further registered in the second file information storage unit in association with the identification information of the remote file,
When the reference request is received from the first server and it is determined that the identification information exists, the update date / time information associated with the identification information included in the reference request is obtained from the second file information storage unit. And sending the remote file including the acquired update date and time information to the first server,
In the first server,
Appendix 18 wherein when the remote file is received from the second server, the update date and time information is further registered in the first file information storage unit in association with the received remote file identification information. 21. The remote file processing method according to any one of items 1 to 20.

(Supplementary note 22) When referring to the remote file in the first server, a reference request including identification information of the remote file is transmitted by broadcast to a plurality of servers including the second server,
The remote file processing method according to any one of supplementary notes 16 to 21, wherein the second server receives the reference request from the first server by broadcast.

(Additional remark 23) When referring to the remote file stored in the external server, the file information storage unit for storing the file identification information and the storage destination server information for specifying the server storing the file in association with each other A determination process for determining whether or not the remote file identification information exists,
When it is determined by the determination process that the identification information of the remote file exists, the file information storage unit is referred to and the server specified from the storage destination server information associated with the identification information of the remote file A file request process for transmitting a remote file reference request to a plurality of servers when it is determined that the remote file identification information does not exist;
When the remote file is received as a response from the server that has received the reference request, file processing that performs processing based on the received remote file;
A remote file processing program for causing a computer to execute remote file processing including

(Additional remark 24) When the said determination process receives the said remote file, it determines whether the identification information of the said received remote file exists in the said file information storage part,
If it is determined by the determination processing that the received remote file identification information does not exist, the server of the remote file transmission source is the storage server information, and the remote file identification information is associated with the received remote file identification information. 24. The remote file processing program according to appendix 23, further causing the computer to execute file information registration processing registered in a file information storage unit.

  (Supplementary Note 25) When the file processing generates a file, the file information registration processing uses the storage server information as the storage server information and registers it in the file information storage unit in association with identification information of the generated file. 25. The remote file processing program according to supplementary note 24.

(Supplementary Note 26) When the determination process receives the reference request from an external server, the file processing unit refers to the file information storage unit, and identifies itself as the storage server information in the file identification information. Determine whether the identification information of the remote file requested by the reference request exists,
If the determination process determines that the requested remote file identification information exists, the computer performs a file transmission process for transmitting a file corresponding to the requested remote file identification information to the external server. 26. The remote file processing program according to any one of appendices 23 to 25, which is further executed.

(Additional remark 27) Let the said computer further perform the hash value calculation process which calculates a hash value from the file name of a file,
The file request process includes the hash value calculated by the hash value calculation process from the file name of the remote file as the identification information of the remote file, and transmits the reference request.
The remote file processing program according to any one of appendices 23 to 26, wherein the file information storage unit stores the hash value calculated by the hash value calculation unit as identification information of the file. .

  (Supplementary note 28) The remote file processing program according to supplementary note 27, wherein the hash value calculation processing calculates the hash value including a directory name in which the file is stored.

  (Supplementary note 29) The remote file processing according to any one of supplementary notes 23 to 28, wherein the file information storage unit stores update date and time information of the file in association with identification information of the file. program.

  (Supplementary note 30) The remote file processing program according to any one of supplementary notes 23 to 29, wherein in the file request processing, a remote file reference request is transmitted by broadcast to the plurality of servers.

DESCRIPTION OF SYMBOLS 100 Distributed file system 1 Network 2 Server 21 Control part 22 File information storage part 221 File identification information 222 Storage destination server information 3 Server 31 HDD
311 Remote file 4a,... 4n Server 200 Network file system 2a Server 201 CPU
202 RAM
203 ROM
204 Communication Unit 210 Application 220 Monitoring Daemon 230 HDD
231 Generated file 232 OS
233 Application program 234 Monitoring daemon program 240 File information storage unit 241 File path name 242 Hash value 243 Storage destination server information 244 Update date / time information 251 Hash calculation unit 252 Judgment unit 253 File request unit 254 File processing unit 255 File information registration unit 256 file Transmission unit 3a server 310 application 320 monitoring daemon 330 HDD
331 Generated file 340 File information storage unit 341 File path name 342 Hash value 343 Storage destination server information 344 Update date / time information 5a,... 5n Server 900 Network file system 91 NAS
910 HDD
911 remote file 92 server 920 application 93 server 930 application 94a, ... 94n server

Claims (10)

A first server that includes a first file information storage unit that stores storage destination server information that identifies a server in which a file is stored, and that processes a remote file stored in an external server;
A second server for storing the remote file,
When the first server refers to the remote file, the first server transmits a request for referring to the remote file to a plurality of servers including the second server,
The second server sends the remote file requested by the reference request to the first server;
The first server is
When the remote file is received from the second server, the second server is used as the storage destination server information, and is registered in the first file information storage unit in association with the received remote file identification information. ,
When referring to the remote file again, the storage server information associated with the identification information of the remote file is identified as the second server by referring to the first file information storage unit. A distributed file system, wherein a reference request including identification information of the remote file is transmitted to the second server.   When the first server refers to the remote file, the first server determines whether or not the remote file identification information exists in the first file information storage unit. When a reference request including the remote file identification information is transmitted to the second server specified from the storage destination server information associated with the file identification information, and the second request is determined not to exist, the second The distributed file system according to claim 1, wherein a reference request including identification information of the remote file is transmitted to a plurality of servers including the server. The second server is
A second file information storage unit for storing storage destination server information indicating a server in which the file is stored;
When the remote file is generated, the self is set as the storage destination server information, and is registered in the second file information storage unit in association with the identification information of the remote file,
When referring to the remote file, the first server transmits the reference request including identification information of the remote file,
The second server is
When the reference request is received from the first server, the reference request is referred to the second file information storage unit, and the reference request is included in the identification information of the file associated as the storage destination server information. To determine whether the identification information contained in exists,
3. The distributed file system according to claim 1, wherein when it is determined that the identification information exists, the remote file is transmitted to the first server. 4. When the second server generates the remote file, the second server calculates a hash value from the file name of the generated remote file, sets self as the storage destination server information, and associates the hash value with the second file. Register to the information storage unit,
The first server calculates a hash value from the file name of the remote file, transmits the reference request including the hash value,
The second server is
When the reference request is received from the first server, the second file information storage unit is referred to and is included in the reference request in the hash value associated with the storage destination server information. Determine whether there is a hash value to be
4. The distributed file system according to claim 3, wherein when it is determined that the hash value exists, the remote file is transmitted to the first server.   The distributed file system according to claim 4, wherein the first server and the second server generate the hash value including a directory name in which the remote file is stored. The second server is
When the remote file is updated, the update date and time information of the remote file is further registered in the second file information storage unit in association with the identification information of the remote file,
When the reference request is received from the first server and it is determined that the identification information exists, the update date / time information associated with the identification information included in the reference request is obtained from the second file information storage unit. And sending the remote file including the acquired update date and time information to the first server,
When the first server receives the remote file from the second server, the first server further registers the update date / time information in the first file information storage unit in association with identification information of the received remote file. The distributed file system according to any one of claims 3 to 5, wherein the file system is a distributed file system. When the first server refers to the remote file, it transmits a reference request by broadcast to a plurality of servers including the second server, including identification information of the remote file,
The distributed file system according to claim 1, wherein the second server receives the reference request from the first server by broadcast. A file information storage unit that stores the identification information of a file in association with storage destination server information that identifies a server in which the file is stored;
A determination unit that determines whether or not the remote file identification information exists in the file information storage unit when referring to a remote file stored in an external server;
When the determination unit determines that the identification information of the remote file exists, the file information storage unit is referred to and the server identified from the storage destination server information associated with the identification information of the remote file A file request unit that transmits a remote file reference request to a plurality of servers when it is determined that the remote file identification information does not exist;
A file processing unit that performs processing based on the received remote file when the remote file is received as a response from the server that has received the reference request;
A remote file processing server comprising: A first server that includes a first file information storage unit that stores storage destination server information that identifies a server in which a file is stored, and that processes a remote file stored in an external server;
A second server for storing the remote file,
In the first server, when referring to the remote file, a request to reference the remote file is transmitted to a plurality of servers including the second server,
In the second server, the remote file requested by the reference request is transmitted to the first server,
In the first server,
When the remote file is received from the second server, the second server is used as the storage destination server information, and is registered in the first file information storage unit in association with the received remote file identification information. ,
When referring to the remote file again, refer to the first file information storage unit, specify that the storage destination server information associated with the identification information of the remote file is the second server,
A remote file processing method, comprising: transmitting a reference request including identification information of the remote file to the identified second server. When referring to a remote file stored in an external server, the remote file is stored in a file information storage unit that stores the file identification information in association with storage destination server information that identifies the server in which the file is stored. Determination processing for determining whether or not the identification information exists;
When it is determined by the determination process that the identification information of the remote file exists, the file information storage unit is referred to and the server specified from the storage destination server information associated with the identification information of the remote file A file request process for transmitting a remote file reference request to a plurality of servers when it is determined that the remote file identification information does not exist;
When the remote file is received as a response from the server that has received the reference request, file processing that performs processing based on the received remote file;
A remote file processing program for causing a computer to execute remote file processing including

Images