JP5271392B2 - Distributed file management system, distributed file arrangement method and program - Google Patents

Description

  The present invention relates to a distributed file management system that manages a file by dividing it into a plurality of blocks, and more particularly to a technique for distributing and replicating block replicas to a plurality of machines.

  As a distributed file management system having fault tolerance, there is Google File System (GFS) (for example, see Non-Patent Document 1). In this GFS, a file is divided into fixed-length (64 MB) blocks called chunks, and replicas of the blocks are distributed to a plurality of machines for management.

  FIG. 13 is a diagram for explaining the relationship among files, blocks, and replicas.

  As shown in FIG. 13, for example, a file is divided into three blocks, and a replica is distributed and managed in three machines for each of the three blocks. At this time, in order to withstand not only a machine level failure but also an edge switch level failure containing a group of machines, a plurality of replicas are arranged on machines under different edge switches. As a result, even if a failure occurs in one edge switch, only one replica is lost per block, and it is avoided that all replicas are lost. In addition, by independently determining the replica placement destination for each block, even if a failure occurs in one edge switch, the probability that one replica of all the blocks making up one file will be lost at a stretch is reduced. .

Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung. 2003. The Google file system. In Proceedings of the nineteenth ACM symposium on Operating systems principles (SOSP '03). ACM, New York, NY, USA, 29-43. DOI = 10.1145 / 945445.945450 http://doi.acm.org/10.1145/945445.945450

  However, the replica placement method using GFS as described above has the following problems.

  The file has high fault tolerance, but the file writing efficiency and reading efficiency are poor.

  FIG. 14 is a diagram for explaining a problem at the time of file writing in the replica placement method by GFS.

  As shown in FIG. 14, in the replica placement method using GFS, a file is configured in order to place a plurality of replicas for each block into which a file is divided in machines 4 under different edge switches 1-1 to 1-9. At the time of writing each block, the number of times of crossing between the edge switches 1-1 to 1-9 is large, and thereby the file writing efficiency is deteriorated.

  FIG. 15 is a diagram for explaining a problem in reading a file in the replica placement method by GFS.

  As shown in FIG. 15, in the replica placement method using GFS, since the edge switches 1-1 to 1-9 of the replica placement destination are independently determined for each block constituting the file, all blocks constituting the file are determined. At the time of reading, the probability of straddling the edge switches 1-1 to 1-9 is high, and thereby the file reading efficiency deteriorates.

  Since the GFS replica placement method as described above is fixed in the system, the file write efficiency and read efficiency must be sacrificed even when the fault tolerance of the file is not so much required.

  The present invention has been made in view of the problems of the conventional techniques as described above, and allows an application developer to select a trade-off between file fault tolerance and file write efficiency and read efficiency. Another object of the present invention is to provide a distributed file management system, a distributed file arrangement method, and a program that can be optimized for each file according to the use of each file.

In order to achieve the above object, the present invention provides:
A distributed file management system that divides a file into a plurality of blocks and places R replicas for each of the plurality of blocks on a plurality of machines under an edge switch,
A first machine that arranges i (0 ≦ i ≦ R) replicas of the R replicas, and j (0 ≦ j ≦ (R−i)) of (R−i) replicas A second machine in which the replica is placed, and the first machine is a machine under an edge switch under the control of a client machine that writes a file for each block, and the second machine is for each file. A block replica generation processing unit that selects a machine under a specific edge switch,
A file writing processing unit that performs processing of writing a file by placing a replica on the first and second machines selected by the block replica generation processing unit;
A file read processing unit that performs file read processing as a client machine that reads a file from a machine under the specific edge switch under the second machine selected by the block replica generation processing unit; Have.

Also, a distributed file placement method for dividing a file into a plurality of blocks and placing R replicas for each of the plurality of blocks on a plurality of machines under an edge switch,
A first machine that arranges i (0 ≦ i ≦ R) replicas of the R replicas, and j (0 ≦ j ≦ (R−i)) of (R−i) replicas A second machine in which the replica is placed, and the first machine is a machine under an edge switch under the control of a client machine that writes a file for each block, and the second machine is for each file. Block replica generation processing to select to be a machine under a specific edge switch,
A file writing process for writing a file by placing a replica on the first and second machines selected in the block replica generation process;
A file read process for performing a file read process, with the machine under the specific edge switch under the second machine selected in the block replica generation process serving as a client machine for reading the file.

In addition, the file is divided into a plurality of blocks, and R replicas for each of the plurality of blocks are arranged on a computer under a plurality of edge switches.
A first machine that arranges i (0 ≦ i ≦ R) replicas of the R replicas, and j (0 ≦ j ≦ (R−i)) of (R−i) replicas A second machine in which the replica is placed, and the first machine is a machine under an edge switch under the control of a client machine that writes a file for each block, and the second machine is for each file. A block replica generation procedure to select a machine under a specific edge switch,
A file writing procedure for performing a process of writing a file by placing a replica on the first and second machines selected in the block replica generating procedure;
A file read procedure for performing a file read process as a client machine for reading a file, the machine under the specific edge switch under the second machine selected in the block replica generation procedure Program for.

  The present invention provides a first machine that arranges i (0 ≦ i ≦ R) replicas of R replicas, and j (0 ≦ j ≦ (R−) of (R−i) replicas. i)) the second machine on which the replica is placed, the first machine becomes a machine under the edge switch under the control of the client machine that writes the file for each block, and the second machine is the file Each time it is selected to be a machine under a specific edge switch, a file is written by placing a replica on the selected first and second machines, and the selected second machine is also subordinate The machine under the specified edge switch is configured to read the file as a client machine that reads the file, so the fault tolerance of the file and the file write efficiency and read The trade-off between out efficiency, can select an application developer, it may be optimized file units according to the application of each file.

It is a figure for demonstrating the effect at the time of the file writing by the distributed file arrangement | positioning method of this invention. It is a figure for demonstrating the effect at the time of the file reading by the distributed file arrangement | positioning method of this invention. It is a figure which shows one Embodiment of the hardware constitution assumed with the distributed file management system of this invention. It is a figure which shows one Embodiment of the software structure assumed with the distributed file management system of this invention. FIG. 5 is a diagram showing an assumed scenario for explaining a distributed file arrangement method in the distributed file management system shown in FIGS. 3 and 4. It is a figure which shows the file block information (after file generation) registered into the file block information storage part shown in FIG. 5 is a flowchart for explaining block replica generation processing executed by a block replica generation processing unit shown in FIG. 4. It is a figure which shows an example of the worker machine information registered into the worker machine information storage part shown in FIG. It is a figure which shows the file block information (after block production | generation) registered into the file block information storage part shown in FIG. It is a figure which shows the block information registered into the block information storage part shown in FIG. FIG. 5 is a diagram illustrating client machine information registered in a client machine information storage unit illustrated in FIG. 4. It is a figure which shows other embodiment of the hardware constitution assumed with the distributed file management system of this invention. It is a figure for demonstrating the relationship between a file, a block, and a replica. It is a figure for demonstrating the problem at the time of the file writing in the replica arrangement | positioning method by GFS. It is a figure for demonstrating the problem at the time of the file reading in the replica arrangement | positioning method by GFS.

  Embodiments of the present invention will be described below with reference to the drawings.

  First, an outline of the present invention will be described.

  In the present invention, the following {i, j} pairs are defined as replica placement methods for blocks constituting a file.

  Of the R replicas, i (0 ≦ i ≦ R) are arranged on a machine (first machine) under one local edge switch that accommodates a machine serving as a file writer.

  Of the (R−i) replicas, j (0 ≦ j ≦ (R−i)) are machines (seconds) under a specific edge switch that the system randomly determines for each file when each file is generated. 2 machines). Note that (R-i-j) replicas are arranged in machines under edge switches different from the above. In addition, a means for acquiring specific edge switch information determined for each file at the time of file generation is provided, and a machine under the specific edge switch is used as a reader of the file.

  Then, the replica placement method {i, j} is held for each file, and the replica placement method {i, j} is applied for each file.

  Below, the effect by the structure mentioned above is demonstrated.

  FIG. 1 is a diagram for explaining the effect at the time of file writing by the distributed file arrangement method of the present invention, and shows the case of R = 3, {i, j} = {2, 1}.

  In the example shown in FIG. 1, for each of the three blocks constituting a file, local edge switches 1-1 and 1 each accommodating two replicas out of three replicas and a machine 4 serving as a file writer. -4 and 1-7 are arranged in the machine 4 and one replica is arranged in the machine 4 under the specific edge switch 1-5.

  In this way, of the R replicas, i (0 ≦ i ≦ R) are the machines 4 under the local edge switches 1-1, 1-4, and 1-7 in which the machine 4 serving as the file writer is accommodated. And a specific edge switch 1 in which the system randomly determines j (0 ≦ j ≦ (R−i)) of (R−i) replicas for each file when generating each file. The number of replicas of a certain block increases when the failure of one edge switch increases as i is increased by placing them on the machines 4 under 5 respectively. At the time of writing, the file writing efficiency is improved by the amount that the number of times between the edge switches 1-1 to 1-9 is reduced. When i is increased from 0 to 1, the maximum number of lost replicas does not increase.

  FIG. 2 is a diagram for explaining the effect at the time of file reading by the distributed file arrangement method of the present invention, and shows the case of R = 3, {i, j} = {2, 1}.

  In the example shown in FIG. 2, two of the three replicas are arranged in the machines 4 under the edge switches 1-1, 1-4, and 1-7 for each of the three blocks constituting the file. One replica is arranged in the machine 4 under the edge switch 1-5, which will be a file reader later.

  In this way, i (0 ≦ i ≦ R) of the R replicas are arranged in the machines 4 under the edge switches 1-1, 1-4, and 1-7, and (R−i) J of replicas (0 ≦ j ≦ (R−i)) are respectively placed on the machines 4 under the specific edge switch 1-5 that the system randomly determines for each file when each file is generated, By using this machine 4 as the reader of the file, the more j is increased, the higher the probability that one replica of all the blocks that make up a file will be lost at once when a failure occurs in one edge switch. However, the file reading efficiency is improved as much as the file reading is closed in the edge switch. Note that when j is increased to two or more, the file reading efficiency is improved by the amount that the reading of the file closed in the edge switch can be distributed to two or more.

  FIG. 3 is a diagram showing an embodiment of a hardware configuration assumed in the distributed file management system of the present invention.

In this embodiment, as shown in FIG. 3, the edge switches 1-1 to 1-9 accommodate a plurality of groups of machines 4, and the core switch 2 accommodates the groups of edge switches 1-1 to 1-9. The IP address of each machine 4 is given so that it can be determined whether or not it is accommodated under the same edge switch by applying a certain mask (referred to as a machine group mask). In this form, for example, if the machine group mask “255.255.255.0” is applied to the machine “192.168.1.1” and the machine “192.168.1.10”, the same machine group “192.168.1. ^* ” Is obtained. 1.1 "and the machine" 192.168.1.10 "are stored in the same edge switch.

  FIG. 4 is a diagram showing an embodiment of a software configuration assumed in the distributed file management system of the present invention.

  As shown in FIG. 4, the present embodiment includes a client function 10, a master function 20, and a worker function 30, and the client function 10 includes a common function 10a and an individual function 10b. . The master function 20 operates only on one certain machine, and the client function 10 and the worker function 30 operate on a plurality of machines. The client function 10, the master function 20, and the worker function 30 may coexist on the same machine.

  The individual function 10 a of the client function 10 includes a client processing unit 11 and a client machine information storage unit 12. The common function 10b of the client function 10 includes a file generation processing unit 13, a specific edge switch information acquisition processing unit 14, a file writing processing unit 15, and a file reading processing unit 16.

  The master function 20 includes a file information update processing unit 21, a file information reference processing unit 22, a file / block information storage unit 23, a writing block information acquisition processing unit 24, a reading block information acquisition processing unit 25, The block replica generation processing unit 26 and the worker machine information storage unit 20 are configured.

  The worker function 30 includes a block generation processing unit 31, a block writing processing unit 32, a block reading processing unit 33, and a block information storage unit 34.

  A distributed file arrangement method in the distributed file management system configured as described above will be described below.

  FIG. 5 is a diagram showing an assumed scenario for explaining a distributed file arrangement method in the distributed file management system shown in FIGS. 3 and 4.

  Based on the assumed scenario shown in FIG. 5, (1) file generation processing, (2) file writing processing, (3) specific edge switch information acquisition processing, and (4) file reading processing will be described separately. In particular, an algorithm and data structure related to block replica generation processing, which is a point of the present invention, will be described in detail.

(1) File Generation Processing First, the client processing unit 11 uses the API provided by this distributed file management system to program values input by the application developer, so that the client processing unit 11 generates a file name, the number of replicas, and a replica. The arrangement method is input to the file generation processing unit 13.

  Then, the file generation processing unit 13 transmits the input file name, the number of replicas, and the replica arrangement method to the file information update processing unit 21.

  The file information update processing unit 21 receives the file name, the number of replicas, and the replica arrangement method transmitted from the file generation processing unit 13, and stores the file name, the number of replicas, and the replica arrangement method in the file block information storage unit. 23.

  FIG. 6 is a diagram showing file / block information (after file generation) registered in the file / block information storage unit 23 shown in FIG.

  As shown in FIG. 6, a file name, the number of replicas, and a replica arrangement method are registered in association with each other in the file / block information storage unit 23 after file generation. In this embodiment, the generated file name Is “file 1”, the number of replicas of the blocks constituting the file is “3”, and the replica arrangement method is “{2, 1}”.

(2) File Write Processing First, in the same manner as described above, the client processing unit 11 inputs a file name to be written, a write offset, a write size, and write data to the file write processing unit 15.

  Then, the file write processing unit 15 calculates the index of the block to be written (the number of the block to be written from the beginning of the file) from the input write offset and write size, and together with the input file name And sent to the writing block information acquisition processing unit 24.

  If the block indicated by the received file name and index is not generated with reference to the file / block information storage unit 23, the writing block information acquisition processing unit 24 sends the block / replication generation processing unit 26 to the block replica generation processing unit 26. The file write processing unit 15 receives the block ID generated by the block replica generation processing unit 26 and the worker machine information (machine information on which the worker function operates: IP address) where the replica of the block is arranged. Send back. If the block indicated by the received file name and index has been generated, the generated machine ID and the worker machine information where the replica of the block is arranged are returned to the file write processing unit 15.

  Here, the block replica generation processing executed by the block replica generation processing unit 26 will be described in detail.

  FIG. 7 is a flowchart for explaining the block replica generation processing executed by the block replica generation processing unit 26 shown in FIG.

  First, the block / replica generation processing unit 26 refers to the file / block information storage unit 23, and acquires the number of replicas of the file name as R and the replica placement method as {i, j} (step 1).

  Next, the block replica generation processing unit 26 applies the machine group mask to the IP address of the client machine, and acquires the client machine group to which the client machine belongs (step 2).

  Next, the block replica generation processing unit 26 applies the machine group mask to the IP addresses of all worker machines described in the worker machine information registered in the worker machine information storage unit 27, and all the worker machines are registered. A set of worker machine groups to which the machine belongs is acquired (step 3).

  FIG. 8 is a diagram illustrating an example of worker machine information registered in the worker machine information storage unit 27 illustrated in FIG. 4.

  As shown in FIG. 8, machine information on which the worker function 30 operates is registered in the worker machine information storage unit 27, and the block / replica generation processing unit 26 uses this machine information to identify all worker machines. Get the set of worker machine groups to which the belongs.

  Next, the block replica generation processing unit 26 selects the same worker machine group as the client machine group acquired in step 2 from the set of worker machine groups acquired in step 3, and the worker machine belonging to the worker machine group is selected. I units are selected from the machines (step 4). In selecting a worker machine, if there is a worker machine identical to the client machine, it is selected with the highest priority, and if there is no worker machine identical to the client machine, it is randomly selected.

  Next, the block replica generation processing unit 26 refers to the file / block information storage unit 23 and checks whether or not the specific edge switch information of the file name is registered (step 5).

  If the specific edge switch information of the file name is not registered as a result of step 5, the block replica generation processing unit 26 acquired in step 2 from the set of worker machine groups acquired in step 3. A worker machine group different from the client machine group is randomly selected (R-i), and one worker machine is selected from each of the worker machines belonging to the (R-i) worker machine groups. Select at random (step 6).

  Next, the block replica generation processing unit 26 randomly selects j from the (R-i) number of worker machine groups selected in step 6, and identifies the j worker machine groups for the file name. It is registered in the file / block information storage unit 23 as edge switch information (step 7).

  If the specific edge switch information of the file name is registered as a result of step 5, the block replica generation processing unit 26 refers to the file / block information storage unit 23 and corresponds to the file name. Acquire j worker machine groups registered as specific edge switch information, and from among a plurality of worker machines belonging to j worker machine groups (excluding i worker machines selected in step 4), worker machines One worker machine is selected at random for each group (step 8).

  Next, the block replica generation processing unit 26 selects a worker machine that is different from the client machine group acquired in Step 2 and the j worker machine groups acquired in Step 8 from the set of worker machine groups acquired in Step 3. Select (Rij) groups at random, and randomly select one worker machine from each of the worker machines belonging to (Rij) worker machine groups. (Step 9).

  After that, the block replica generation processing unit 26 corresponds to the file name and the index to the block generation processing units 31 on the total R worker machines selected in step 4, 6 or steps 4, 8, and 9. A block ID is requested to be generated, and the selected block ID and R pieces of worker machine information are registered in the file / block information storage unit 23 (step 10).

  FIG. 9 is a diagram showing file / block information (after block generation) registered in the file / block information storage unit 23 shown in FIG.

As shown in FIG. 9, the file name, the number of replicas, the replica placement method, the specific edge switch information, the block ID, and the worker machine information are registered in the file / block information storage unit 23 after the block is generated. In this embodiment, the writer client machine with the block ID “block 1” constituting the file name “file 1” is “192.168.1.10”, and the writer client machine with “block 2” is “192.168.4.10”. The writer client machine of “Block 3” is “192.168.7.10”. Of the “3” replicas of each block, “2” must be located in a worker machine (including the same worker machine as the writer client machine) belonging to the same local machine group as each writer client machine. Recognize. Further, it can be seen that “1” out of “3” replicas of each block is arranged in a worker machine belonging to a specific machine group “192.168.5. ^* ”. In the block information storage unit 34 on each worker machine, the correspondence relationship between the block ID and the block entity is registered as block information by the block generation processing unit 31.

  FIG. 10 is a diagram showing block information registered in the block information storage unit 34 shown in FIG.

  As shown in FIG. 10, in the block information storage unit 34 on the worker machine, the correspondence between block IDs and block entities is registered as block information. The block entity is a pointer to a virtual storage area on the worker machine corresponding to the block ID, and the virtual storage area is physically mapped to a disk or a memory.

  Thereafter, the file write processing unit 15 sends the received block ID, the write offset within the block, the write size, and the write to the block write processing units 32 on all worker machines described in the received worker machine information. Send data and.

  Then, the block write processing unit 32 refers to the block information storage unit 34 and writes the received write data corresponding to the received write size from the received write offset with respect to the block entity of the received block ID.

  In the case of file writing across a plurality of blocks, the above process is repeated for the number of blocks to be written.

(3) Specific Edge Switch Information Acquisition Processing First, the client processing unit 11 inputs a file name corresponding to the specific edge switch information to be acquired to the specific edge switch information acquisition processing unit 14 as described above.

  Then, the specific edge switch information acquisition processing unit 14 transmits the input file name to the file information reference processing unit 22.

The file information reference processing unit 22 refers to the file / block information storage unit 23, acquires specific edge switch information corresponding to the received file name, and returns the specific edge switch information to the specific edge switch information acquisition processing unit 14. In the example illustrated in FIG. 9, when the file name for acquiring the specific edge switch information is “file 1”, the machine group “192.168.5. ^* ” Is returned as the specific edge switch information.

  The specific edge switch information processing unit 14 outputs the received specific edge switch information to the client processing unit 11.

(4) File Read Processing When the client machine does not belong to any of the machine groups output as specific edge switch information corresponding to the read file name, the client processing unit 11 responds to the read file name as will be described later. One client machine belonging to one of the machine groups output as the specific edge switch information is selected, and the client processing unit 11 on the selected client machine is instructed to read the file.

  Here, a method of selecting client machines belonging to any of the machine groups will be described.

  FIG. 11 is a diagram showing client machine information registered in the client machine information storage unit 12 shown in FIG.

  The client processing unit 11 obtains a set of client machine groups for all client machines by applying a machine group mask to the IP addresses that are all client machine information described in the client machine information as shown in FIG. To do. From the set of client machine groups, one client machine group identical to one of the machine groups output as specific edge switch information corresponding to the file name to be read is selected at random, and the selected client machine Randomly select one of the client machines that belong to the group.

When the specific edge switch information corresponding to the file name “file 1” to be read is “192.168.5. ^* ”, One client machine belonging to the machine group “192.168.5. ^* ” Is selected at random. . In the scenario shown in FIG. 5, “192.168.5.10” is selected as the client machine.

  The client processing unit 11 instructed to read the file inputs the file name to be read, the read offset, and the read size to the file read processing unit 16.

  Then, the file read processing unit 16 calculates the index of the block to be read (how many blocks are read from the beginning of the file) from the input read offset and read size, and together with the input file name, The data is transmitted to the read block information acquisition processing unit 25.

  The read block information acquisition processing unit 25 refers to the file / block information storage unit 23 based on the received file name and index, and generates the block only when the block indicated by the received file name and index has been generated. All the worker machine information in which the block ID and the replica of the block are arranged are acquired and returned to the file read processing unit 16.

  The file read processing unit 16 selects one of all the worker machines indicated by all the received worker machine information by a method described later, and receives the received block read processing unit 33 on the selected worker machine. A block ID, a read offset and a read size in the block are transmitted.

  In selecting a worker machine, a worker machine that is closest to the client machine that reads a file in terms of the network (with the smallest exclusive OR of IP addresses) is selected. As a result, a worker machine that is the same as the client machine, a worker machine that belongs to the same machine group as the client machine, and a worker machine that belongs to a machine group different from the client machine are selected in order.

  In the scenario shown in FIG. 5, since the client machine is “192.168.5.10”, “192.168.5.1” and “Block 2” are read as the “Block 1” read-out worker machines constituting “File 1”. “192.168.5.10” is selected as the worker machine, and “192.168.5.20” is selected as the reading destination worker machine of “Block 3”.

  The block read processing unit 33 refers to the block information storage unit 34, reads the block entity of the received block ID by the read size received from the received read offset, and returns the read data to the file read processing unit 16. To do.

  In the case of reading a file across a plurality of blocks, the above-described processing is repeated for the number of blocks to be read.

  The file read processing unit 16 outputs the received read data to the client processing unit 11.

  In the above-described embodiment, there are a plurality of edge switches 1-1 to 1-9 under the core switch 2 as shown in FIG. 3, and each edge switch 1-1 to 1-9 includes a plurality of edge switches 1-1 to 1-9. Although the method for arranging replicas in the system to which the machine 4 is connected has been described, the present invention may be applied to a virtual machine environment.

  FIG. 12 is a diagram showing another embodiment of the hardware configuration assumed in the distributed file management system of the present invention.

  As shown in FIG. 12, the core switch 2 shown in FIG. 3 is replaced with a switch 102, the edge switches 1-1 to 1-9 are replaced with physical machines 3-1 to 3-9, and each physical machine 3-1 to By replacing the machine 4 connected to 3-9 with the virtual machine 5, the usage mode in the virtual machine system can be obtained.

  In this way, by managing as a specific physical machine instead of a specific edge switch, when a failure occurs in one physical machine, the maximum number of replicas of a certain block increases, but each of the files that make up the file When writing a block, file writing efficiency is improved by the number of times of crossing between physical machines, and when a failure occurs in one physical machine, one replica of all the blocks that make up a file Although the probability of being lost at a stretch increases, when reading all the blocks that make up a file, the virtual machine under the specific physical machine determined for each file is used as the reader of the file, so that the file can be read within the physical machine. The file reading efficiency is improved by the amount that is closed. That is, in the system using the virtual machine shown in FIG. 12, by managing the relationship between the virtual machine and the physical machine, the same effect as the replica arrangement similar to the system configuration shown in FIG. 3 can be obtained.

  Note that the processing described above may be realized by dedicated hardware in addition to the software configuration shown in FIG. 4, and a program for realizing the function may be executed by a computer. The program may be recorded on a readable recording medium, and the program recorded on the recording medium is read by a computer and executed. The computer-readable recording medium includes an IC card, a memory card, a removable recording medium such as a floppy disk (registered trademark), a magneto-optical disk, a DVD, and a CD, and an HDD built in the computer. Point to. The program recorded on this recording medium is read by a control block, for example, and the same processing as described above is performed under the control of the control block.

1-1 to 1-9 Edge switch 2 Core switch 3-1 to 3-9 Physical machine 4 Machine 5 Virtual machine 10 Client function 10a Individual function 10b Common function 11 Client processing unit 12 Client machine information storage unit 13 File generation processing unit DESCRIPTION OF SYMBOLS 14 Specific edge switch information acquisition process part 15 File write process part 16 File read process part 20 Master function 21 File information update process part 22 File information reference process part 23 File block information storage part 24 Write block information acquisition process part 25 Read Block information acquisition processing unit 26 block replica generation processing unit 27 worker machine information storage unit 30 worker function 31 block generation processing unit 32 block write processing unit 33 block read processing unit 34 block information憶部 102 switch

Claims (3)

A distributed file management system that divides a file into a plurality of blocks and places R replicas for each of the plurality of blocks on a plurality of machines under an edge switch,
A first machine that arranges i (0 ≦ i ≦ R) replicas of the R replicas, and j (0 ≦ j ≦ (R−i)) of (R−i) replicas A second machine in which the replica is placed, and the first machine is a machine under an edge switch under the control of a client machine that writes a file for each block, and the second machine is for each file. A block replica generation processing unit that selects a machine under a specific edge switch,
A file writing processing unit that performs processing of writing a file by placing a replica on the first and second machines selected by the block replica generation processing unit;
A file read processing unit that performs file read processing as a client machine that reads a file from a machine under the specific edge switch under the second machine selected by the block replica generation processing unit; Having a distributed file management system. A distributed file placement method for dividing a file into a plurality of blocks and placing R replicas for each of the plurality of blocks on a machine under a plurality of edge switches,
A first machine that arranges i (0 ≦ i ≦ R) replicas of the R replicas, and j (0 ≦ j ≦ (R−i)) of (R−i) replicas A second machine in which the replica is placed, and the first machine is a machine under an edge switch under the control of a client machine that writes a file for each block, and the second machine is for each file. Block replica generation processing to select to be a machine under a specific edge switch,
A file writing process for writing a file by placing a replica on the first and second machines selected in the block replica generation process;
A file read process for performing a file read process as a client machine for reading a file from a machine under the specific edge switch under the second machine selected by the block replica generation process File placement method. A computer for dividing a file into a plurality of blocks and arranging R replicas for each of the plurality of blocks on machines under a plurality of edge switches,
A first machine that arranges i (0 ≦ i ≦ R) replicas of the R replicas, and j (0 ≦ j ≦ (R−i)) of (R−i) replicas A second machine in which the replica is placed, and the first machine is a machine under an edge switch under the control of a client machine that writes a file for each block, and the second machine is for each file. A block replica generation procedure to select a machine under a specific edge switch,
A file writing procedure for performing a process of writing a file by placing a replica on the first and second machines selected in the block replica generating procedure;
A file read procedure for performing a file read process as a client machine for reading a file, the machine under the specific edge switch under the second machine selected in the block replica generation procedure Program for.

Images