JP5758449B2 - Data rearrangement apparatus, method and program - Google Patents

Description

  The present invention relates to a data rearrangement apparatus, method, and program.

  In recent years, a distributed file system has been widely used. The distributed file system is composed of a large number of servers connected via a network, and realizes a huge storage device. Typically, a distributed file system is configured by connecting several tens to several hundreds of servers that extend over a plurality of racks.

  As examples of such a distributed file system, Google File System (hereinafter referred to as GFS, Non-Patent Document 1) and Hadoop Distributed File System (hereinafter referred to as HDFS, Non-Patent Document 2) are known. In these systems, a file is divided into fixed-length blocks and stored in each server constituting the distributed file system. In order to increase availability and fault tolerance, each block is duplicated and stored in a server of a predetermined redundancy number. Thereby, even if a failure occurs in a certain server, the processing can be continued using the block stored in another server. Also, when storing a block in a server, the storage destination server is determined so that availability, fault tolerance, and read / write performance are maximized.

  In a conventional distributed file system, when an external application reads data, a block is read from a server having the closest network topology to a server (also referred to as an operation server) on which the external application is operating. This suppresses the time and processing amount required for network transfer of blocks.

  However, when a distributed file system is operated for a long period of time, the storage usage between servers becomes uneven, and there is a possibility that the read / write processing load is biased. Thus, a data relocation technique has been proposed in which data is moved from a server with a large amount of data storage to a server with a small amount of data, and the storage use of the server is leveled as a whole distributed file system.

Sanjay Ghemawat, Howard Gobioff, Shun-Tak Leung, The Google File System, Proceedings of the 19th ACM Symposium on Operating Systems Principles, pages 29-43, October, 2003. Konstantin Shvachko, Hairong Kuang, Sanjay Radia, Robert Chansler, The Hadoop Distributed File System, Proceedings of the 26th IEEE Symposium on Mass Storage Systems and Technologies, pages 1-10, May, 2010.

  In the conventional data rearrangement technique, a block stored in the operation server as a result of processing may move to another server. As a result, when an external application attempts to read a block, block transfer is executed from a server other than the operation server. For this reason, the reading performance is degraded.

  The disclosed technique has been made in view of the above, and an object thereof is to provide a data rearrangement apparatus, method, and program for a distributed file system that can prevent a decrease in read performance of an external application.

  In order to solve the above-described problems and achieve the object, the data relocation apparatus, method, and program according to the embodiment divide file data written by an external application into fixed-length blocks and have a predetermined redundancy. When data is stored in a number of servers, a relocation priority indicating the priority of data relocation processing is assigned to each block stored in each server, and block data relocation processing is performed based on the relocation priority. It is characterized by performing.

  The disclosed data rearrangement apparatus, method, and program have an effect of preventing a decrease in read performance of an external application.

FIG. 1 is a diagram illustrating a configuration example of a data rearrangement system according to the first embodiment. FIG. 2 is a diagram illustrating the relationship between file data and fixed-length blocks in the first embodiment. FIG. 3 is a diagram illustrating a configuration example of the master server in the first embodiment. FIG. 4 is a diagram illustrating a configuration example of a block position management table according to the first embodiment. FIG. 5 is a diagram illustrating a configuration example of a data server state management table in the first embodiment. FIG. 6 is a diagram illustrating a configuration example of a rearrangement source block information table according to the first embodiment. FIG. 7 is a diagram illustrating a configuration example of a relocation destination data server identifier list in the first embodiment. FIG. 8 is a diagram illustrating a configuration example of the data server in the first embodiment. FIG. 9 is a diagram illustrating a configuration example of a block information table according to the first embodiment. FIG. 10 is a flowchart illustrating an example of an operation when the block location management unit of the master server receives a block generation request from an external application in the block generation processing according to the first embodiment. FIG. 11 is a flowchart illustrating an example of an operation when the data access unit of the data server receives a block generation request in the block generation process according to the first embodiment. FIG. 12 is a flowchart illustrating an example of the operation of the data rearrangement control unit of the master server in the data rearrangement execution process according to the first embodiment. FIG. 13 is a flowchart illustrating an example of an operation when the data rearrangement execution unit of the rearrangement source data server receives the block information table acquisition request in the data rearrangement execution process according to the first embodiment. FIG. 14 is a flowchart illustrating an example of an operation performed when the data relocation execution unit of the relocation destination data server receives the data relocation request in the data relocation execution process according to the first embodiment. FIG. 15 is a flowchart illustrating an example of an operation when the data access unit of the block copy source data server receives a block copy request in the data rearrangement execution process according to the first embodiment. FIG. 16 is a flowchart illustrating an example of an operation when the block location management unit of the master server receives a block registration request in the data rearrangement execution process according to the first embodiment. FIG. 17 is a flowchart illustrating an example of an operation when the data access unit of the relocation source data server receives a block deletion request in the data relocation execution process according to the first embodiment. FIG. 18 is a diagram illustrating that the information processing by the data rearrangement program, which is a program for executing a series of processes by the data rearrangement apparatus, is specifically realized using a computer.

  Hereinafter, embodiments of a disclosed data rearrangement apparatus, method, and program will be described in detail with reference to the drawings. The embodiments do not limit the disclosed technology. Moreover, each embodiment can be combined suitably.

[Block storage processing in GFS and HDFS]
First, an example of block storage processing in GFS and HDFS will be described as a premise of the embodiment.

In GFS, when deciding which server stores a block, the following processing is performed.
(1) A block is created on a server with a low storage usage rate.
(2) In order to prevent the load from being concentrated on a specific server, an upper limit is set on the number of recent block generations at each server.
(3) A block is created in a plurality of servers across racks.

  In HDFS, for example, when the redundancy is 3, the server that creates the block is selected as follows. First, the first block is created on the same server as the server on which the external application that requested writing is operating. The second block is created on an arbitrary server in a rack different from that of the first server. Further, the third block is created in an arbitrary server that is in the same rack as the second server but is different from the second server.

[Data read processing by external application]
In GFS and HDFS, when an external application reads data, in order to suppress the network transfer of the block, the block is read from the server closest in network topology to the server on which the external application is operating.

  For example, in HDFS, when an external application writes data and then reads the data, the first block of the data is stored in the same server as the server on which the external application operates. For this reason, network transfer of blocks does not occur during data reading. Therefore, reading can be performed at a higher speed than when reading a block from a server different from the server on which the external application is operating via the network.

  In this way, processing is performed so that blocks are efficiently stored and read. On the other hand, file generation and deletion, and server addition and deletion are performed by long-term operation of the distributed file system. Then, the storage usage among servers becomes uneven, and the load of reading and writing may be biased by the servers. Therefore, in the distributed file system, data rearrangement is executed in order to level the load. That is, data is relocated from a server that stores a large amount of data to a server that stores only a small amount of data, and a data relocation process is performed to level the storage usage of the server in the entire distributed file system. In GFS, a data rearrangement process is automatically executed. In HDFS, a command for executing data relocation processing is provided.

(First embodiment)
FIG. 1 is a diagram illustrating a configuration example of a data rearrangement system according to the first embodiment. In the first embodiment, a data relocation apparatus will be described as a master server and a data server in a distributed file system.

[Example of Configuration of Data Relocation System of First Embodiment]
As shown in FIG. 1, the data relocation apparatus according to the first embodiment is configured as a master file 100 constituting a distributed file system, that is, a data relocation system, and a plurality of data servers 200, 300, 400, 500, 600. Is done. In FIG. 1, five data servers are shown as an example, but the number of data servers is not limited to five. The external application 700 requests reading and writing of files.

  A data server identifier (ID: Identifier) for uniquely identifying each data server is assigned to each of the data servers 200 to 600. For example, an IP address assigned to a server on which the data servers 200 to 600 operate is used as the data server identifier. In the example of FIG. 1, data server identifiers ds2, ds3, ds4, ds5, and ds6 are assigned to the data servers 200, 300, 400, 500, and 600, respectively.

  File data written to the data server by the external application 700 is divided into fixed-length blocks, and a block identifier for uniquely identifying the block is assigned to each block. Each block is stored in a data server of a predetermined redundancy number.

  An example of file data is shown in FIG. FIG. 2 is a diagram illustrating the relationship between file data and fixed-length blocks in the first embodiment. In the example of FIG. 2, the file data 710 is divided into fixed-length blocks 711, 712, and 713. Block identifiers b1, b2, and b3 are assigned to the blocks 711, 712, and 713, respectively. If the file size is not a multiple of the fixed length, the size of the end block 713 is smaller than the fixed length. If there is a section in the file where no data exists, the block may be smaller than the fixed length.

[Example of Configuration of Master Server 100 of First Embodiment]
FIG. 3 is a diagram illustrating a configuration example of the master server 100 according to the first embodiment. The master server 100 includes a block location management unit 110, a data server state management unit 120, a rearrangement priority assignment unit 130, and a data rearrangement control unit 140. The block position management unit 110 holds a block position management table 111. The data server state management unit 120 holds a data server state management table 121. The data relocation control unit 140 includes a relocation source block information table 141 and a relocation destination data server identifier list 142. It is assumed that the relocation source block information table 141 and the relocation destination data server identifier list 142 are created during the data relocation execution process and are not stored at other times.

  Hereinafter, the details of the processing and stored information by the master server 100 will be described.

  The block position management unit 110 manages allocation of blocks to the data servers 200 to 600. The block position management unit 110 manages a block position management table 111 that stores information for managing allocation of blocks to the data servers 200 to 600.

  FIG. 4 is a diagram illustrating a configuration example of the block position management table 111 according to the first embodiment. The block position management table 111 is a list of a block identifier for uniquely identifying the block in association with each block and a data server identifier for uniquely identifying each of a plurality of data servers storing the block. A data server identifier list is maintained. For example, in the example of FIG. 4, it is indicated that the block identified by the block identifier “b1” is stored in three data servers identified by the data server identifiers “ds2,” “ds4,” and “ds5”. Has been. Further, it is indicated that the block identified by the block identifier “b2” is stored in three data servers identified by the data server identifiers “ds2”, “ds4”, and “ds6”. In addition, it is indicated that the block identified by the block identifier “b3” is stored in three data servers identified by the data server identifiers “ds2”, “ds3”, and “ds6”. Further, it is indicated that the block identified by the block identifier “b4” is stored in three data servers identified by the data server identifiers “ds2”, “ds4”, and “ds5”.

  The data server status management unit 120 manages the status of the data servers 200 to 600. The data server state management unit 120 manages a data server state management table 121 that stores information regarding the states of the data servers 200 to 600.

  FIG. 5 is a diagram illustrating a configuration example of the data server state management table 121 according to the first embodiment. The data server state management table 121 stores the maximum capacity of data that can be stored in the data storage unit (described later) of the data server and the free capacity of the data storage unit in association with the data server identifier of each data server. To do. In the example of FIG. 5, the maximum capacity of data that can be stored in the data storage unit of the data server identified by the data server identifier “ds2” is 2.0 terabytes (TB), and the current free capacity is 1.0 terabytes. (TB).

  The rearrangement priority assigning unit 130 assigns a rearrangement priority to each block. The rearrangement priority is stored in a block information table of the data server described later (FIG. 9). Details of the rearrangement priority assignment processing by the rearrangement priority assignment unit 130 will be described later.

  The data rearrangement control unit 140 selects a rearrangement source data server and a rearrangement destination data server, and blocks from the rearrangement source data server to the rearrangement destination data server based on the rearrangement priority assigned to the block. Control the relocation of

  In addition, the data relocation control unit 140 manages a relocation source block information table 141 that stores information about a block that is a target of the data relocation execution process when the data relocation execution process is executed. In addition, the data relocation control unit 140 manages a relocation destination data server identifier list 142 that is a list of relocation destination data servers that are candidates for the relocation destination of the block that is the target of the data relocation execution process. FIG. 6 is a diagram illustrating a configuration example of the relocation source block information table 141 according to the first embodiment. FIG. 7 is a diagram illustrating a configuration example of the relocation destination data server identifier list 142 according to the first embodiment. The configuration of the relocation source block information table 141 is the same as the block information table (FIG. 9) described later. Details of data relocation execution processing by the data relocation control unit 140, processing related to the relocation source block information table 141 and the relocation destination data server identifier list 142 will be described later.

[Configuration Example of Data Server of First Embodiment]
FIG. 8 is a diagram illustrating a configuration example of the data server in the first embodiment. The configurations of the data servers 200 to 600 are all the same, and the data server 200 will be described below as a representative example. In the following description, reference numerals of constituent elements of other corresponding data servers 300 to 600 are shown in parentheses as necessary.

  The data server 200 (300, 400, 500, 600) includes a data storage unit 210 (310, 410, 510, 610), a data access unit 220 (320, 420, 520, 620), and a data rearrangement execution unit 230 (330). , 430, 530, 630) and a status notification unit 240 (340, 440, 540, 640).

  The data storage unit 210 (310, 410, 510, 610) stores all blocks stored in the data server 200 (300, 400, 500, 600) and a block information table 211 (311, 411, 511) corresponding to the block. 611).

  FIG. 9 is a diagram illustrating a configuration example of the block information table 211 (311, 411, 511, 611) according to the first embodiment. The block information table 211 (311, 411, 511, 611) according to the first embodiment includes, for each block to be stored, a block identifier for uniquely identifying the block, and a data storage unit 210 (310, 410, 510, 610) and the storage location of the block in association with each other. The block information table 211 (311, 411, 511, 611) further holds, for each block, the size of the block and the rearrangement priority assigned to the block. For example, in the example of FIG. 9, it is indicated that the block identified by the block identifier “b1” is stored at the position identified by “p1” in the data storage unit 210. In addition, it is indicated that the size of the block is 64 megabytes (MB), and the reallocation priority given is “low”.

  The data access unit 220 (320, 420, 520, 620) reads blocks existing in the data storage unit 210 (310, 410, 510, 610) from the data storage unit 210 (310, 410, 510, 610). Run. Further, the data access unit 220 (320, 420, 520, 620) executes reading of information from the block information table 211 (311, 411, 511, 611). Further, the data access unit 220 (320, 420, 520, 620) writes a block to the data storage unit 210 (310, 410, 510, 610) or to the block information table 211 (311, 411, 511, 611). Write the information.

  The data rearrangement execution unit 230 (330, 430, 530, 630) executes a data rearrangement execution process related to the data server 200 (300, 400, 500, 600). Details of the data rearrangement execution processing by the data rearrangement execution unit 230 (330, 430, 530, 630) will be described later.

  The status notification unit 240 (340, 440, 540, 640) monitors the status of the data storage unit 210 (310, 410, 510, 610) and notifies the master server 100 of the status. Specifically, the status notification unit 240 (340, 440, 540, 640) includes the maximum capacity that can be stored in the data storage unit 210 (310, 410, 510, 610) and the data storage unit 210 (310, 410, 510). , 610) free space is periodically detected. Then, the status notification unit 240 (340, 440, 540, 640) periodically notifies the detected maximum capacity and free capacity to the data server status management unit 120 of the master server 100. Here, the state is periodically detected and notified, but detection and notification may be executed in response to the occurrence of some event or in response to an operator input or the like.

[Example of Flow of Data Relocation Processing in First Embodiment]
Data relocation processing by the data relocation apparatus according to the first embodiment will be described. The data rearrangement process in the first embodiment includes a rearrangement priority assignment process and a data rearrangement execution process.

  The rearrangement priority assigning process is a process for assigning a rearrangement priority indicating the priority of the data rearrangement process for each block to each block stored in the data server. In the example described below, the rearrangement priority assignment process is executed as part of the block generation process. The block generation processing includes block registration by the block position management unit 110 of the master server 100, assignment of rearrangement priority to the block by the rearrangement priority assignment unit 130, and block generation processing by the data server. Including.

  The data rearrangement execution process is a process for executing the rearrangement of the block based on the rearrangement priority assigned to each block. In the example described below, the data rearrangement execution process includes the data rearrangement control by the data rearrangement control unit 140 of the master server 100 and the data rearrangement execution unit of the data server 200 (300, 400, 500, 600). 230 (330, 430, 530, 630).

[Block generation process flow]
The block generation process will be described in more detail.

[Processing in Master Server 100]
FIG. 10 is a flowchart illustrating an example of an operation when the block position management unit 110 of the master server 100 receives a block generation request from the external application 700 in the block generation processing according to the first embodiment. First, processing executed by the master server 100 in the block generation processing will be described with reference to FIG.

  In the block generation process, first, the external application 700 transmits a block generation request to the block position management unit 110 of the master server 100 when writing file data. When receiving the block generation request, the block position management unit 110 generates blocks in the data servers having the redundancy number, and the block position management table 111 stores the block identifier of the block and the data server identifier list of the data server storing the block. Register with.

  The operation when the block location management unit 110 of the master server 100 receives a block generation request from the external application 700 in the block generation process will be described below with reference to FIG.

  First, the external application 700 transmits a block generation request to the master server 100. When the master server 100 receives the block generation request (step S1201), the master server 100 sends the block generation request to the block location management unit 110.

  Then, the block location management unit 110 acquires the free capacity of the data storage unit in each data server from the data server state management table 121 of the data server state management unit 120. Then, the block location management unit 110 selects a data server having a predetermined redundancy number from data servers having a free space in the data storage unit equal to or greater than a predetermined threshold (step S1202).

  The block location management unit 110 selects a server as a block storage destination based on the free capacity of each data server and the rack in which the data server is placed. That is, the block location management unit 110 selects a data server whose free space in the data storage unit is equal to or greater than a predetermined threshold. In addition, the block location management unit 110 selects a data server so that the data server storing the block spans a plurality of racks.

  For example, assume that the predetermined redundancy is 3, and the blocks are stored in data servers that operate on servers arranged in different racks. In this case, the block location management unit 110 selects a data server operating on the same server as the server on which the external application 700 that transmitted the block generation request is operating as the first data server. Then, the block location management unit 110 selects a data server operating on a server in a rack different from that of the first data server as the second data server. Furthermore, the block location management unit 110 selects a data server operating as a third data server that is operating on servers in different racks from the first rack and the second rack.

  Then, the block position management unit 110 generates a data server identifier list of the data server that stores the block. Note that the block location management unit 110 transmits a block generation failure to the external application 700 when a data server having a predetermined redundancy number cannot be selected due to insufficient free space or the like.

  Next, the block position management unit 110 generates a block identifier of the block to be stored (step S1203), and instructs the rearrangement priority assignment unit 130 to calculate rearrangement priority.

  The rearrangement priority assigning unit 130 calculates rearrangement priority according to the instruction (step S1204). For example, the relocation priority assigning unit 130 calculates a low relocation priority for a block to be held in a data server operating on the same server as the server on which the external application that transmitted the block generation request is operating. Give. Further, the rearrangement priority assigning unit 130 calculates and assigns a higher rearrangement priority to the blocks held by the data servers operating on other servers.

  When the calculation of the rearrangement priority is completed, the master server 100 transmits a block generation request to the data access unit of the data server indicated by each data server identifier in the data server identifier list generated by the block location management unit 110 (step S1205). At this time, the master server 100 transmits the block identifier generated in step S1203 and the rearrangement priority calculated in step S1204 together with the block generation request.

  As described above, in the first embodiment, the IP address of the data server is used as the data server identifier. Therefore, the master server 100 can access the data access unit of the data server indicated by the IP address based on the data server identifier. However, when information other than the IP address is used as the data server identifier, a mechanism for associating the data server identifier with the IP address is prepared separately.

  After transmitting a block generation request to the corresponding data server, the master server 100 receives a block generation response from each data server (step S1206). When the block generation response is received, the master server 100 registers a block identifier for identifying the generated block and a data server identifier list indicating a plurality of data servers that have generated and stored the block in the block location management table 111 ( Step S1207). Then, the master server 100 transmits to the external application 700 a block identifier for identifying the generated block and a data server identifier list indicating a plurality of data servers that have generated and stored the block (step S1208).

  When the external application 700 receives the block identifier and the data server identifier list, the external application 700 writes the data to the block identified by the block identifier held by the data server in the data server identifier list. This completes the processing in the master server 100.

[Processing in Data Server 200]
Next, processing on the data server side in the block generation processing will be described. FIG. 11 is a flowchart illustrating an example of an operation when the data access unit of the data server receives a block generation request in the block generation process according to the first embodiment. Here, it is assumed that the data server 200 has received a block generation request.

  First, the data server 200 receives a block generation request, a block identifier, and a rearrangement priority from the block position management unit 110 of the master server 100 (step S1301). Then, the data access unit 220 generates an empty block and stores it in the data storage unit 210 (step S1302). The data access unit 220 also registers the block identifier, rearrangement priority, and block size (0 at this time) received from the master server 100 in the block information table 211 (step S1302).

  Then, the data server 200 transmits a block generation response to the block position management unit 110 of the master server 100 (step S1303), and ends the process.

[Flow of data relocation execution processing]
Next, the data rearrangement execution process will be described. In the data rearrangement apparatus of the first embodiment, the data rearrangement execution process is periodically performed.

[Processing in Master Server 100]
FIG. 12 is a flowchart illustrating an example of the operation of the data relocation control unit 140 of the master server 100 in the data relocation execution process according to the first embodiment. Processing in the master server 100 will be described with reference to FIG.

  First, the data relocation control unit 140 of the master server 100 determines whether data relocation execution processing is necessary (step S1401). If the data rearrangement control unit 140 determines that the data rearrangement execution process is necessary (Yes at step S1401), the data rearrangement control unit 140 proceeds to step S1402. On the other hand, when the data rearrangement control unit 140 determines that the data rearrangement execution process is not necessary (No in step S1401), the data rearrangement control unit 140 ends the process.

  For example, the data relocation control unit 140 acquires the data server state management table 121 from the data server state management unit 120, and when the following conditions (1) and (2) are satisfied, data relocation execution processing is necessary. Is determined.

(1) The value obtained by summing the free capacity of the data storage unit in each data server for all data servers is larger than the product of a predetermined fixed capacity and the number of data servers.
(2) The usage rate of the data storage unit in each data server is calculated based on the value of the data server state management table 121, and 1 data server has a usage rate of the data storage unit equal to or greater than a predetermined threshold. There are one or more data servers in which there is at least one and the free capacity of the data storage unit is larger than the sum of the free capacity of the data storage unit of the data server and a predetermined free capacity margin.

  When it is determined that the data rearrangement execution process is necessary (Yes at Step S1401), the data rearrangement control unit 140 selects a rearrangement source data server (Step S1402). For example, the data relocation control unit 140 acquires the data server state management table 121 from the data server state management unit 120. Then, the data relocation control unit 140 calculates the usage rate of the data storage unit in each data server based on the value of the data server state management table 121. The data rearrangement control unit 140 randomly selects one data server from among data servers whose calculated usage rate is equal to or greater than a predetermined threshold. Then, the data rearrangement control unit 140 sets the selected data server as the rearrangement source data server. As an example, here, it is assumed that the data server 200 is selected as the relocation source data server.

  Next, the data relocation control unit 140 sets a maximum relocation amount and initializes the cumulative relocation amount to zero (step S1403). Here, the maximum rearrangement amount is a data amount to be rearranged by a single data rearrangement execution process. The maximum rearrangement amount may be determined in advance, or may be determined every time the data rearrangement execution process is executed. The cumulative rearrangement amount refers to the amount of data that has been rearranged so far in a single data rearrangement execution process.

  Then, the data rearrangement control unit 140 transmits a block information table acquisition request to the data rearrangement execution unit 230 of the rearrangement source data server 200 (step S1404). The data rearrangement control unit 140 receives the block information table 211 from the data rearrangement execution unit 230 of the rearrangement source data server 200 (step S1405).

  The data rearrangement control unit 140 sorts the entries in the block information table 211 of the rearrangement source data server 200 in descending order of rearrangement priority to obtain the rearrangement source block information table 141 (FIG. 6) (step S1406). .

  Next, the data relocation control unit 140 determines whether or not the processing of the last entry in the relocation source block information table 141 has been completed (step S1407). If it is determined that the processing of the last entry has been completed (Yes at step S1407), the data rearrangement control unit 140 ends the processing. When the data rearrangement control unit 140 determines that the process of the last entry has not been completed (No at Step S1407), the process proceeds to Step S1408. The data rearrangement control unit 140 executes data rearrangement processing in order from the block set in the head entry of the rearrangement source block information table 141.

  The data relocation control unit 140 creates a relocation destination data server identifier list 142 (FIG. 7) for the block of the current entry in the relocation source block information table 141, and determines whether or not the list is empty (Step S1). S1408).

  When the data relocation control unit 140 determines that the relocation destination data server identifier list 142 is empty (Yes in step S1408), the process transitions to step S1414.

  On the other hand, when the data relocation control unit 140 determines that the relocation destination data server identifier list 142 is not empty (No at Step S1408), the process transitions to Step S1409.

  The relocation destination data server identifier list 142 is a list of relocation destination data servers that can be relocation destinations of a predetermined block. For example, the data relocation control unit 140 creates the relocation destination data server identifier list 142 according to the following conditions.

  For example, assume that the blocks are stored in data servers that each run on a separate rack server. In this case, a data server that satisfies all of the following conditions is registered in the relocation destination data server identifier list 142 for the block that is the target of the data relocation execution process.

(1) The data server is a data server that does not hold a block that is a target of data relocation execution processing.
(2) The free capacity of the data storage unit of the data server is larger than the sum of the free capacity of the data storage unit of the relocation source data server and a predetermined free capacity margin.
(3) The rack of servers on which the data server is operating does not include the servers on which the data servers other than the relocation source data server are operating among the data servers holding the block.

  In step S1409, the data relocation control unit 140 selects a relocation destination data server from data servers whose data server identifiers are registered in the relocation destination data server identifier list 142.

  For example, the data relocation control unit 140 sets the data server having the lowest relocation cost from the data servers in the relocation destination data server identifier list 142 as the relocation destination data server. The rearrangement cost is a parameter indicating the processing load required for the data rearrangement execution process.

For example, the rearrangement cost is calculated as follows. Assume that the number of data servers indicated by the data server identifiers in the relocation destination data server identifier list 142 is two or more. At this time, the relocation cost Cds of the data server ds is calculated by the following equation.
Cds = Fds + W × Dds
Here, Fds is a free capacity cost, Dds is a distance cost, and W is a predetermined weighting factor.

The free capacity cost Fds is calculated by the following equation.
Fds = (Fmax−fds + 1) / (Fmax−Fmin + 1)
Here, Fmax and Fmin are the maximum value and the minimum value of the free capacity of all data servers, respectively, and fds is the free capacity of the data server.

The distance cost Dds is calculated by the following equation.
Dds = (1.0−cos (π × i / (n−1))) / 2
Here, n is the number of data servers in the relocation destination data server identifier list 142, i is the data server indicated by each data server identifier in the relocation destination data server identifier list 142 using the distance from the relocation source data server. The order starts from 0 when sorted in ascending order. The distance between the two data servers is a value obtained by adding 1 to the natural logarithm of the exclusive OR of the IP addresses assigned to the two data servers.

  As an example, here, it is assumed that the data server 300 is selected as the relocation destination data server.

  Next, the data relocation control unit 140 determines whether the sum of the cumulative relocation amount and the block size of the current entry is equal to or smaller than the maximum relocation amount (step S1410). If the data rearrangement control unit 140 determines that the total is larger than the maximum rearrangement amount (No in step S1410), the process is terminated. When it is determined that the total is equal to or less than the maximum arrangement amount (Yes at Step S1410), the data rearrangement control unit 140 proceeds to the process at Step S1411.

  In step S1411, the data rearrangement control unit 140 adds the block size of the current entry to the cumulative rearrangement amount. Then, the data rearrangement control unit 140 acquires the data server identifier list of the data server storing the block to which the block identifier of the current entry is assigned from the block location management table 111 of the block location management unit 110, The block copy source data server identifier list is set (step S1412).

  Then, the data relocation control unit 140 sends a data relocation request to the data relocation execution unit 330 of the relocation destination data server 300, the block identifier of the current entry, the relocation source data server identifier, and the block copy source data server identifier list. Is transmitted (step S1413). Next, the data relocation control unit 140 proceeds to the processing of the next entry in the relocation source block information table 141 (step S1414). Then, the process returns to step S1407. If the process of the last entry is not completed (No at step S1407), the data rearrangement control unit 140 repeats the process from step S1408 to step S1414. When the process of the last entry is completed (Yes at step S1407), the process ends.

[Processing in Relocation Source Data Server 200]
FIG. 13 is a flowchart illustrating an example of an operation when the data rearrangement execution unit 230 of the rearrangement source data server 200 in the data rearrangement execution process according to the first embodiment receives a block information table acquisition request. With reference to FIG. 13, an example of an operation when the data rearrangement execution unit 230 of the rearrangement source data server 200 in the data rearrangement execution process receives a block information table acquisition request will be described.

  First, the data relocation execution unit 230 of the relocation source data server 200 receives a block information table acquisition request from the data relocation control unit 140 of the master server 100 (see step S1501, steps S1404 and S1405 in FIG. 12).

  In response to the block information table acquisition request, the data rearrangement execution unit 230 instructs the data access unit 220 to acquire the block information table 211 from the data storage unit 210 (step S1502). Then, the data rearrangement execution unit 230 transmits the block information table 211 to the data rearrangement control unit 140 of the master server 100 (step S1503), and ends the process.

[Processing in Relocation Destination Data Server 300]
FIG. 14 is a flowchart illustrating an example of an operation when the data rearrangement execution unit 330 of the rearrangement destination data server 300 receives a data rearrangement request in the data rearrangement execution process according to the first embodiment. With reference to FIG. 14, an example of an operation when the data rearrangement execution unit 330 of the rearrangement destination data server 300 receives a data rearrangement request in the data rearrangement execution process will be described.

  First, the data relocation execution unit 330 of the relocation destination data server 300 receives a data relocation request from the data relocation control unit 140 of the master server 100, the block identifier of the block to be relocated, and the relocation source data server identifier. The block copy source data server identifier list is received (see step S1601, step S1413 in FIG. 12).

  The data rearrangement execution unit 330 selects a block copy source data server, that is, a data server as a block copy source, from the data servers included in the received block copy source data server identifier list (step S1602). For example, the data server closest to the relocation destination data server 300 is set as the block copy source data server.

  Here, for example, the distance between two data servers is a value obtained by adding 1 to the natural logarithm of the exclusive OR of the IP addresses assigned to the two data servers. The block copy source data server is not necessarily the same as the relocation source data server. As an example, here, it is assumed that the data server 400 is selected as the block copy source data server. Thus, by making the distance between the block copy source data server and the relocation destination data server the shortest, efficient block copy can be realized and the processing speed can be increased. However, the block copy source data server can be selected based on another criterion.

  The data rearrangement execution unit 330 transmits the block copy request and the block identifier of the block to be rearranged to the data access unit 420 of the block copy source data server 400 (step S1603). The data rearrangement execution unit 330 receives a block from the data access unit 420 of the block copy source data server 400 (step S1604). The data rearrangement execution unit 330 instructs the data access unit 320 to store the received block in the block storage unit 310 (step S1605). Also, the data rearrangement execution unit 330 instructs the data access unit 320 to add an entry corresponding to the stored block to the block information table 311 and update the block information table 311 (step S1605). The data rearrangement execution unit 330 registers the received block size as the size of the block, and sets the rearrangement priority to “high”.

  The data rearrangement execution unit 330 transmits a block registration request, the block identifier of the stored block, and the data server identifier of the rearrangement source data server 200 to the block position management unit 110 of the master server 100 (step S1606). This completes the processing in the relocation destination data server 300.

[Processing in Block Copy Source Data Server 400]
FIG. 15 is a flowchart illustrating an example of an operation when the data access unit 420 of the block copy source data server 400 receives a block copy request in the data rearrangement execution process according to the first embodiment. With reference to FIG. 15, an example of an operation when the data access unit 420 of the block copy source data server 400 in the data relocation execution process receives a block copy request will be described.

  First, the data access unit 420 of the block copy source data server 400 receives a block copy request and a block identifier from the data relocation execution unit 330 of the relocation destination data server 300 (see step S1701 and step S1603 of FIG. 14). Then, the data access unit 420 reads the received block identifier from the data storage unit 410 (step S1702). The data access unit 420 transmits the read block to the data rearrangement execution unit 330 of the rearrangement destination data server 300 (step S1703). This completes the processing in the block copy source data server 400.

[Processing for Block Registration Request in Master Server 100]
FIG. 16 is a flowchart illustrating an example of an operation when the block location management unit 110 of the master server 100 receives a block registration request in the data rearrangement execution process according to the first embodiment. With reference to FIG. 16, an example of an operation when the block location management unit 110 of the master server 100 receives a block registration request in the data rearrangement execution process will be described.

  First, the block location management unit 110 of the master server 100 receives a block registration request, a block identifier, and a rearrangement source data server identifier from the data rearrangement execution unit 330 of the rearrangement destination data server 300 (step S1801, FIG. 14). (See step S1606). The block location management unit 110 adds the data server identifier of the relocation destination data server 300 to the data server identifier list associated with the received block identifier in the block location management table 111 (step S1802). Further, the block location management unit 110 deletes the data server identifier of the relocation source data server 200 from the data server identifier list (step S1802).

  Then, the block position management unit 110 transmits the block deletion request and the block identifier received from the relocation destination data server 300 to the data access unit 220 of the relocation source data server 200 (step S1803). The block location management unit 110 receives a block deletion response from the data access unit 220 of the relocation source data server 200 (step S1804). Thereby, the process in the master server 100 corresponding to the block registration request is completed.

[Processing for block deletion request at the relocation source data server]
FIG. 17 is a flowchart illustrating an example of an operation when the data access unit 220 of the relocation source data server 200 receives a block deletion request in the data relocation execution process in the first embodiment. With reference to FIG. 17, an example of an operation when the data access unit 220 of the relocation source data server 200 receives a block deletion request in the data relocation execution process will be described.

  First, the data access unit 220 of the relocation source data server 200 receives a block deletion request and a block identifier from the data relocation control unit 140 of the master server 100 (see step S1901 and step S1803 of FIG. 16). Then, the data access unit 220 deletes the entry corresponding to the received block identifier from the block information table 211 (step S1902). In addition, the data access unit 220 deletes the received block with the block identifier from the block storage unit 210 (step S1902). Then, the data access unit 220 transmits a block deletion response to the data rearrangement control unit 140 of the master server 100 (step S1903). As a result, the processing corresponding to the block deletion request in the relocation source data server 200 ends.

[Effect of the first embodiment]
As described above, the data rearrangement device according to the first exemplary embodiment allows each server to store the file data written by the external application into fixed-length blocks and store them in a predetermined number of redundancy servers. For each block to be stored, a rearrangement priority assigning unit that assigns a rearrangement priority indicating the priority of the data rearrangement process, and a data rearrangement process that executes block data rearrangement processing based on the rearrangement priority An arrangement execution unit. For this reason, it is possible to suppress the movement of the blocks whose rearrangement is not preferable based on the rearrangement priority, and to prevent the reading performance of the external application from being deteriorated.

  Further, according to the data rearrangement device 1 according to the first embodiment, the rearrangement priority assigning unit operates the external application among the blocks obtained by dividing the file data written by the external application into a fixed length. A low relocation priority is assigned to a block held by the same server as the server to be executed, and a high relocation priority is assigned to a block held by a server different from the server on which the external application operates. Then, the data rearrangement execution unit executes the data rearrangement process from the block having the higher rearrangement priority.

  For this reason, the block stored in the same server as the server on which the external application operates is likely to remain in the same server even if the data rearrangement process is performed. Therefore, when an external application reads data, the probability of reading a block from the same server as the server on which it operates is higher than the probability of reading a block from another server. Reading by an external application can be speeded up as compared with the case where the arrangement process is executed.

(Second Embodiment)
Although the embodiments of the present invention have been described so far, the present invention may be implemented in other embodiments besides the above-described embodiments. Other embodiments will be described below.

[Application to other distributed file systems]
The above embodiment has been described on the premise of processing in a distributed file system including a master server that manages the position of a block and a data server that holds the block, such as GFS and HDFS. However, the present invention is not limited to this, and can be applied to any storage system or distributed file system as long as the data storage location is not limited to a storage system or distributed file system.

[System configuration]
Also, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually can be performed. All or a part can be automatically performed by a known method. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or a part of the distribution / integration may be functionally or physically distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. For example, in the example shown in FIG. 9, the relocation priority is described as being stored in the block information table 211 (311, 411, 511, 611), but the relocation priority is managed on the master server 100 side. It may be configured.

[program]
FIG. 18 is a diagram illustrating that the information processing by the data rearrangement program, which is a program for executing a series of processes by the data rearrangement apparatus, is specifically realized using a computer. As illustrated in FIG. 18, the computer 1000 includes, for example, a memory 1010, a CPU (Central Processing Unit) 1020, a hard disk drive 1080, and a network interface 1070. Each part of the computer 1000 is connected by a bus 1100.

  The memory 1010 includes a ROM 1011 and a RAM 1012 as illustrated in FIG. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System).

  Here, as illustrated in FIG. 18, the hard disk drive 1080 stores, for example, an OS 1081, an application program 1082, a program module 1083, and program data 1084. That is, the data rearrangement program according to the disclosed technique is stored in, for example, the hard disk drive 1080 as the program module 1083 in which an instruction to be executed by the computer is described. For example, the hard disk drive 1080 stores a program module 1083 in which each procedure for executing the same information processing as each unit of the master server 100 and the data servers 200 to 600 is described.

  Further, data used for information processing by the data rearrangement program, such as data stored in the master server 100 and the data servers 200 to 600, is stored as, for example, the hard disk drive 1080 as the program data 1084. Then, the CPU 1020 reads the program module 1083 and program data 1084 stored in the hard disk drive 1080 to the RAM 1012 as necessary, and executes various procedures.

  Note that the program module 1083 and the program data 1084 related to the data rearrangement program are not limited to being stored in the hard disk drive 1080. For example, the program module 1083 and the program data 1084 may be stored in a removable storage medium. In this case, the CPU 1020 reads data via a removable storage medium such as a disk drive. Similarly, the program module 1083 and program data 1084 related to the update program may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). . In this case, the CPU 1020 reads various data by accessing another computer via the network interface 1070.

[Others]
The data rearrangement program described in this embodiment can be distributed via a network such as the Internet. The data rearrangement program can also be executed by being recorded on a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, an MO, and a DVD, and being read from the recording medium by the computer. .

DESCRIPTION OF SYMBOLS 100 Master server 110 Block position management part 111 Block position management table 120 Data server state management part 121 Data server state management table 130 Relocation priority assignment part 140 Data relocation control part 141 Relocation source block information table 142 Relocation destination data Server identifier list 200, 300, 400, 500, 600 Data server 210, 310, 410, 510, 610 Data storage unit 211, 311, 411, 511, 611 Block information table 220, 320, 420, 520, 620 Data access unit 230, 330, 430, 530, 630 Data relocation execution unit 240, 340, 440, 540, 640 Status notification unit 700 External application 710 File data 711, 712, 713 Click 1000 computer 1010 memory 1011 ROM
1012 RAM
1020 CPU
1070 Network interface 1080 Hard disk drive 1081 OS
1082 Application program 1083 Program module 1084 Program data 1100 Bus

Claims (3)

When the file data written by the external application is divided into fixed-length blocks and stored in a predetermined number of redundant servers , the block written by the external application is stored for each block stored in each server. If the server and the server running the external application are the same, move the block from the server than when the server storing the block written by the external application is different from the server running the external application. A rearrangement priority assigning unit that assigns a rearrangement priority indicating the priority of the data rearrangement process by setting the rearrangement priority indicating the priority to be low ,
Based on at least the free capacity and the usage rate of the data storage unit of each server, it is determined whether or not to execute a data rearrangement process for moving a block stored in each server, and it is determined to execute the data rearrangement process In this case, each block stored in the server that is the rearrangement source of the data rearrangement process is rearranged in descending order of the rearrangement priority, and the data amount that has completed the data rearrangement process becomes a predetermined data amount. A data rearrangement control unit that terminates the data rearrangement process,
A data rearrangement device comprising: When the file data written by the external application is divided into fixed-length blocks and stored in a predetermined number of redundant servers , the block written by the external application is stored for each block stored in each server. If the server and the server running the external application are the same, move the block from the server than when the server storing the block written by the external application is different from the server running the external application. A rearrangement priority assigning step for assigning a rearrangement priority indicating the priority of the data rearrangement process by setting the rearrangement priority indicating the priority to be low ;
Based on at least the free capacity and the usage rate of the data storage unit of each server, it is determined whether or not to execute a data rearrangement process for moving a block stored in each server, and it is determined to execute the data rearrangement process In this case, each block stored in the server that is the rearrangement source of the data rearrangement process is rearranged in descending order of the rearrangement priority, and the data amount that has completed the data rearrangement process becomes a predetermined data amount. A data rearrangement control step for ending the data rearrangement process upon reaching,
A data rearrangement method comprising: A data rearrangement program for causing a computer to function as the data rearrangement device according to claim 1 .

Images