KR101371202B1 - Distributed file system having multi MDS architecture and method for processing data using the same - Google Patents

Abstract

The present invention is a distributed file system having a multi-metadata server structure capable of increasing scalability and availability of metadata by configuring a plurality of metadata servers (MDS) even when intensive access is made from a plurality of clients. And at least two or more data servers for storing and managing file data, and periodically reporting its status information to the metadata server, and at least two or more DHTs. A metadata server for storing metadata for all files of a file system and managing state information for the data server based on the state information provided from the data server, wherein each metadata is disposed by applying a distributed hash table) concept; The metadata server while running a client application Accessing the file from the metadata and may consists of, including the file system client for inputting and outputting file data from the data server.

Description

Distributed file system having multi MDS architecture and method for processing data using the same}

The present invention relates to a distributed file system that distributes access to data in which intensive access is made from clients. In particular, a plurality of metadata servers (MDS) can be configured to increase scalability and availability of metadata. The present invention relates to a distributed file system having a multi-metadata server structure and a data processing method using the same.

Most of the data stored in a conventional storage environment is business-related data generated by a corporation or an institution. However, the rapid development of Internet technology has rapidly increased the storage ratio of multimedia data such as blogs, photographs, and moving images . In particular, large portal companies that provide Internet services at home and abroad are newly generating, storing and managing tens of terabytes of data every few terabytes (TB). However, the existing storage structure environment has many problems in storage scalability and ease of management, so it is not enough to replace the variable service environment.

Thus, recent technological advances in storage systems or file systems are due to improvements in scalability and performance of storage systems. In particular, some systems in the file system architecture have improved the scalability and performance of the distributed storage system by separating the data input / output path of the file and the metadata management path of the file. By applying this structure, the file system client can directly access the storage devices, and by distributing the metadata, the bottleneck caused by frequent file metadata access is eliminated to increase the storage scalability.

Enterprise-class storage solutions based on this architecture include IBM's StorageTank, Panasas's ActiveScale Storage Cluster, Cluster filesystems luster, and Google's Google Filesystem. In particular, Google Filesystem replicated block data for one file to multiple data servers to make it more usable.

In such a network-based distributed file system environment, file system clients, metadata servers (MDSs), and data servers (Data Servers) communicate with each other over a network to provide input and output of data. In order to access a specific file, the file system client obtains the location information of the block in which the actual data of the file is stored from the metadata server (MDS), accesses the data server where the block is located, and reads the block data.

1 is a configuration diagram schematically illustrating a configuration of a general distributed file system.

As shown in FIG. 1, the distributed file system includes a file system client 10 that accesses file metadata from the metadata server 20 and inputs and outputs file data from the data server 30 while executing a client application. The metadata server 20 stores and manages metadata of all files of the system, manages state information of all data servers 30, and stores and manages file data, and periodically the metadata server 20. Includes a data server 30 for reporting its status information. In this case, the file system client 10, the metadata server 20, and the data server 30 are interconnected to the network 40.

In the distributed file system having such a configuration, file data is divided into chunk units having a predetermined size and distributed and stored in the data servers 30.

Then, the metadata server 20 selects the data server 30 in which file data is to be generated, and selects data servers 30 to store the replica. In this case, the file data is distinguished by a unique identifier, and the identifier information on the generated file data is stored in the metadata server 20 as metadata information of the file. In addition, when a user's request to write a new file arrives, an appropriate data server 30 is selected based on the state information reported by the data server 30.

In addition, the metadata server 20 manages the state of the data server 30 and, if a problem occurs, changes the data server 30 in which the problem occurs to an unusable state. Therefore, when a user's request to write a file comes in, he selects the currently available data server 30.

Phil Schwan's “Lustre: Building a file system for 1000-node clusters”, presented at the Linux Symposium 2003 in July 2003, allows a client to map a particular data server among multiple data servers for a given file or directory path. We proposed a method using a metadata server and a hash function. This means that if the hash function for mapping evenly distributes the client's request to multiple data servers, the client can directly access the corresponding data server by mapping the values obtained through the hash function. This has the effect of distributing the load evenly when multiple clients select data servers.

However, in the case of such a conventional distributed file system, as the single metadata server 20 processes the request of the multiple file system clients 10, metadata related to the single metadata server 20 may be processed. The number of operations is limited. Therefore, the configuration of a single metadata server 20 has a problem that the scalability and availability for the metadata is not good.

In order to solve this problem, various studies have recently been conducted to configure a plurality of metadata servers to improve such metadata processing performance. In particular, the configuration of the plurality of metadata server 20 has a great advantage in that it has the ability to cope with the failure and improve the processing performance, the introduction accordingly is more necessary.

The present invention has been made to solve the above problems, even when intensive access from a plurality of clients to configure a plurality of metadata server (Metadata Server: MDS) to increase the scalability and availability of metadata An object of the present invention is to provide a distributed file system having a multi-metadata server structure and a data processing method using the same.

Another object of the present invention is to apply the concept of Distributed Hash Table (DHT) to the arrangement of a plurality of metadata servers, and to define one of the values present in the hash space as a token value of the metadata servers. It is to provide a distributed file system and a data processing method using the same that can effectively manage the status information for all data servers even in the event of a failure.

A feature of a distributed file system having a multi-metadata server structure according to the present invention for achieving the above object is at least two or more that stores and manages file data, and periodically reports its status information to the metadata server. It is composed of a data server and at least two, and each is arranged by applying the concept of a distributed hash table (DHT), and stores metadata about all files of the file system, and based on the status information provided from the data server. It includes a metadata server that manages state information about the server, and a file system client that accesses file metadata from the metadata server and inputs and outputs file data from the data server while executing a client application.

Preferably, the data server divides the input original file into at least one piece of data, generates one or more pieces of divided data each of which combines at least two or more parity data with the divided pieces of original file data, and then generates each of the divided pieces. The data is divided and stored again.

Preferably, the generated divided data is generated with a codeword size of local storage including parity data through encoding.

Preferably, the metadata server defines one of the values existing in the hash space as a token value, and the data server is located in the existing data server along the hash ring in a clockwise or counterclockwise direction. It manages state information about the server and defines its own management area until the point where it meets another metadata server.

Preferably, the metadata server may include a token generation function configured to generate a token value and set the metadata server so that the metadata server is located in the entire hash space using a hash algorithm.

Preferably, the hash space is configured as a partition map including a token value of the metadata server and an access address of the metadata server.

A feature of the data processing method using a distributed file system having a multi-metadata server structure according to the present invention for achieving the above object is to store at least two or more data servers, metadata about a file, A data processing method using a distributed file system, comprising: a metadata server managing state information of a server; and a file system client configured to perform input and output of file data from a data server while executing a client application. More than two are arranged using the concept of Distributed Hash Table (DHT), and one of the values existing in the hash space is defined as a token value.

Preferably, the metadata server defines its own management area up to the point where it encounters another metadata server, and provides status information on the existing data server along a hash ring in a clockwise or counterclockwise direction. It is characterized by managing.

Preferably, the state information reported by the data server is delivered to only one metadata server, and the metadata server that has received the state information shares the reported state information with another metadata server in a hash ring. It is characterized by including.

Preferably, the metadata server sets a token value generated in the metadata server so that the metadata server is located in the entire hash space using a hash algorithm, token and server connection address information of the metadata server. Publishing a partition map having a system shared information storage unit (iLock) or Gossip protocol, and detecting a metadata server in a hash space by a data server or a file system client based on the published partition map; and receiving status information (repoinfo) of the data server including the disk status information managed by the data server from the data server where a register is made and keepalive is maintained.

Preferably, after the second metadata server MDS # 2, which is newly added to the hash ring, reads the partitioning map, recording its token and server connection address information determined through a hash algorithm in the partitioning map. And a right secured by the second metadata server MDS # 2 by the first metadata server MDS # 1, which has already taken part or all of the management rights on the hash ring, which is naturally defined as a result of this procedure. Abandoning an area, detecting a metadata server in a hash space by a data server or file system client whose management authority has changed based on the partitioning map, registers, and keepalives are maintained Receiving status information (repoinfo) of a data server including disk status information managed by the data server from the data server And the like.

Preferably, upon receiving a stop request of an existing metadata server, the requested stop metadata server deletes its token from the partition map and publishes and terminates it using the system shared information storage unit (iLock) or Gossip protocol. And a step in which the neighboring metadata server in the hash ring increases its area of responsibility to the hash area of the deleted metadata server.

Preferably, when the fourth metadata server of the metadata server located in the hash space cannot be operated normally due to an unexpected situation, the system shared information storage unit (iLock) or the Gossip protocol turns out to be a failure. Revealing that the token value of the fourth metadata server has been removed and subjecting the monitored object to another neighboring third metadata server that is monitoring the fourth metadata server determined as the failure; And changing to a fifth metadata server monitored by the server.

Preferably, the metadata server continuously monitors a metadata server corresponding to another neighboring node in a hash ring structure.

As described above, a distributed file system having a multi-metadata server structure and a data processing method using the same have the following effects.

First, by configuring a plurality of metadata servers, the client selects an appropriate metadata server in consideration of the memory cache efficiency of the metadata server, thereby improving metadata processing performance and response performance of the metadata server.

Second, by configuring the deployment of multiple metadata servers by applying the concept of Distributed Hash Table (DHT), metadata servers can be easily added so that they can actively cope with the increase in service capacity and consequently receive more requests from clients. You can do it faster.

Third, the client prevents access to the failed metadata server and can easily replace the access to the failed metadata server with other metadata servers, effectively preventing all the metadata servers from occurring. You can manage the status information for the data server.

1 is a configuration diagram schematically showing the configuration of a general distributed file system
2 is a diagram illustrating a distributed file system having a multi-metadata server structure according to an embodiment of the present invention.
3 is a view for explaining a distributed storage method of data according to an embodiment of the present invention.
4 is a view for explaining a layout structure of several metadata servers in the distributed file system of the present invention.
5 is a diagram illustrating a process of detecting a metadata server in charge of a corresponding area when a data server and a client are located in a hash space in the distributed file system of the present invention.
FIG. 6 is a diagram illustrating a hash space represented by a circle illustrated in FIG. 4 in a map form. FIG.
7 is a diagram illustrating a process of adding a new metadata server in the distributed file system of the present invention.
8 is a diagram illustrating a recovery method in an unexpected situation of an existing metadata server in the distributed file system of the present invention.

Other objects, features and advantages of the present invention will become apparent from the detailed description of the embodiments with reference to the accompanying drawings.

A preferred embodiment of a distributed file system having a multi-metadata server structure and a data processing method using the same according to the present invention will be described with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. It is provided to let you know. Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention and are not intended to represent all of the technical ideas of the present invention. Therefore, various equivalents It should be understood that water and variations may be present.

2 is a diagram illustrating a distributed file system having a multi-metadata server structure according to an embodiment of the present invention.

As shown in FIG. 2, the distributed file system stores and manages file data, and includes at least two or more data servers 300 that periodically report their status information to the metadata server 20, and at least two or more data servers. And are respectively arranged by applying a DHT (Distributed Hash Table) concept, and store metadata about all files of the file system, and store the metadata for all data servers 300 based on the reporting data provided from the data server 300. A metadata server 200 that manages state information and a file system client 100 that accesses file metadata from the metadata server 200 and inputs and outputs file data from the data server 300 while executing a client application. Include. In this case, each data server 300 configures its own local storage, and is divided and stored in units of codeword sizes corresponding to one frame of local storage in each data server 300. The file system client 100, the metadata server 200, and the data server 300 are interconnected to the network 400.

3 is a diagram illustrating a distributed storage method of data according to an embodiment of the present invention.

As shown in FIG. 3, one original file is divided into d pieces, and d ₁ to d _d divided data are sequentially arranged. P parity data (three (p1 to p3) parity data are described here for ease of explanation) for each of the divided data. At this time, the d + p is preferably equal to or less than the number of the data server 300.

As described above, the divided data divided into the divided data and the parity data are respectively transmitted to the plurality of data servers 300 to store the first divided data d1 in the first local storage in the first data server, and to store the divided data d1 in the second data server. The second divided data d2 is stored in the second local storage, and the third divided data d3 is stored in the third local storage in the third data server. In this manner, the n th partition data dn is stored in the n th local storage in the n th data server.

The distributed file system stores meta information including location information of divided data stored in local storage of the data servers 30 in a separate metadata server (MDS) 200.

In this way, one original file is divided into d pieces and stored in local storage in each data server.

As shown in FIG. 4, the metadata server 200 is composed of at least two and arranged by applying the concept of a distributed hash table (DHT). In addition, one of the values existing in the hash space is defined as a token value.

The metadata server 200 manages state information on the existing data server 300 while following a hash ring in a clockwise or counterclockwise direction, and meets another metadata server 200. Define up to the point as your management area. For example, the area of responsibility of the first metadata server MDS # 1 is 4 to 8, and the area of responsibility of the second metadata server MDS # 2 is 8-16, 0-4. Is defined as

Through this configuration, when the data server 300 and the file system client 100 visit the metadata server 200, the token generating function unit 500 generates a unique token as shown in FIG. 5. When the token generated in this way is placed in the hash space, it is set to the metadata server 200 in charge of the area. That is, as shown in FIG. 5, the data server 300 registers at the second metadata server 200b which is in charge of an area where its token is located and maintains a keep-alive. Report status information (repoinfo). In addition, the file system client 100 visits the first metadata server 200a and performs input / output of file data.

At this time, the state information reported by the data server 300 is transmitted to only one metadata server. Accordingly, the second metadata server 200b shares the information it has known to other MDSs in the hash ring with the status information reported from the data server 300. The sharing method uses a probabilistic Gossip protocol or a system shared information storage unit (ilock).

On the other hand, when defining a token value, which is one of the values existing in the hash space, to the metadata server 200, the metadata server 200 should be defined to be evenly generated in the entire hash space. . To do this, we use a hash algorithm such as the MD5 algorithm (Message-Digest algorithm 5) and the SHA algorithm (Secure Hash algorithm).

That is, the request message of the file system client (or WebFront) 100 combines a user, a directory, and an object string, and hashes using a hash algorithm to generate a token. do. In addition, the data server 300 generates a token by hashing its unique identifier (GUID) with a hash algorithm. For reference, the hash algorithm is a 128-bit cryptographic hash function designated by RFC 1321, and is a known method used mainly for checking integrity of a program or a file, .

As such, by defining tokens in the metadata server 200 using a hash algorithm, loads are placed on the metadata servers 200 existing in the hash ring without considering load balancing. It is divided evenly across the hash space.

The hash space represented by the circle illustrated in FIG. 4 may be represented as shown in FIG. 6 in the form of a map. The SIOR in the division map shown in FIG. 6 represents an access address (IP, port) of the metadata server 200.

In addition, the partition map having the token and server connection address information shown in FIG. 6 is disclosed using a system shared information storage unit (iLock) or a probability-based Gossip protocol, and only the metadata server 200 edits the partition map. can do. The data server 300 or the file system client 100 reads this partition map and searches the hash space to find the metadata server 200 in the corresponding area. At this time, the method using the shared information storage unit is a centralized information sharing method, and the method using the Gossip protocol is an embodiment of a non-centralized information sharing method. In the case of the centralized information sharing method, the system sharing information storage unit is used for the disclosure of the divided map, and in the case of the non-centralized information sharing method, the Gossip protocol for efficiently propagating information to a plurality of servers based on the probability It is possible to use a method of using it. It is also possible to use the traditional multicast method.

Since the hash space used in the present invention is 2 ¹²⁸ , a token having a value between 0 and 2 ¹²⁸ -1 is used. For reference, the split map is recorded as a hex value.

7 is a diagram for explaining a process of adding a new metadata server.

As shown in FIG. 7, only the first metadata server MDS # 1 currently exists in the hash space, and only the token and server connection address information of the first metadata server are stored in the partition map. Doing.

At this time, when the second metadata server (MDS # 2) is added to the hash ring, the second metadata server (MDS # 2) reads the partition map, and then the MD5 algorithm (Message-Digest algorithm 5), SHA algorithm (Secure) It records its token value and server connection address value determined through hash algorithm such as Hash algorithm.

Then, the first metadata server MDS # 1 provides the memory cache of the area secured by the second metadata server MDS # 2 in its own area, and the data server 300 that is out of its area is removed. 2 Manage to movemove (MistRegistration) to metadata server (MDS # 2).

As such, when a new second metadata server (MDS # 2) enters between the existing first metadata server MDS # 1, the token value of the existing first metadata server MDS # 1 is It does not change, only the area of management changes.

On the contrary, upon receiving a stop request of an existing metadata server, the stopped metadata server deletes its token from the partitioning map and terminates.

In addition, the neighboring metadata server in the hash ring increases its area up to the hash area of the deleted metadata server. At this point, the extended metadata server does nothing differently than when it was added.

On the other hand, if the normal operation cannot be performed due to an unexpected situation (failure, etc.) among the metadata servers located in the hash space, a method for recovering the metadata server should be made. To this end, the metadata server 200 performs a normal operation according to an unexpected situation of the metadata server by using a method of continuously monitoring the metadata server 200 corresponding to another neighboring node in a hash ring structure. To compensate.

Referring to FIG. 8, a recovery method in an unexpected situation of the metadata server will be described.

As shown in FIG. 8, the first, second, and third metadata servers exist in the hash space, the first metadata server MDS # 1 has a token value of 4, and the second metadata server MDS # 2. Has 8 as the token value and the third metadata server MDS # 3 has 14 as the token value.

At this time, if it is determined that the second metadata server (MDS # 2) is a failure due to an unexpected situation, the second metadata server that is determined to be broken using the system shared information storage unit iLock or Gossip protocol ( The token value of MDS # 2 is released, and the neighboring first metadata server MDS # 1 that is monitoring the second metadata server MDS # 2 monitors the second object. The data is changed to the third metadata server MDS # 3 monitored by the metadata server MDS # 2.

Accordingly, even when a failure of the metadata server occurs, it is possible to effectively manage the status information for all data servers.

Although the technical spirit of the present invention described above has been described in detail in a preferred embodiment, it should be noted that the above-described embodiment is for the purpose of description and not of limitation. It will be apparent to those skilled in the art that various modifications may be made without departing from the scope of the present invention. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

Claims (14)

At least two data servers that store and manage file data and periodically report their status information to a metadata server,
It is composed of at least two and arranged respectively by applying the concept of Distributed Hash Table (DHT), and stores metadata about all files of the file system, and the state of the data server based on the state information provided from the data server. A metadata server for managing information,
And a file system client that accesses file metadata from the metadata server and inputs and outputs file data from a data server while executing a client application.
In this case, the metadata server defines one of the values existing in the hash space as a token value, and the data server is located along the hash ring in the clockwise or counterclockwise direction. A distributed file system having a multi-metadata server structure, which manages state information for a server and defines its own management area until a point where another metadata server is encountered. The method of claim 1,
The data server divides the input original file into at least one or more data, generates one or more pieces of divided data in which at least two or more parity data are combined with the divided original file data, and then divides each generated piece of data. Distributed file system having a multi-metadata server structure, characterized in that divided and stored again. 3. The method of claim 2,
The generated partitioned data is generated as a codeword size of local storage including parity data through encoding. delete The method of claim 1,
The metadata server structure may include a token generation function configured to generate a token value so that the metadata server is located in the entire hash space using a hash algorithm and set the metadata server as a metadata server. Distributed file system with. The method of claim 1,
The hash space is a distributed file system having a multi-metadata server structure, comprising a partition map including a token of a metadata server and access address information of the metadata server. At least two data servers that store and manage file data, periodically report their status information to the metadata server, and a metadata server that stores metadata about the file and manages status information about the data server. And a file system client that accesses file metadata from the metadata server and inputs and outputs file data from the data server while executing a client application. In the data processing method,
At least two metadata servers are respectively arranged by applying the concept of a distributed hash table (DHT), and one of the values existing in the hash space is defined as a token value.
In addition, the metadata server defines its own management area before the point where it meets another metadata server, and provides status information about the existing data server along a hash ring in a clockwise or counterclockwise direction. A data processing method using a distributed file system having a multi-metadata server structure, which is managed. delete The method of claim 7, wherein
The state information reported by the data server is delivered to only one metadata server,
And receiving the status information from the metadata server and sharing the reported status information with another metadata server in a hash ring. The method of claim 7, wherein the metadata server
Setting a token value generated in the metadata server so that the metadata server is located throughout the hash space using a hash algorithm;
The partition map with the metadata server's token and server connection address information is published using the system shared information storage unit (iLock) or the Gossip protocol, and based on the published partition map, the data server or file system client is meta-based in the hash space. Detecting a data server;
And receiving status information (repoinfo) of the data server including the disk status information managed by the data server from the data server that is registered and keep keeps alive (KeepAlive). Data processing method using a distributed file system having a multi-metadata server structure. 11. The method of claim 10,
After the second metadata server (MDS # 2) newly added to the hash ring reads the partitioning map, the partitioning map recording the token and server connection address information determined through a hash algorithm is stored in the system shared information storage unit. using iLock or Gossip protocol
Giving up the rights area secured by the second metadata server (MDS # 2) by the first metadata server (MDS # 1) that has already taken part or all of the management rights on the hash ring defined above;
Detecting a metadata server in a hash space by a data server or a file system client whose management authority is changed based on the partitioning map;
And receiving status information (repoinfo) of the data server including the disk status information managed by the data server from the data server where a register is made and keepalive is maintained. Data processing method using a distributed file system having a multi-metadata server structure. 11. The method of claim 10,
Upon receiving a stop request of an existing metadata server, the stopped metadata server publishes a partitioned map from which its token is deleted using the system shared information storage unit (iLock) or Gossip protocol. Terminating;
And a step in which the neighboring metadata server in the hash ring increases its area up to the hash area of the deleted metadata server. 11. The method of claim 10,
If the fourth metadata server of the metadata server located in the hash space is not expected to operate normally due to an unexpected situation, the fourth metadata server found to be broken using the system shared information storage unit (iLock) or Gossip protocol Revealing that the token value of the metadata server has been removed,
And changing the monitoring target to a fifth metadata server monitored by the fourth metadata server in another neighboring third metadata server monitoring the fourth metadata server determined as the failure. A data processing method using a distributed file system having a multi-metadata server structure, characterized in that the configuration. 11. The method of claim 10,
The metadata server continuously monitors a metadata server corresponding to another neighboring node in a hash ring structure.

Images