KR20120044694A - Asymmetric distributed file system, apparatus and method for distribution of computation - Google Patents

Abstract

PURPOSE: An asymmetric distribution file system, an operation distributing device, and a method thereof are provided to distribute various operations to a main metadata server and a sub metadata server, thereby guaranteeing stable performance despite increases in the number of clients and data. CONSTITUTION: A main metadata server(100) executes operations which need to modify metadata. A sub metadata server(110-1~110-m) executes operations which do not need to modify metadata. A data server(120-1~120-n) stores the original copy and a replica of data which are generated or modified by the operations. The main metadata server determines a sub metadata server which requests the operations.

Description

Asymmetric Distributed File System, Computation Distribution Device and Method {ASYMMETRIC DISTRIBUTED FILE SYSTEM, APPARATUS AND METHOD FOR DISTRIBUTION OF COMPUTATION}

The present invention relates to an asymmetric distributed file system, and more particularly, to an asymmetric distributed file system including a primary metadata server and a secondary metadata server, and to any one of a metadata server and an auxiliary metadata server in such an asymmetric distributed file system. The present invention relates to an operation distribution apparatus and a method for determining an operation subject of an individual operation and requesting an operation.

As is well known, an asymmetric distributed file system is a system that stores and manages metadata and actual data of a file separately. The metadata is managed in a metadata server and the actual data is distributed and stored in multiple data servers. The metadata managed by the metadata server includes information about the data server where the actual data is stored.

These metadata servers and data servers are distributed over a network. Therefore, the client terminal is separated from the file metadata and the path for accessing the data. That is, in order to access the file, the client terminal first accesses the metadata of the file in the metadata server to obtain information about the data servers in which the actual data is stored. Then input and output to the actual data is done through the data servers.

The conventional asymmetric distributed file system is composed of one metadata server and a plurality of data servers. When the number of registered clients is small, that is, the number of client terminals is small, the data access is guaranteed, but the number of clients is fast. If the data accumulation progresses above a certain level due to the increase of, a performance bottleneck occurs due to the structure of a single metadata server. For example, in a situation where a large number of files of multiple clients are registered, when a search operation is performed on the registered files, the search area range is increased, and system performance that other clients currently connected to is rapidly degraded.

The present invention has been proposed to solve the problems of the prior art, and provides an asymmetric distributed file system that minimizes the performance bottleneck by distributing the functions of the existing metadata server to the primary metadata server and the secondary metadata server. .

The present invention provides an operation distribution apparatus and method capable of appropriately distributing various individual operations to a primary metadata server and an auxiliary metadata server of an asymmetric distributed file system.

As a first aspect of the present invention, an asymmetric distributed file system includes a main metadata server for performing an operation requiring modification of metadata among various individual operations, and an auxiliary meta for performing an operation without modification of the metadata among the individual operations. And a data server for storing originals and replicas of data generated or modified by the individual operations.

The asymmetric distributed file system may include a plurality of auxiliary metadata servers, and the main metadata server may determine a supplementary metadata server to request the individual operation among the plurality of auxiliary metadata servers.

In accordance with a second aspect of the present invention, an asymmetric distributed file system may perform data synchronization with a main metadata server performing various individual operations and the main metadata server to support search operations among the individual operations, It may include a secondary metadata server for performing the various individual operations in the event of a data server failure, and a data server for storing the original and the copy of the data generated or modified by the individual operations.

According to a third aspect of the present invention, an operation distribution device includes a reference information storage unit for storing reference information related to the execution of various individual operations, and a result of comparing the operation information required to be performed among the individual operations with the reference information. An operation subject determining unit which determines one of a metadata server and an auxiliary metadata server as a subject of the individual operation, and the one of the primary metadata server and the auxiliary metadata server according to the determination of the operation subject determining unit. It may include an operation request unit for requesting an individual operation.

Here, the reference information storage unit stores operation information that requires modification of the metadata among the individual operations and operation information that does not require modification of the metadata, and the operation subject determining unit stores the operation information that requires the execution of the meta information. In the case of operation information requiring modification of data, the main metadata server is determined as the subject of the individual operation, and in the case in which the operation information required to be performed is operation information that does not require modification of the metadata, the auxiliary metadata server May be determined as the subject of the individual operation.

The reference information storage unit stores search operation information among the individual operations, and when the operation information required for performing the search operation information is the search operation information, the auxiliary metadata server as a subject of the individual operations. If the operation information required to be performed is not the search operation information, the main metadata server may be determined as a subject of the individual operation.

According to a fourth aspect of the present invention, an operation distribution method includes receiving reference information related to performance of various individual operations, comparing operation information required to be performed among the individual operations with the reference information, and performing the comparison. Determining one of the primary metadata server and the secondary metadata server as the subject of the individual operation based on a result; and determining the individual operation to any one of the primary metadata server and the auxiliary metadata server according to the determination. It may include the step of requesting.

Here, the step of receiving the operation, the operation information that requires modification of the metadata and the operation information that does not need to modify the metadata of the individual operation is provided, the determining step, the operation information required to perform the operation is the meta In the case of operation information requiring modification of data, the main metadata server is determined as the subject of the individual operation, and in the case in which the operation information required to be performed is operation information that does not require modification of the metadata, the auxiliary metadata server May be determined as the subject of the individual operation.

The receiving step may be provided with search operation information among the individual operations, and the determining may include: when the operation information required to be performed is the search operation information, the auxiliary metadata server as a subject of the individual operation. If the operation information required to be performed is not the search operation information, the main metadata server may be determined as a subject of the individual operation.

According to an exemplary embodiment of the present invention, by performing various distributed operations between the primary metadata server and the secondary metadata server, it is possible to support stable performance by minimizing a performance bottleneck even with an increase in the number of clients and an increase in the amount of data. . For example, in a situation where a large number of files of multiple clients are registered, the auxiliary metadata server supports a search operation on the registered files so that the system performance of other clients connected to the system is not drastically degraded.

1 is a block diagram of an asymmetric distributed file system according to an embodiment of the present invention.
Figure 2 is a block diagram of a calculation distribution device according to an embodiment of the present invention.
3 is a flowchart illustrating an example of an operation process of an asymmetric distributed file system to which an operation distribution device according to an exemplary embodiment of the present invention is applied.
4 is a flowchart illustrating another example of an operation process of an asymmetric distributed file system to which an operation distribution device according to an exemplary embodiment of the present invention is applied.
5 is a block diagram of an asymmetric distributed file system according to another embodiment of the present invention.

Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but can be implemented in various different forms, and only the embodiments make the disclosure of the present invention complete, and the general knowledge in the art to which the present invention belongs. It is provided to fully inform the person having the scope of the invention, which is defined only by the scope of the claims.

In describing the embodiments of the present invention, if it is determined that a detailed description of a known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, terms to be described below are terms defined in consideration of functions in the embodiments of the present invention, which may vary according to intentions or customs of users and operators. Therefore, the definition should be based on the contents throughout this specification.

Each block of the accompanying block diagrams and combinations of steps of the flowchart may be performed by computer program instructions. These computer program instructions may be mounted on a processor of a general purpose computer, special purpose computer, or other programmable data processing equipment such that instructions executed through the processor of the computer or other programmable data processing equipment may not be included in each block or flowchart of the block diagram. It will create means for performing the functions described in each step. These computer program instructions may be stored in a computer usable or computer readable memory that can be directed to a computer or other programmable data processing equipment to implement functionality in a particular manner, and thus the computer usable or computer readable memory. It is also possible for the instructions stored in to produce an article of manufacture containing instruction means for performing the functions described in each block or flowchart of each step of the block diagram. Computer program instructions may also be mounted on a computer or other programmable data processing equipment, such that a series of operating steps may be performed on the computer or other programmable data processing equipment to create a computer-implemented process to create a computer or other programmable data. Instructions that perform processing equipment may also provide steps for performing the functions described in each block of the block diagram and in each step of the flowchart.

In addition, each block or step may represent a portion of a module, segment or code that includes one or more executable instructions for executing a specified logical function (s). It should also be noted that in some alternative embodiments, the functions noted in the blocks or steps may occur out of order. For example, the two blocks or steps shown in succession may in fact be executed substantially concurrently or the blocks or steps may sometimes be performed in the reverse order, depending on the functionality involved.

1 is a block diagram of an asymmetric distributed file system according to an embodiment of the present invention.

As shown therein, the asymmetric distributed file system 100 includes a primary metadata server 110, m secondary metadata servers 110-1, ..., 110-m, and n data servers 120-1, 120-. 2, 120-3, ..., 120-n, and may be configured for a network of k client terminals 10-1, 10-2, 10-3, 10-4, ..., 10-k. It provides data storage and management.

In FIG. 1, m auxiliary metadata servers 110-1 to 110-m are shown. However, the number of auxiliary metadata servers is, for example, starting from the first one, that is, the number of clients, that is, the client terminal. Can be generated dynamically increase as the number increases.

As described above, the functions of terminals and servers constituting the asymmetric distributed file system are as follows.

The client terminals 10-1, 10-2, 10-3, 10-4,..., 10-k perform the client application while the primary metadata server 110 or the secondary metadata server 110-1,. Access file metadata from 110-m) and input and output file data from data servers 120-1, 120-2, 120-3, ..., 120-n.

Primary metadata server 110 and secondary metadata servers 110-1, ..., 110-m store and manage metadata for all files in the file system, and all data servers 120-1, 120-2. , 120-3, ..., 120-n). The main metadata server 110 performs an operation requiring modification of metadata among various individual operations generated by the client terminals 10-1, 10-2, 10-3, 10-4,..., 10-k, or You can also perform all individual operations. The auxiliary metadata server 110-1,..., 110-m may transmit metadata from various individual operations generated by the client terminals 10-1, 10-2, 10-3, 10-4,..., 10-k. It is possible to perform a role of the main metadata server 110 by performing a variety of separate operations in the event of an error of the main metadata server 110 while performing operations that do not require modification or support of search operations among the individual operations. To this end, the auxiliary metadata servers 110-1,..., 110-m may periodically perform data synchronization with the primary metadata server 110.

The data servers 120-1, 120-2, 120-3,..., 120-n are individual operations performed by the primary metadata server 110 or the secondary metadata server 110-1,..., 110-m. Can store originals and replicas of data created or modified by.

2 is a block diagram of an operation distribution device according to an exemplary embodiment of the present invention.

As described above, the calculation distribution device 200 may include an operation subject determining unit 210, a reference information storage unit 220, an operation requesting unit 230, and the like. The operation distribution device 200 may be mounted in the form of hardware or software in the main metadata server.

The reference information storage unit 220 stores reference information related to performance of various individual operations. For example, the reference information storage unit 220 may classify and store operation information that requires modification of metadata and operation information that does not require modification of metadata among individual operations. Alternatively, the search operation information may be stored among individual operations.

The calculation subject determining unit 210 determines one of the primary metadata server and the secondary metadata server according to a result of comparing the operation information required to be performed among the individual operations with the reference information of the reference information storage unit 220. It can be decided by the subject. For example, the operation subject determining unit 210 determines that the main metadata server is the subject of an individual operation when the operation information required to be performed is operation information requiring modification of the metadata, and the operation information required to be executed is metadata. In the case of operation information that does not need to be modified, the auxiliary metadata server may be determined as a subject of an individual operation. Alternatively, the operation subject determining unit 210 determines the auxiliary metadata server as the subject of the individual operation when the operation information required to be performed is search operation information, and when the operation information required to be performed is not the search operation information. You can determine the primary metadata server as the subject of an individual operation.

The operation requesting unit 230 may request an individual operation from one of the primary metadata server and the auxiliary metadata server according to the determination of the operation subject determining unit 210.

3 is a flowchart illustrating an example of an operation process of an asymmetric distributed file system to which an operation distribution device according to an exemplary embodiment of the present invention is applied.

As described above, the operation process of the asymmetric distributed file system may include receiving and storing operation information requiring modification of meta data and operation information without modification of meta data as reference information related to performing various individual operations (S301); Determining one of the main metadata server and the auxiliary metadata server as the subject of the individual operation based on a result of comparing the operation information required to be performed according to the occurrence of the operation with previously stored reference information (S303 and S305); Requesting an individual operation from the primary metadata server or the secondary metadata server to store the metadata by determining the operation subject (S307 and S309); and a data source created or modified according to the execution of the individual operation. And storing the replica in the data server (311).

4 is a flowchart illustrating another example of an operation process of an asymmetric distributed file system to which an operation distribution device according to an exemplary embodiment of the present invention is applied.

For this embodiment, the operation distribution device receives and stores search operation information among individual operations as reference information related to the execution of various individual operations, and compares the operation information and the search operation information required to be performed according to the occurrence of the operation. If the operation information required to be performed is search operation information, the secondary metadata server is determined as the subject of the individual operation. If the operation information required to be performed is not search operation information, the main metadata server is designated as the subject of the individual operation. Can be determined.

In the operation process of the asymmetric distributed file system applying the operation distribution device, in the initial state that the main metadata server is normally operating, the main metadata server performs various individual operations to store metadata (S401 to S405). ), Determining that an error occurs in the primary metadata server (S407), and when the error occurs in the primary metadata server, the secondary metadata server performs various individual operations to store the metadata (S411), and the like. It may include.

5 is a block diagram of an asymmetric distributed file system according to another embodiment of the present invention. The embodiment shown here adds a service server 500 between the client terminals 10-1, 10-2, 10-3, 10-4, ..., 10-k and the asymmetric distributed file system 100 '. It is shown that it can be implemented in, and the operation distribution device as shown in the embodiment of Figure 2 can be mounted on the service server 500.

Hereinafter, an operation distribution process and an operation process of the asymmetric distributed file system by the asymmetric distributed file system and the operation distribution device configured as described above will be described. Hereinafter, the first embodiment will be described with reference to FIGS. 1 to 3, and the second embodiment will be described with reference to FIGS. 1, 2, and 4.

(First embodiment)

First, the reference information storage unit 220 of the calculation distribution device 200 receives operation information that requires modification of metadata and operation information that does not require metadata modification from a manager or the like as reference information related to performance of various individual operations. Save (S301). For example, the mkdir command for creating a directory, the rm command for deleting files and directories, the del command for deleting files, and the like can be stored as operation information requiring modification of metadata, the dir command for displaying a list of directories, The ls command to check the contents and the find command to search the file can be stored as operation information that does not require modification of metadata, that is, read-only operation information.

If an operation that requires execution occurs according to the client application processing of the client terminals 10-1, 10-2, 10-3, 10-4, ..., 10-k, the operation subject determining unit 210 requests the execution. Based on a result of comparing the calculated operation information and the reference information previously stored in the reference information storage unit 220, any one of the primary metadata server 110 and the auxiliary metadata server 110-1, ..., 110-m is selected. Determined by the subject of an individual operation. Herein, in case of an operation requiring modification of meta data, the main metadata server 110 is determined as a subject of an individual operation, and in case of an operation requiring no modification of meta data, the auxiliary metadata server 110-1,. -m) may be determined as a subject of an individual operation (S303).

The operation requesting unit 230 performs an individual operation on any one of the primary metadata server 110 and the auxiliary metadata server 110-1,..., 110-m according to the operation subject determined by the operation subject determining unit 210. Ask. For example, in the case of an operation requiring modification of metadata, the main metadata server 110 is allocated to store metadata while performing individual operations (S309). On the other hand, if the operation does not need to modify the metadata, secondary metadata servers 110-1, ..., 110-m are allocated (S305), and in this case, the primary metadata server 110 is the secondary metadata server. Recommends and assigns any one auxiliary metadata server in the idle state among (110-1, ..., 110-m) as the subject of the individual operation, and the assigned auxiliary metadata server performs the individual operation while performing the individual operation. The data is stored (S307).

Thereafter, the data servers 120-1, 120-2, 120-3,..., 120-n store the data source and the replica which are generated or modified according to the execution of the individual operation (S311).

(Second embodiment)

First, the reference information storage unit 220 of the calculation distribution apparatus 200 receives and stores search operation information among individual operations as reference information related to the execution of various individual operations.

If an operation that requires execution occurs according to the client application processing of the client terminals 10-1, 10-2, 10-3, 10-4, ..., 10-k, the operation subject determining unit 210 requests the execution. Based on a result of comparing the calculated operation information and the reference information previously stored in the reference information storage unit 220, any one of the primary metadata server 110 and the auxiliary metadata server 110-1, ..., 110-m is selected. Determined by the subject of an individual operation. In this case, when the operation information required to be performed is search operation information, the auxiliary metadata servers 110-1,..., And 110-m are determined as subjects of the individual operations, and the operation information required to be performed is search operation information. If not, the main metadata server 110 may be determined as a subject of an individual operation. Here, the primary metadata server 110 may recommend and allocate any secondary metadata server in the idle state among the auxiliary metadata servers 110-1,..., 110-m as a subject of an individual operation.

As described above, the auxiliary metadata server 110-1,..., 110-m supports only search operations, and when an error occurs in the main metadata server 110, the auxiliary metadata servers 110-1,. Function can be performed.

In the initial state that the main metadata server 110 is operating normally, when the operation subject determining unit 210 allocates the main metadata server 110 as a subject of an individual operation, for example, in the case of a data write operation, The data server 110 allocates a data write position chunk (block) (S403), and the main metadata server 110 performs various individual operations to store metadata (S405). To this end, the primary metadata server 110 and the secondary metadata server 110-1,..., 110-m periodically perform data synchronization with each other (S401).

In this situation, the calculation subject determining unit 210 determines the occurrence of an error of the main metadata server (S407).

If it is determined in step S407 that an error has occurred in the primary metadata server 110, the computational subject determining unit 210 assigns the supplementary metadata servers 110-1, ..., 110-m to the computational subject. In the case of a data write operation, the auxiliary metadata server allocates a data write location chunk (block) (S409), and the auxiliary metadata server performs the corresponding individual operation according to the operation request of the operation requesting unit 230. Meta data is stored (S411).

The calculation distribution method according to the embodiment of the present invention can be created by a computer program. The code and code segments that make up this computer program can be easily deduced by a computer programmer in the field. In addition, the computer program may be stored in a computer readable medium, and read and executed by a computer or a main metadata server or a service server according to an embodiment of the present invention to implement a method of distributing calculations. Can be. The information storage medium includes a magnetic recording medium, an optical recording medium and a carrier wave medium.

Embodiments of the present invention provide a data storage service server called a term such as web hard and internet hard, a service system using the same, a method of operating the same, and a cloud storage server providing virtualized storage resources according to a user's request. The service system, its operation method, and the multiple data centers can be used for the wired or wireless communication service system and its service method, which are common, and the performance bottleneck is caused by the increase in the number of clients and the amount of data. It can support stable performance by minimizing the

10-1, 10-2, 10-3, 10-4,... , 10-k: client terminal
100, 100 ': Asymmetric distributed file system
110: primary metadata server
110-1,... , 110-m: secondary metadata server
120-1, 120-2, 120-3,... , 120-n: data server
200: operation distribution device
210: operation subject determining unit
220: reference information storage unit
230: operation request unit

Claims (9)

A main metadata server for performing operations requiring modification of metadata among various individual operations,
An auxiliary metadata server performing an operation that does not require modification of the metadata among the individual operations;
A data server storing originals and replicas of data generated or modified by said individual operations
Asymmetric distributed file system comprising a. The method of claim 1,
The asymmetric distributed file system includes a plurality of the secondary metadata servers,
The primary metadata server determines a secondary metadata server to request the individual operation from among the plurality of auxiliary metadata servers.
Asymmetric Distributed File System. A main metadata server that performs various individual operations,
A secondary metadata server that performs data synchronization with the primary metadata server to support search operations among the individual operations, and performs the various individual operations in the event of a failure of the primary metadata server;
A data server storing originals and replicas of data generated or modified by said individual operations
Asymmetric distributed file system comprising a. A reference information storage unit for storing reference information related to the performance of various individual operations;
An operation subject determining unit which determines one of a main metadata server and an auxiliary metadata server as a subject of the individual operation according to a comparison result of the operation information required to be performed among the individual operations and the reference information;
An operation request unit for requesting the individual operation from one of the primary metadata server and the auxiliary metadata server according to the determination of the operation subject determining unit;
Computation distribution device comprising a. The method of claim 4, wherein
The reference information storage unit stores operation information that requires modification of the metadata among the individual operations and operation information that does not require modification of the metadata,
The operation subject determining unit determines that the main metadata server is a subject of the individual operation when the operation information required to be performed is operation information requiring modification of the metadata, and the operation information required to perform the operation is determined. In the case of operation information that does not require modification of metadata, the auxiliary metadata server is determined as a subject of the individual operation.
Computing distribution device. The method of claim 4, wherein
The reference information storage unit stores search operation information among the individual operations,
The operation subject determining unit determines that the auxiliary metadata server is the subject of the individual operation when the operation information required to be performed is the search operation information, and the operation information required to be performed is not the search operation information. In the case of determining the main metadata server as the subject of the individual operation
Computing distribution device. Receiving reference information related to the performance of various individual operations;
Comparing the operation information required to be performed among the individual operations with the reference information;
Determining one of the primary metadata server and the secondary metadata server as the subject of the individual operation based on a result of the comparison;
Requesting the individual operation from either the primary metadata server or the secondary metadata server according to the determination.
Operation distribution method comprising a. The method of claim 7, wherein
The receiving step may be provided with operation information that requires modification of the metadata and operation information that does not require modification of the metadata among the individual operations,
The determining may include determining that the main metadata server is the subject of the individual operation when the operation information required to be performed is operation information that requires modification of the metadata, and the operation information required to be performed is determined by the operation information. In the case of operation information that does not require modification of metadata, the auxiliary metadata server is determined as a subject of the individual operation.
Operational Distribution Method. The method of claim 7, wherein
In the receiving step, the search operation information is provided from the individual operations,
The determining may include determining that the auxiliary metadata server is a subject of the individual operation when the operation information required to be performed is the search operation information, and the operation information required to be performed is not the search operation information. In the case of determining the main metadata server as the subject of the individual operation
Operational Distribution Method.

Images