WO2014114089A1

WO2014114089A1 - Method and system for optimizing load balance of distributed file system

Info

Publication number: WO2014114089A1
Application number: PCT/CN2013/083227
Authority: WO
Inventors: 金泉洁
Original assignee: 中兴通讯股份有限公司
Priority date: 2013-01-28
Filing date: 2013-09-10
Publication date: 2014-07-31
Also published as: CN103973583A; CN103973583B

Abstract

A method and system for optimizing the load balance of a distributed file system. The system comprises a metadata server, a storage server and a database server, and the database server receives a file write-in request sent from a client, writes in a file according to the file write-in request, and labels each fragment of each file; the metadata server initiates a fragment migration request to the storage server when it is monitored that the state of the storage server is abnormal; according to the labels of the each fragment of a local storage file and abnormal information carried by the fragment migration request, the storage server reallocates fragment positions and initiates migration; and when the migration succeeds, new position information about the fragments is reported to the database server. In the embodiments of the present invention, fragments are readjusted in the case of an abnormal storage server, hard disk failure and capacity expansion, achieving the effects of optimizing the load balance, saving the expense of a metadata server and improving the entire performance under a small file model.

Description

Method and system for optimizing load balancing of distributed file system

Technical field

The present invention relates to the field of distributed file system load balancing technology, and in particular, to a method and system for optimizing load balancing by automatically adjusting the distribution of distributed file system file fragments.

Background technique

Compared with traditional magnetic array devices, distributed file systems are compatible with different types of devices, easy to access, and easy to expand. Distributed file systems are highly demanding for load balancing because they work together to provide a large number of storage devices.

Distributed file systems typically cut large files into shards of the same size and then store the shards separately in the storage device to be served. If you need to achieve load balancing, you must solve the following two problems:

First, when a user requests to write or read, the metadata server must balance the resource information of the existing storage server, and select a storage server that is more conducive to overall optimization to provide services for the user. Second, the location of file fragmentation must be reasonable. Otherwise, no matter how the metadata server optimizes the storage server, it must obtain data from the storage server that stores the corresponding data.

For the second problem mentioned above, how to perform secondary adjustment and optimize load balancing becomes a problem to be solved when the file storage location is unreasonable.

File shard storage location is unreasonable. The vast majority of cases are caused by hard disk failure and expansion. In the process of hard disk failure and capacity expansion, there are two situations that affect the even distribution of file fragments. In one case, the state of the hard disk is normal, but the reading and writing is slow, so that the original database is forced to readjust according to the load balancing allocation scheme; in another case, the hard disk state is bad and the capacity is expanded, and some fragments need to be re-adjusted to Newly added or replaced on the hard drive.

A related solution is to report the hard disk space utilization by the storage server, and the metadata server periodically polls. After the hard disk space utilization is found to be greater than a certain threshold, the metadata server initiates the fragment migration. However, this solution has the following drawbacks:

First, the hard disk space utilization can only reflect the number of shards on the hard disk, and does not represent the distribution of files on all hard disks. Therefore, after shard migration, a large number of files will be stored in less On the hard disk, it is not possible to make all the hard disks work together for the user.

Second, the hard disk space utilization cannot reflect the storage information of the first fragment of the file. The migration may cause the first fragment of many files to be stored in a small number of hard disks. When the user reads the file, it starts from the first slice, which results in insufficient capacity.

Third, all migrations are done by the metadata server, which undoubtedly increases the burden on the metadata server. Especially in the model with a large proportion of small files, it will affect the overall performance of the file system.

Summary of the invention

The main object of the present invention is to provide a method and system for optimizing load balancing of a distributed file system, which aims to solve the problem of load balancing failure after migration fragmentation.

In order to achieve the above object, the present invention provides a method for optimizing load balancing in a distributed file system, including:

When the metadata server monitors that the status of the storage server is abnormal, the device initiates a fragment migration request to the storage server; performs the abnormal information carried, redistributes the fragmentation location, and initiates the migration; after the migration succeeds, the fragmentation is new. The location information is reported to the database server.

Preferably, when the metadata server monitors that the storage server status is abnormal, before the sending the fragment migration request to the storage server, the method further includes:

The database server receives a file write request sent by the client, writes a file according to the file write request, and tags each fragment of each file.

Preferably, the database server labels each fragment of each file, including: when the database server writes the first file, label the file with a label of 1, and each time a file is added, the label is incremented by one. The label is recycled when the label reaches 65535.

Preferably, the storage server status abnormality includes at least an addition, deletion, and hard disk failure or expansion of the storage server.

Preferably, the monitoring, by the metadata server, that the hard disk fault comprises:

The storage server sends the hard disk abnormality information to the metadata server when detecting the hard disk failure;

After receiving the hard disk abnormality information, the metadata server monitors the hard disk failure. Preferably, when the metadata server monitors that the storage server status is abnormal, initiating a fragment migration request to the storage server includes:

When the metadata server monitors that the storage server status is abnormal and the abnormal time is greater than the predetermined threshold, the fragment migration request is initiated to the storage server. The invention also provides a distributed file system for optimizing load balancing, comprising: a metadata server, a storage server and a database server, wherein:

The metadata server is configured to: initiate a fragment migration request to the storage server when monitoring that the storage server status is abnormal;

The storage server is configured to: re-allocate the fragmentation location according to the label of each fragment of the local storage file and the abnormal information carried by the fragment migration request, and initiate a migration; after the migration succeeds, the fragmentation is new. The location information is reported to the database server.

Preferably, the database server is further configured to: receive a file write request sent by the client, write a file according to the file write request, and label each fragment of each file.

Preferably, the database server is further configured to: label the file when the first file is written, and increase the label by 1 for each subsequent file, and then cycle the label when the label reaches 65535.

Preferably, the storage server is further configured to: send a hard disk abnormality information to the metadata server when detecting a hard disk failure;

The metadata server is further configured to: after receiving the abnormal information of the hard disk, monitor the fault of the hard disk.

Preferably, the metadata server is further configured to: initiate a fragment migration request to the storage server when the storage server status is abnormal and the abnormal time is greater than a predetermined threshold. The method and system for optimizing load balancing of a distributed file system according to an embodiment of the present invention, when the metadata server monitors that the storage server is abnormal, initiates a fragment migration request to the storage server; and the storage server stores the files according to the local storage file. The label of the slice and the exception information carried by the fragment migration request, reallocate the fragmentation location, and initiate the migration; when the migration is successful, the fragmentation will be new. The location information is reported to the database server, so that the fragmentation is re-adjusted in the case of the storage server abnormality and the hard disk failure and capacity expansion, thereby achieving the effect of optimizing the load balancing, saving the expenditure of the metadata server, and improving the small file model. Overall performance. BRIEF abstract

1 is a schematic structural diagram of an embodiment of a distributed file system for optimizing load balancing according to the present invention; FIG. 2 is a flowchart of a hard disk fault fragmentation migration according to an embodiment of the present invention;

3 is a flowchart of migration of a hard disk expansion fragment in the embodiment of the present invention;

4 is a flow chart showing an embodiment of a method for optimizing load balancing of a distributed file system according to the present invention.

Preferred embodiment of the invention

The main idea of the solution in the embodiment of the present invention is: when the metadata server monitors that the storage server is abnormal, the device initiates a fragment migration request to the storage server; the storage server carries the label according to each fragment of the local storage file and the fragment migration request. The abnormal information, reallocate the fragmentation location, and initiate the migration; when the migration is successful, report the new location information to the database server to optimize the load balancing, save the cost of the metadata server, and improve the small file model. Overall performance. As shown in FIG. 1 , a distributed file system for optimizing load balancing according to an embodiment of the present invention includes: a metadata server 10, a storage server 20, and a database server 30, wherein: the database server 30 is configured to receive a client. a file write request sent by the terminal 40, writing a file according to the file write request, and labeling each slice of each file;

The metadata server 10 is configured to initiate a fragment migration request to the storage server 20 when the storage server 20 is abnormal in status; the abnormal information carried by the slice migration request, reallocate the fragmentation location, and initiate migration; After the migration is successful, the new location information of the fragment is reported to the database server 30.

Specifically, in this embodiment, the storage server 20 includes a storage server monitoring module 202, The storage server reporting module 201 and the reset fragmentation module 203, wherein:

The storage server monitoring module 202 is configured to monitor the hard disk and network resource information on the storage server 20, and immediately report the storage server reporting module 201 when the status is abnormal;

The storage server reporting module 201 is configured to receive the message of the storage server monitoring module 202, and report the information monitored by the storage server monitoring module 202 to the metadata server 10 when the hard disk and the network are abnormal.

The reset fragmentation module 203 is configured to determine the optimal location information that the fragment should be stored according to the label of the fragment when the file is first stored, and the current storage server 20 and the hard disk state parameter delivered by the trigger mechanism of the metadata server 10, and initiate fragmentation. migrate.

The metadata server 10 includes a metadata server monitoring module 101 and a migration triggering module 102, wherein:

The metadata server monitoring module 101 is configured to monitor the status of each storage server 20, and find that the storage server 20 is offline or newly added, that is, it is responsible for monitoring the change information of all storage servers 20, such as addition, deletion, abnormality, etc., and receiving the storage server report. The hard disk status information of the module 201 is used by the migration triggering module 102 to initiate a request for resizing the storage server 20 after the specified threshold time period when the storage server 20 is offline or newly added, or the hard disk is abnormal or the expansion occurs. , triggers fragment migration.

The database server 30 includes: a labeling module 301 and a database module 302, wherein: the labeling module 301 is configured to send a label to each fragment of the file when the client 40 requests to write a file for the first time, and all the fragments are Label it. When tagging, each file is tagged with a range of 1-65535, and is incremented one by one in the order in which the request is written.

Specifically, when the first file is written, the file is tagged 1 , and the tag is incremented by 1 for each additional file, and the tag is recycled when the tag reaches 65535.

The database module 302 is configured to record the storage information of the fragment and the status information of the storage server 20 and the hard disk.

The following describes the hard disk abnormal migration process and the capacity expansion process respectively.

As shown in Figure 2, the process of automatic disk fragmentation triggered by automatic disk failure is as follows:

Step 1: After the hard disk is abnormal, the storage server monitoring module 202 sends the abnormal information of the hard disk to the storage server reporting module 201.

Step 2: The storage server reporting module 201 collects all the hard disks on the storage server 20 The message is forwarded to the metadata server monitoring module 101.

Step 3: After collecting the abnormality of the hard disk, the metadata server monitoring module 101 sends a message to the migration triggering module 102 and carries the abnormal hard disk information when the abnormal time is greater than the threshold.

Step 4: The migration triggering module 102 sends a migration request to the reset distribution module, and carries the abnormal hard disk information.

Step 5: The reset fragmentation module 203 of the storage server 20 recalculates the location information that the fragment should be stored according to the abnormal hard disk information and the label, and directly initiates the migration.

Step 6: After the migration is successful, the new location information of the fragment is reported to the database module 302 for storage.

As shown in Figure 3, the hard disk expansion trigger fragment automatic migration process is as follows:

Step 1: The metadata server monitoring module 101 finds that the hard disk is expanded or there is a storage server 20 abnormality (it is found to increase or decrease) and exceeds the time threshold.

Step 2: After the abnormality is found, the metadata server monitoring module 101 sends a migration request to the migration triggering module 102, and carries the abnormal storage server 20 information.

Step 3: The migration triggering module 102 sends a migration request to the reset allocation module, and carries the abnormal storage server 20 information.

Step 4: The reset fragmentation module 203 determines which fragments need to be re-adjusted based on the abnormal storage server 20 information and tags, thereby reallocating the fragmentation locations and initiating the migration.

Step 5: The storage server 20 initiates a migration request, and after the migration succeeds, the new location information is sent to the database module 302.

Compared with the prior art, the foregoing embodiment achieves the improvement of the storage server 20 abnormality and the re-adjustment and equalization of the fragmentation in the case of the hard disk failure and the expansion, and achieves the effect of optimizing the load balancing, thereby saving the expenditure of the metadata server 10. Improved overall performance under the small file model. As shown in FIG. 4, a method for optimizing load balancing of a distributed file system according to an embodiment of the present invention is implemented based on the distributed file system of the foregoing embodiment, and the method includes:

Step S101: When the metadata server monitors that the storage server status is abnormal, initiate a fragment migration request to the storage server.

Step S102, the storage server re-allocates the fragmentation location according to the label of each fragment of the local storage file and the abnormal information carried by the fragment migration request, and initiates migration; After the work, the new location information of the fragment is reported to the database server.

Specifically, in this embodiment, the database server receives the file write request sent by the client, writes the file according to the file write request, and tags each fragment of each file. When the database server labels each fragment of each file, the following scheme is specifically used: When the database server writes the first file, the file is labeled 1 , and each subsequent file is added, the label is increased by 1. Recycle the label when the label reaches 65535.

The storage server monitoring module monitors the hard disk and network resource information on the storage server in real time. When the status is abnormal, the storage server reports the module. The storage server reports the module to receive the message from the storage server monitoring module. When the hard disk and the network are abnormal, the data server is sent to the metadata server. Report the information monitored by the storage server monitoring module.

The metadata server monitoring module is responsible for monitoring the change information of all storage server additions, deletions, exceptions, and the like, and receiving the hard disk status information of the storage server reporting module; when receiving the hard disk status change (including hard disk failure or expansion) or the server is abnormal, After the migration trigger module sends a request for resizing the fragment to the storage server after the specified threshold time, the migration triggers the fragment migration.

The storage server's reset fragmentation module determines the best location information that the fragment should be stored according to the label of the fragment when the file is first stored, and the current storage server and disk state parameters delivered by the metadata server trigger mechanism, and initiates the fragment migration. After the migration is successful, the new location information of the fragment is reported to the database module for storage. The following describes the hard disk abnormal migration process and the capacity expansion process respectively.

Step 1: After the hard disk is abnormal, the storage server monitoring module sends the abnormal information of the hard disk to the storage server reporting module.

Step 2: The storage server reporting module collects all the hard disk abnormal messages on the storage server, and forwards the hard disk abnormality information to the metadata server monitoring module.

Step 3: After collecting the abnormality of the hard disk, the metadata server monitoring module sends a message to the migration trigger module and carries the abnormal hard disk information when the abnormal time is greater than the threshold.

Step 4: The migration trigger module sends a migration request to the reset distribution module, and carries the abnormal hard disk information.

Step 5: The storage server's reset fragmentation module recalculates based on the abnormal hard disk information and labels. Calculate the location information that the shard should store and initiate the migration directly.

Step 6: After the migration is successful, the new location information of the fragment is reported to the database module for storage. As shown in Figure 3, the hard disk expansion trigger fragment automatic migration process is as follows:

Step 1: The metadata server monitoring module finds that the hard disk is expanded or has a storage server abnormality (recovering its increase or decrease) and exceeding the time threshold.

Step 2: After the abnormality is found, the metadata server monitoring module sends a migration request to the migration triggering module, and carries the abnormal storage server information.

Step 3: The migration trigger module sends a migration request to the reset distribution module, and carries the abnormal storage server information.

Step 4: The reset fragmentation module determines which fragments need to be re-adjusted based on the abnormal storage server information and labels, and then reallocates the fragmentation locations and initiates the migration.

Step 5: The migration request is initiated between the storage servers. After the migration is successful, the new location information is reported to the database module.

In this embodiment, after the storage server is abnormal and the hard disk is faulty and expanded, the fragment is re-adjusted to achieve the effect of optimizing the load balancing, saving the expenditure of the metadata server, and improving the overall size of the small file model. performance. The above is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, and equivalent structural or process changes made by the present specification and drawings may be directly or indirectly applied to other related technical fields. The same is included in the scope of patent protection of the present invention.

Industrial Applicability A method and system for optimizing load balancing of a distributed file system according to an embodiment of the present invention, when a metadata server monitors that the storage server is abnormal, initiates a fragment migration request to the storage server; and the storage server stores the file according to the local storage file. The label of each fragment and the exception information carried by the fragment migration request, reallocate the fragmentation location, and initiate the migration; after the migration succeeds, the new location information of the fragment is reported to the database server, thereby the storage server exception and the hard disk In the case of faults and capacity expansion, the shards are re-adjusted to achieve the effect of optimizing load balancing, saving the cost of the metadata server and improving the overall performance under the small file model.

Claims

Claim

A method for optimizing load balancing in a distributed file system, comprising:

The method according to claim 1, wherein, when the metadata server monitors that the storage server is in an abnormal state, before the sending the fragment migration request to the storage server, the method further includes: the database server receiving the sending by the client A file write request writes a file according to the file write request and tags each shard of each file.

The method according to claim 2, wherein the database server tags each fragment of each file to include:

When the database server writes the first file, it tags the file 1 , and each time a file is added, the tag is incremented by 1, and the tag is recycled when the tag reaches 65535.

The method according to claim 1, 2 or 3, wherein the storage server status abnormality includes at least an addition, deletion, and hard disk failure or expansion of the storage server.

The method according to claim 4, wherein the monitoring, by the metadata server, that the hard disk failure comprises:

After receiving the hard disk abnormality information, the metadata server monitors the hard disk failure.

The method of claim 1, wherein when the metadata server monitors that the storage server status is abnormal, initiating a fragment migration request to the storage server includes:

When the metadata server monitors that the storage server is abnormal, and the abnormal time is greater than the predetermined threshold At this time, a fragment migration request is initiated to the storage server.

7. A distributed file system for optimizing load balancing, comprising: a metadata server, a storage server, and a database server, wherein:

8. The system according to claim 7, wherein the database server is further configured to: receive a file write request sent by the client, write a file according to the file write request, and assign each file to each file The film is tagged.

9. The system according to claim 8, wherein the database server is further configured to: when the first file is written, label the file 1 , and each time a file is added, the label is incremented by 1 when the label is added. Recycled and tagged when it reaches 65535.

The system according to claim 7, 8 or 9, wherein the storage server status abnormality includes at least an addition, deletion, and hard disk failure or expansion of the storage server.

11. The system of claim 10, wherein

The storage server is further configured to: send a hard disk abnormality information to the metadata server when detecting a hard disk failure;

The system according to claim 7, wherein the metadata server is further configured to: initiate a fragment migration request to the storage server when the storage server status is abnormal and the abnormal time is greater than a predetermined threshold.