WO2017084563A1

WO2017084563A1 - Distributed system-based file reading method and device

Info

Publication number: WO2017084563A1
Application number: PCT/CN2016/105957
Authority: WO
Inventors: 谢晓芹
Original assignee: 华为技术有限公司
Priority date: 2015-11-19
Filing date: 2016-11-15
Publication date: 2017-05-26
Also published as: CN105426483B; CN105426483A

Abstract

A distributed system-based file reading method and device. The method comprises: determining, according to an offset address and a reading data quantity carried in a file reading request sent by a client side, at least one target data block the client side needs to read and a target data stripe to which the target data block belongs; when the at least one target data block is partial data in the target data stripe, sending to a lock server a data stripe version number acquisition request in respect of the target data stripe; sending a data block acquisition request to each storage server for storing the target data block; and when a data stripe version number fed back by the lock server is identical to a data block version number corresponding to the target data block fed back by each storage server, sending the target data block to the client side. By adopting the method, when a data block a client side needs to read is partial data in a data stripe, the whole data stripe is prevented from being read in a file system, and system performance is effectively increased.

Description

File reading method and device based on distributed system

Technical field

The present invention relates to the field of communications technologies, and in particular, to a file reading method and apparatus based on a distributed system.

Background technique

The distributed system may include a file interface server, a storage server, and a lock server for implementing a file interface service, a storage service, and a lock service, wherein the file interface server is configured to process a network file protocol, complete metadata processing and file read and write operations, and store The server is used to complete the storage of files and metadata; the lock server is used to complete the management of distributed locks. In a distributed system, in order to improve the reliability of file data and improve the utilization of storage space, the file data is stored in N+M protection mode, in which the file data is divided into at least N data blocks and passed through EC (Erasure). Code, data check erasure code) algorithm obtains M check data blocks. It is known that N+M storage servers form an N+M data protection group through the network, and then N data blocks and M check data. The block is stored in the data protection group. A file may include at least one data stripe, and a data stripe may include at least one data block and at least one check data block, the data striping is written by the Quorum transaction mechanism to ensure consistency, and each data in the storage server Blocks have a version number. Under normal circumstances, the version numbers of N+M data blocks in a data strip are the same.

Traditionally, after receiving the file read request sent by the client, the file interface server reads the data block from at least N storage servers, and the storage server returns the data block and the version number to the file interface server, and the file interface server according to the Quorum transaction The mechanism is configured to determine whether the version numbers of the at least N data blocks are the same. When the version numbers of the at least N data blocks are the same, determine that the data stripe is valid, and then cache the read N data blocks, and cache the N data blocks. The data block required by the client is sent to the client in the data block. The limited cache capacity causes the data block cached by the file interface server to be deleted quickly. Each time the client sends a file read request, the storage server needs to read at least N data blocks and their version numbers through the storage server, and the storage server reads the data pressure. Increased, resulting in client response delay, system performance is low.

Summary of the invention

The present invention provides a file reading method and device based on a distributed system. When a data block that a client needs to read is part of data in a data stripe, the file system can be prevented from reading the entire data stripe and effectively improving. System performance.

The first aspect provides a file reading method based on a distributed system, the method is applied to a file interface server, and the file interface server communicates with a client, a lock server, and a storage server, and the method includes:

Receiving a file read request sent by the client, and the file read request carries an offset address and a read data amount;

Determining, according to the offset address and the amount of read data, at least one target data block that the client needs to read and at least one target data segment to which the target data block belongs, and the file corresponding to the file handle includes at least one data stripe, each data segment The bar includes at least one data block;

When at least one target data block is part of the data in the target data stripe, sending a data stripe version number acquisition request for the target data stripe to the lock server, and receiving the lock server response data stripe version number obtaining request feedback Data striping version number;

Sending a data block acquisition request to each storage server storing the target data block, and receiving the target data block and the corresponding data block version number fed back by each storage server in response to the data block acquisition request;

When the data stripe version number is the same as the data block version number corresponding to each target data block, each target data block is sent to the client.

In the technical solution, the lock server can cache the data stripe version number of each data stripe. If the client needs to read part of the data in the target data stripe, the file interface server can obtain the target data strip by using the lock server. The data striping version number, the file interface server can also determine the target data block required by the client, obtain the target data block and the data block version number corresponding to the target data block through the storage server storing the target data block, when the target data block corresponds When the data block version number and the data stripe version number are the same, the file interface server can determine that the target data block is valid and send the target data block to the client. In the conventional file reading method, the file interface server needs to acquire all the data blocks in the target data strip by the storage server storing the data block in the target data stripe, regardless of whether the client requests part of the data or all the data of the target data stripe. And the data block version number corresponding to each data block. When each data block version number is the same, the file interface server may determine that each data block is valid, and send the data block required by the client in the data block to the client. Client in this application When the data block is part of the data in the target data strip, the file interface server obtains the target data block and the data block version number corresponding to the target data block through the storage server, and the data block version number and the data stripe version corresponding to the target data block. When the number is the same, the file interface server can determine that the target data block is valid, and it is not necessary to read all the data blocks in the target data stripe, which can effectively improve system performance.

Taking the protection diagram of the file storage shown in FIG. 2 as an example, in the conventional file reading method, when the data that the client needs to read is the data block 1-1, the file interface server determines the data to which the data block 1-1 belongs. The stripe is data strip 1, the data strip 1 includes a data block 1-1, a data block 1-2, and a data block 1-3, and the file interface server sends a data block acquisition request to the storage server 1 storing the data block 1-1. And receiving the data block 1-1 fed back by the storage server 1 and the data block version number corresponding to the data block 1-1. Similarly, the file interface server sends a data block acquisition request to the storage server 2 storing the data block 1-2, and Receiving the data block 1-2 and the data block version number corresponding to the data block 1-2 fed back by the storage server 2, the file interface server sends a data block acquisition request to the storage server 3 storing the data block 1-3, and receives the feedback from the storage server 3. Data block 1-3 and the data block version number corresponding to the data block 1-3, when the data block version number corresponding to the data block 1-1, the data block version number corresponding to the data block 1-2, and the data block 1-3 correspond to When the data block version numbers are the same, 1-1 transmits the data block to the client.

In the present application, when the data that the client needs to read is 1-1, the file interface server may send a data stripe version number acquisition request for the data stripe 1 to the lock server, and receive the data stripe fed back by the lock server. The data stripping version number of the data interface server may also send a data block acquisition request to the storage server 1 storing the data block 1-1, and receive the data block 1-1 and the data block 1-1 corresponding to the storage server 1 The data block version number is sent to the client when the data stripe version number of the data strip 1 and the data block version number corresponding to the data block 1-1 are the same. Compared with the traditional file reading method, when the data block that the client needs to read is part of the data stripe, the file interface server does not need to read all the data blocks in the data stripe, which can effectively improve system performance.

Wherein, the offset address is an offset from the first address (ie, the start address of the file) in the file.

The at least one target data block is part of the data in the target data stripe, that is, the data quantity of the at least one target data block is smaller than the data quantity of the target data stripe, and the protection map of the file storage shown in FIG. 2 is taken as an example, at least When one target data block is data block 1-1, data block 1-1 is partial data in data strip 1; when at least one target data block is data block 1-1 and data block 1-2, data block 1 1 and data block 1-2 are partial data in data strip 1; at least one target data block is data block 1-1, In the case of the data block 1-2 and the data block 1-3, the data block 1-1, the data block 1-2, and the data block 1-3 are all data in the data stripe 1.

In the above technical solution, optionally, if the client needs to read all the data in the target data stripe, all the data blocks in the target data stripe can be read by a conventional file reading method.

In the above technical solution, optionally, after the file interface server receives the target data block and the corresponding data block version number fed back by each storage server in response to the data block acquisition request, if the data stripe version number corresponds to each target data block, The data block version numbers are not identical, and all data blocks in the file can be read by the conventional file reading method.

In the above technical solution, optionally, the file interface server may receive a file write request sent by the client, and the file write request carries a file handle, an offset address, a write data, and a write data amount; and the file handle is obtained by using the version number allocator. The data striping version number of the file; determining the data block to which the write data belongs and the data strip to which the data block belongs according to the offset address and the amount of write data; and sending the write data to a storage server storing the data block to which the write data belongs; When the write data is successfully sent, the data stripe version number is sent to the lock server to notify the lock server to update the data stripe version number of the data stripe.

In a specific implementation, the version number allocator can maintain the data stripe version number of each data stripe in the file interface server. The user can initiate a file write request to the file interface server through the client, and the file write request can carry the file handle, the offset address, the write data, and the write data amount, and the file interface server obtains the file corresponding to the file handle in the local cache through the file handle. File attributes, where file attributes can include: the total file size of the file, the data EC protection level, and the storage path of each data block. The file interface server can obtain the data block to which the write data belongs by using the total data amount, the offset address, and the write data amount, and send the write data to the storage server storing the data block according to the storage path of the data block to which the write data belongs. After the storage server receives the write data, the data block is updated. After the file interface server determines that the write data is successfully sent, the data stripe version number of the data strip obtained by the version number allocator may be sent to the lock server. The lock server may update the data stripe version number of the data stripe to ensure that the data stripe version number of the lock server cache is the latest data stripe version number, so that the file interface server receives the file read request sent by the client. Determine whether the target data block is valid by comparing whether the data stripe version number of the lock server cache is the same as the data block version number corresponding to the target data block, and the data stripe version number is the latest data stripe version number, and the file can be upgraded. The interface server determines the accuracy of whether the target data block is valid.

In the above technical solution, optionally, after the file interface server reads all the data blocks in the file by using the traditional file reading method, when the data block version numbers corresponding to the respective data blocks obtained by the storage server are the same, the file The interface server can determine that each data block is valid, and then the data block version number corresponding to the data block is used as the data stripe version number of the data strip to which each data block belongs, and the file interface server can send the data stripe version number to the lock server, and the lock The server can update the data stripe version number of the data stripe to ensure that the data striping version number of the lock server cache is the latest data stripe version number, which can improve the accuracy of the file interface server for determining whether the data block is valid.

In the above technical solution, optionally, the file interface server sends a read lock acquisition request to the lock server, the read lock acquisition request carries the data stripe identifier of the target data stripe, and the file interface server can receive the lock server response to the read lock acquisition request. The read read authorization response message corresponding to the target data stripe corresponding to the data stripe identifier, wherein the read grant response message may carry the data stripe version number of the target data stripe.

In a specific implementation, before the file interface server reads the file through the storage server, it needs to send a read lock acquisition request to the lock server, and if the lock server determines that the data barrier is corresponding to the data stripe, the read lock or the write lock is not used by other file interface servers. If it is held, the read authorization response message of the data stripe is fed back, and the file interface server can send a data block acquisition request to the storage server. In addition, when the data required by the client is part of the data stripe, the file interface server needs to obtain the data stripe version number of the data stripe through the lock server, and divide the data stripe version number with the data block of the target data block. The version numbers are compared to determine if the target data block is valid. In order to improve the data transmission efficiency between the file interface server and the lock server, when the file interface server sends a read lock acquisition request to the lock server, the lock server may carry the data packet in the read authorization response message fed back in response to the read lock acquisition request. The data striping version number.

In the above technical solution, optionally, the file interface server sends a lock release request to the lock server, and the lock release request may carry the data stripe identifier and the data stripe version number to notify the lock server to update the data score corresponding to the data stripe identifier. The data striping version number of the strip, and receiving the lock release response message fed back by the lock server in response to the lock release request.

In a specific implementation, after the file interface server reads the file or writes the file through the storage server, the read lock or the write lock may be actively released, that is, the file interface server may send a lock release request to the lock server, and the lock server responds to the lock release request, and Send a lock release response to the file interface server Message. In addition, after the file interface server writes the file through the storage server or reads all the data blocks in the data stripe, the data stripe version number of the data stripe can be sent to the lock server to ensure the data stripe version of the lock server cache. No. is the latest data striping version number. In order to improve the data transmission efficiency between the file interface server and the lock server, when the file interface server sends a lock release request to the lock server, the latest data stripe version number may be sent to the lock release request carrying the data stripe version number. Lock the server.

In the above technical solution, optionally, the file interface server may receive a lock recall request sent by the lock server, the lock recall request carries a data stripe identifier, and the file interface server may send a lock recall response message to the lock server in response to the lock recall request, the lock The recall response message carries the data stripe version number of the data stripe corresponding to the data stripe identifier, to notify the lock server to update the data stripe version number of the data stripe corresponding to the data stripe identifier.

In a specific implementation, the file interface server holds a read lock or a write lock on the data stripe. If another file interface server sends a lock acquisition request to the lock server, the lock server may send a lock recall request to the file interface server, and the file interface server After reading the file or writing the file, the lock recall response message may be sent to the lock server in response to the lock recall request, so that the lock server authorizes the lock of the data strip to the other file interface server. In addition, after the file interface server writes the file through the storage server or reads all the data blocks in the data stripe, the data stripe version number of the data stripe can be sent to the lock server to ensure the data stripe version of the lock server cache. No. is the latest data striping version number. In order to improve the data transmission efficiency between the file interface server and the lock server, when receiving the lock recall request sent by the lock server, the file interface server may carry the data stripe version number in the lock recall response message fed back in response to the lock recall request. In order to send the latest data stripe version number to the lock server.

In the above technical solution, optionally, the file read request may further carry a file handle, and the file interface server locally obtains the file attribute of the corresponding file according to the file handle, and the file attribute includes a storage path of each data block in the file, according to each target data. The storage path of the block determines a storage server of each storage target data block, and sends a data block acquisition request to each determined storage server.

The file handle is used to identify the file, and the different file handles correspond to different files, and the file interface server can locally obtain the file attribute of the corresponding file according to the file handle. The file attribute can include the total amount of data of the file, the data EC protection level (ie, the data block in each data stripe) The number N and the number of check data blocks M) or the storage path of the data block, and the like. The file interface server may determine a storage server of each storage target data block according to a storage path of the target data block, and then send a data block acquisition request to each determined storage server.

In the foregoing technical solution, optionally, the file interface server may pre-establish a correspondence between the data block and the storage server. For example, when the storage server storing the data block 1-1 is the storage server 1, the file interface server may establish the data block 1 - 1 Correspondence relationship with the storage server 1. When the at least one target data block is part of the data in the target data stripe, the file interface server may determine the storage server corresponding to the target data block according to the correspondence between the pre-established data block and the storage server, and send the storage server to the determined storage server. The data block gets the request.

A second aspect provides a file reading apparatus based on a distributed system, where the apparatus may include a request receiving unit, a data block determining unit, a request sending unit, a version number obtaining unit, a data block receiving unit, and a data block sending unit, The device may be used to implement some or all of the steps in connection with the first aspect.

The third aspect provides a server, including a processor and a memory, which may be used to implement some or all of the steps in conjunction with the first aspect.

DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present invention. Other drawings may also be obtained from those of ordinary skill in the art in light of the inventive work.

1 is a schematic diagram of a framework of a distributed system provided in an embodiment of the present invention;

2 is a schematic diagram of protection of file storage provided in an embodiment of the present invention;

3 is a schematic flowchart of a file reading method based on a distributed system according to an embodiment of the present invention;

FIG. 4 is a schematic flowchart diagram of a file reading method based on a distributed system according to another embodiment of the present invention; FIG.

FIG. 5 is a schematic flowchart diagram of a file reading method based on a distributed system according to another embodiment of the present invention; FIG.

6 is a file reading method based on a distributed system according to another embodiment of the present invention. Schematic diagram of the process;

7 is a schematic structural diagram of a file interface server according to an embodiment of the present invention;

FIG. 8 is a schematic structural diagram of a file reading apparatus based on a distributed system according to an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention will be clearly described below with reference to the accompanying drawings in the embodiments of the present invention.

Referring to FIG. 1 , FIG. 1 is a schematic diagram of a framework of a distributed system according to an embodiment of the present invention. The distributed system in the embodiment of the present invention may include at least a client, a file interface server, a storage server, and Lock the server. The client can establish a communication connection with the file interface server through the network, and the file interface server establishes a communication connection with the lock server and the storage server respectively. At least two storage servers can form a cluster of storage servers over the network.

The client is configured to send a file read request to the file interface server, and the file read request carries the offset address and the amount of read data.

a file interface server, configured to determine, according to the offset address and the amount of read data, at least one target data block that the client needs to read and a target data strip to which the at least one target data block belongs, wherein one file includes at least one data stripe Each data strip includes at least one data block; when the at least one target data block is part of the target data stripe, the data stripe version number acquisition request for the target data stripe is sent to the lock server.

The lock server is configured to send a data stripe version number to the file interface server in response to the data stripe version number acquisition request.

The file interface server is further configured to: when the at least one target data block is part of the data in the target data stripe, send a data block acquisition request to each storage server storing the target data block.

And a storage server, configured to send the target data block and the data block version number corresponding to the target data block to the file interface server in response to the data block acquisition request.

The file interface server is further configured to send each target data block to the client when the data stripe version number sent by the lock server is the same as the data block version number corresponding to the target data block sent by each storage server.

Specifically, in a distributed system, in order to improve the reliability of the file, the storage server is saved at the same time. The utilization of the storage space, the file storage adopts the N+M protection mode, that is, the N+M storage servers form an N+M data protection group through the network, and the file is divided into at least N data blocks, and the EC algorithm is used. The data block is processed to obtain M check data blocks, and N data blocks and M check data blocks are stored in N+M storage servers. The N+M data blocks of a file are called a data stripe. When the data strips are written to the storage server, the consistency is maintained by the Quprum transaction mechanism, and in the storage server, each data block corresponds to one data block. Version number. Under normal circumstances, the data block version numbers corresponding to N+M data blocks in one data stripe are the same. The check data block is used by the file interface server to determine that the data stripe is invalid, and the check data block is processed by the EC algorithm to generate a data block.

Taking the protection diagram of the file storage shown in FIG. 2 as an example, the file system includes five storage servers, where N is 3 and M is 2. If a file is divided into 6 data blocks, the file may include 2 data. For the stripe, the data block identifier of each data block may be a stripe number-data block number, for example, the data block identifiers of the above data blocks are 1-1, 1-2, 1-3, 2-1, 2-2, and 2-3, processing the data blocks 1-1, 1-2, and 1-3 by the EC algorithm to obtain the check data block 1 and the check data block 2, wherein the data blocks 1-1, 1-2, and 1-3 The check data block 1 and the check data block 2 form a data stripe 1, and the data block 1-1 and its corresponding data block version number can be stored in the storage server 1, and the data block 1-2 and its corresponding The data block version number is stored in the storage server 2, and the data block 1-3 and its corresponding data block version number are stored in the storage server 3, and the check data block 1 and its corresponding data block version number are stored in the storage server. In 4, the check data block 2 and its corresponding data block version number are stored in the storage server 5. Similarly, the data blocks 2-1, 2-2, and 2-3 are processed by the EC algorithm to obtain a check data block 3 and a check data block 4, where the data blocks 2-1, 2-2, and 2-3, The check data block 3 and the check data block 4 form a data stripe 2, and the data block 2-1 and its corresponding data block version number can be stored in the storage server 1, and the data block 2-2 and its corresponding data can be stored. The block version number is stored in the storage server 2, the check data block 1 and its corresponding data block version number are stored in the storage server 3, and the check data block 2 and its corresponding data block version number are stored in the storage server 4. The data block 2-3 and its corresponding data block version number are stored in the storage server 5.

In an optional embodiment, the file interface server is further configured to send a data block acquisition request to the storage server of each storage target data block when the at least one target data block is all data in the target data strip.

The storage server is further configured to send the target data block and its corresponding data block version number to the file interface server in response to the data block acquisition request.

The file interface server is further configured to send each target data block to the client when the data block version numbers corresponding to the respective target data blocks are the same.

Further, the file interface server is further configured to send a check data block acquisition request to each storage server that stores the check data block when the data block version numbers corresponding to the target data blocks are not completely the same.

The storage server is further configured to send the check data block and the corresponding data block version number to the file interface server in response to the check data block acquisition request.

The file interface server is further configured to process each check data block by using a data check erasure code EC algorithm when the data block version numbers corresponding to the check data blocks are the same, and obtain all the data blocks in the target data stripe. The target data block is determined in all the generated data blocks, and each target data block is sent to the client.

In an optional embodiment, the file interface server is further configured to: when the data block version numbers corresponding to the target data blocks are the same, the data block version number corresponding to the target data block is used as the data stripe version number of the target data stripe. Send the data stripe version number to the lock server.

The lock server is further configured to update the data stripe version number of the target data stripe.

In an optional embodiment, the file interface server is further configured to: when the data stripe version number and the data block version number corresponding to each target data block are not completely the same, send to the storage server of the data block in each storage target data stripe The data block gets the request.

The storage server is further configured to send the data block and its corresponding data block version number to the file interface server in response to the data block acquisition request.

The file interface server is further configured to: when the data block version numbers corresponding to the respective data blocks are the same, determine each target data block in the received data block, and send each target data block to the client.

In an optional embodiment, the file interface server is further configured to receive a file write request sent by the client, where the file write request carries a file handle, an offset address, a write data, and a write data amount; and the file handle is obtained by using a version number allocator. The data striping version number of the file; determining the data block to which the write data belongs and the data strip to which the data block belongs according to the offset address and the amount of write data; and transmitting the write data to a storage server storing the data block to which the write data belongs.

The storage server is also used to update the data block to which the write data belongs.

The file interface server is further configured to send the data stripe version number to the lock server when the write data is successfully sent.

The lock server is also used to update the data stripe version number of the data stripe.

In an optional embodiment, the file interface server is configured to send a lock release request to the lock server, where the lock release request carries the data stripe identifier and the data stripe version number.

The lock server is further configured to update the data stripe version number of the data stripe corresponding to the data stripe identifier, and send a lock release response message to the file interface server in response to the lock release request.

In an optional embodiment, the lock server is further configured to send a lock recall request to the file interface server, where the lock recall request carries the data stripe identifier.

The file interface server is further configured to send a lock recall response message to the lock server in response to the lock recall request, and the lock recall response message carries the data stripe version number of the data stripe corresponding to the data stripe identifier.

The lock server is further configured to update the data stripe version number of the data stripe corresponding to the data stripe identifier.

In an optional embodiment, the file interface server is configured to send a read lock acquisition request to the lock server, where the read lock acquisition request carries the data stripe identifier of the target data strip.

The lock server is further configured to send a read authorization response message to the file interface server to the target data stripe corresponding to the data stripe identifier in response to the read lock acquisition request, where the read authorization response message carries the data stripe version number of the target data stripe.

In an optional embodiment, the file read request may further carry a file handle, and the file interface server sends a data block acquisition request to the storage server storing each target data block, where the file attribute is locally obtained according to the file handle, and the file attribute is used. The storage path of each data block in the file is included; the storage server storing each target data block is determined according to the storage path of each target data block; and the data block acquisition request is sent to each determined storage server.

In the distributed system shown in FIG. 1, the client sends a file read request to the file interface server, the file read request carries the offset address and the read data amount, and the file interface server determines the client according to the offset address and the read data amount. At least one target data block that needs to be read by the end and a target data strip to which the at least one target data block belongs. When at least one target data block is part of the data in the target data stripe, the file interface server sends the target data to the lock server. Striped data stripe version number obtained The request is received, and the lock server responds to the data stripe version number to obtain the data stripe version number fed back by the request, and the file interface server sends a data block acquisition request to each storage server storing the target data block, and receives each storage server response data block. Obtaining the target data block and the corresponding data block version number that are requested by the request, when the data stripe version number and the data block version number corresponding to each target data block are the same, the file interface server sends each target data block to the client. When the data block that the client needs to read is part of the data stripe, the file system can be prevented from reading the entire data stripe internally, thereby effectively improving system performance.

Referring to FIG. 3, FIG. 3 is a schematic flowchart of a file reading method based on a distributed system according to an embodiment of the present invention, where the file reading method based on the distributed system in the embodiment of the present invention is at least Can include:

S301. The client sends a file read request to the file interface server, where the file read request carries an offset address and a read data amount.

S302. The file interface server determines, according to the offset address and the read data quantity, at least one target data block that the client needs to read and a target data strip to which the at least one target data block belongs.

In a specific implementation, the file interface server may obtain the file attribute of the file in the local cache, and the file attributes may include: a total file amount of the file, a data EC protection level, and a storage path of each data block in the file. In turn, the file interface server can determine at least one target data block that the client needs to read according to the offset address, the amount of read data, the total amount of data of the file, and the data EC protection level.

For example, the file interface server determines that the total file size of the file is 30 MB (megabytes), and the data EC protection level is 3+2 (that is, each data strip of the file includes 3 data blocks and 2 check data blocks). If the data volume of each data block is 512 KB, the number of data strips included in the file is: 30*1024/(512*3)=20, when the offset address is 5 MB, the read data amount is 1 MB, The file interface server may determine that the target data block that the client needs to read is the data block 2 and the data block 3 in the data strip 4. The data block identifier of each data block may be: a strip number - a data block number, such as a target The data block identifiers for the data blocks are 4-2 and 4-3.

In an optional embodiment, the file interface server may obtain the data stripe identifier of the target data strip to which the at least one target data block belongs according to the offset address, the read data amount, and the stripe data amount of the data stripe. Wherein, one file may include at least one data stripe, and each data stripe may include at least one data block. Taking the protection diagram of the file storage shown in FIG. 2 as an example, the file includes data stripe 1 and data stripe 2, and data stripe 1 includes data blocks 1-1, 1-2, and 1-3, and data stripe 2 Includes data blocks 2-1, 2-2, and 2-3. The format of the data stripe identifier may be: file identifier-strip number, the file identifier may be the file ID of the file, and the stripe number may be

Where X is the offset address and Y is the stripe data amount of the data stripe. For example, if the offset address is 1MB and the data volume of the data stripe is 1536 KB, the file interface server can determine that the target data stripe is the data stripe 1 in the file. The method for obtaining the strip number includes but is not limited to the foregoing manner, for example, the strip number may be

Where X is the offset address and Y is the stripe data amount of the data stripe. Exemplarily, the offset address is 1MB, and the data volume of the data stripe is 1536 KB, and the file interface server can determine that the target data stripe is the data stripe 0 in the file. It is not specifically limited by the embodiments of the present invention.

Further, the file interface server may send a read lock acquisition request to the lock server, where the read lock acquisition request carries the data stripe identifier, and the lock server may determine whether the other file interface server holds the data stripe corresponding to the data stripe identifier. The read lock or the write lock, when the other file interface server does not hold the read lock and the write lock of the data stripe corresponding to the data stripe identifier, the lock server may send the data score corresponding to the data stripe identifier to the file interface server. The strip reads the authorization response message so that the file interface server obtains the target data block through the storage server.

S303. When at least one target data block is part of data in the target data stripe, the file interface server sends a data stripe version number acquisition request for the target data stripe to the lock server.

Specifically, if the target data block that the client needs to read is part of the data in the target data stripe, the file interface server may send a data stripe version number acquisition request for the target data stripe to the lock server, and the data stripe version number The acquisition request can carry the data stripe identifier. For example, the file interface server divides the file into six data blocks by using the file attribute, and the file includes two data strips, and the data block identifiers of the data block are 1-1, 1-2, 1-3, 2-1, respectively. 2-2 and 2-3, the target data block that the client needs to read is the data block 1-1 and the data block 1-2, and the file interface server can determine that the target data block is part of the data in the data stripe 1, and further The file interface server sends a data stripe version number acquisition request for the data strip 1 to the lock server.

For another example, the file interface server divides the file into six data blocks by using the file attribute, and the file includes two data strips, and the data block identifiers of the data block are 1-1, 1-2, 1-3, and 2-1, respectively. 2-2 and 2-3, the target data block that the client needs to read is data blocks 1-2, 1-3, 2-1, 2-2, and 2-3, and the file interface server can determine the data block 1-2 and the data block. 1-3 is part of the data in the data stripe 1, and the file interface server sends a data stripe version number acquisition request for the data stripe 1 to the lock server. Data blocks 2-1, 2-2, and 2-3 are all data in data stripe 2, and the file interface server can obtain data blocks 2-1, 2-2, and 2-3 through a conventional file reading method. The obtained data blocks 2-1, 2-2, and 2-3 are sent to the client.

S304. The lock server sends a data stripe version number to the file interface server in response to the data stripe version number acquisition request.

In a specific implementation, the lock server may cache the data stripe version number of each data stripe, and update the data stripe version number of the data stripe when the file interface server writes the data or reads the entire data stripe, and the lock server receives the data. After obtaining the request for the data stripe version number sent by the file interface server, the data stripe version number of the target data stripe corresponding to the data stripe identifier may be searched, and the data stripe version number is sent to the file interface server.

It should be noted that, when the lock server fails, the data stripe version number in the cache may be lost, and the lock server cannot find the data stripe version number of the target data stripe corresponding to the data stripe identifier. Or when the lock server is eliminated, the file interface server cannot obtain the data stripe version number of the target data strip through the lock server. Or the lock server is currently running for the first time, and the data stripe version number of each data stripe is not stored in the cache. In the above case, the file interface server cannot obtain the data stripe version number through the lock server, and the file interface server can read the data required by the client through the traditional file reading method.

As an optional manner of steps S303 and S304, the lock server may carry the data stripe version number in the read authorization response message fed back by the read lock acquisition request sent by the response file interface server, so the file interface server does not need to send data to the lock server. The method of obtaining the request by the stripe version number obtains the version number of the data stripe, which improves the data transmission efficiency. In a specific implementation, the file interface server may send a read lock acquisition request to the lock server, the read lock acquisition request carries the data stripe identifier, and the lock server responds to the read lock acquisition request, and may send the data stripe identifier corresponding to the file interface server. The read authorization response message of the file, and the read authorization response message carries the data stripe version number of the file.

S305. The file interface server sends a data block acquisition request to a storage server that stores the target data block.

When the target data block is part of the data stripe, the file interface server may send a data block fetch request to the storage server storing the target data block. For example, if the file interface server determines that the target data block is the data block 1-2 and the data block 1-3, the file interface server may send a data block acquisition request to the storage server storing the data block 1-2, and the data block obtains the request carrying data. The data block identifier of block 1-2. The file interface server may also send a data block acquisition request to the storage server storing the data blocks 1-3, the data block acquisition request carrying the data block identifier of the data blocks 1-3.

In an optional embodiment, after the file interface server obtains the storage path of each data block, the storage server storing the target data block may be determined according to the storage path of the target data block, and then the data block acquisition request is sent to the storage server. For example, the file interface server may determine that the data block 1-1 is stored in the storage server 1 according to the storage path of each data block, the data block 1-2 is stored in the storage server 2, and the data block 1-3 is stored in the storage server 3, When the target data block is the data block 1-2 and the data block 1-3, the file interface server may transmit a data block acquisition request to the storage server 2 and send a data block acquisition request to the storage server 3.

S306. The storage server sends the target data block and its corresponding data block version number to the file interface server in response to the data block acquisition request.

The storage server sends the target data block and its corresponding data block version number to the file interface server in response to the data block acquisition request. For example, the file interface server sends a data block acquisition request to the storage server 2, the storage server 2 feeds back the data block 1-2 and the data block version number corresponding to the data block 1-2, and the file interface server can also send the data block acquisition to the storage server 3. The request, the storage server 3 feeds back the data block 1-3 and the data block version number corresponding to the data block 1-3.

S307. When the data stripe version number is the same as the data block version number corresponding to each target data block, the file interface server sends each target data block to the client.

When the data stripe version number obtained by the file interface server through the lock server is the same as the data block version number corresponding to the target data block obtained by each storage server, the file interface server may determine that each target data block is valid, and then Each target data block is sent to the client. For example, if the data stripe version number is the same as the data block version number corresponding to the data block 1-2, and the data stripe version number is the same as the data block version number corresponding to the data block 1-3, the file interface server may determine the data block. 1-2 and data blocks 1-3 are valid, and data blocks 1-2 and data blocks 1-3 are sent to the client.

In the distributed system-based file reading method shown in FIG. 3, at least one required by the client When the target data block is part of the data in the target data strip, the file interface server sends a data stripe version number acquisition request for the target data stripe to the lock server, and the lock server responds to the data stripe version number to obtain the request to the file interface server. Sending a data stripe version number, the file interface server may further send a data block acquisition request to the storage server of each storage target data block, and the storage server sends the target data block and the corresponding data block version number to the file interface server according to the data block acquisition request. When the data stripe version number is the same as the data block version number corresponding to each target data block, the file interface server sends each target data block to the client, and the data block that the client needs to read is part of the data stripe. When the data is used, the entire data stripe can be prevented from being read internally by the file system, thereby effectively improving system performance.

Referring to FIG. 4, FIG. 4 is a schematic flowchart of a method for reading a file according to an embodiment of the present invention. The method for reading a file in the embodiment of the present invention may include at least:

S401. The client sends a file read request to the file interface server, where the file read request carries an offset address and a read data amount.

S402. The file interface server determines, according to the offset address and the read data quantity, the at least one target data block that the client needs to read and the target data strip to which the at least one target data block belongs.

S403. When at least one target data block is all data in the target data stripe, the file interface server sends a data block acquisition request to the storage server storing the target data block.

When the target data block is all data in the target data stripe, the file interface server may send a data block acquisition request to the storage server storing the target data block. For example, all the data blocks in the target data strip are data block 1-1, data block 1-2, and data block 1-3, and the file interface server determines that the target data block is data block 1-1, data block 1-2. And data blocks 1-3, the file interface server may send a data block acquisition request to the storage server storing the data block 1-1, the data block acquisition request carrying the data block identifier of the data block 1-1. The file interface server may also send a data block acquisition request to the storage server storing the data block 1-2, the data block acquisition request carrying the data block identifier of the data block 1-2. The file interface server may also send a data block acquisition request to the storage server storing the data blocks 1-3, the data block acquisition request carrying the data block identifier of the data blocks 1-3.

S404. The storage server sends the target data block and its corresponding data block version number to the file interface server in response to the data block acquisition request.

S405, when the data block version numbers corresponding to the respective target data blocks are the same, the file interface server will Each target data block is sent to the client.

After receiving the data block version number of each target data block, the file interface server may determine whether the data block version numbers of the respective target data blocks are the same. When the data block version numbers corresponding to the respective target data blocks are the same, the file interface server may determine the same. Each data block in the target data strip is valid, and then each target data block is cached, and each target data block is sent to the client.

In an optional embodiment, when the data block version numbers corresponding to the respective target data blocks are not completely the same, the file interface server may determine that the data blocks in the target data stripe are invalid, and then check according to the target data stripe. The storage path of the data block sends a check data block acquisition request to the storage server storing the check data block, and the storage server sends the check data block and the data block corresponding to the check data block to the file interface server in response to the check data block acquisition request. The version number, the file interface server can compare whether the data block version numbers corresponding to the respective check data blocks are the same. When the data block version numbers corresponding to the check data blocks are the same, the check data blocks are processed by the EC algorithm to obtain Each data block in the target data stripe, and then each data block is sent to the client.

In an optional embodiment, when the data block version numbers corresponding to the target data blocks are the same, the file interface server may use the data block version number corresponding to the target data block as the data stripe version number of the target data stripe, and The data striping version number is sent to the lock server. When the lock server does not store the data stripe version number of the file, the lock server may cache the data stripe version number and the data stripe identifier corresponding to the target data stripe; When the server has stored the data stripe version number of the target data stripe, the lock server may update the data stripe version number of the target data stripe.

In an optional embodiment, after receiving the target data block sent by the storage server, the file interface server may actively release the read lock of the target data stripe, and the file interface server may strip the version number of the data in the process of releasing the read lock. Send to the lock server to improve the data transfer efficiency between the file interface server and the lock server. For example, the file interface server may send a lock release request to the lock server, the lock release request carries the data stripe identifier and the data stripe version number, and the lock server may identify the data stripe version number of the target data stripe corresponding to the data stripe identifier. The update is made and a lock release response message is sent to the file interface server in response to the lock release request.

In an optional embodiment, when receiving the lock recall request sent by the lock server, the file interface server may carry the data stripe version number in the lock recall response message fed back in response to the lock recall request, thereby improving the file interface server and the lock server. The efficiency of data transfer between. For example, the lock server can be essay The interface server sends a lock recall request, and the lock recall request carries the data stripe identifier. The file interface server can send a lock recall response message to the lock server in response to the lock recall request, and the lock recall response message carries the data stripe version number, and the lock server can The data stripe version number of the target data stripe corresponding to the data stripe identifier is updated.

In the distributed system-based file reading method shown in FIG. 4, when at least one target data block required by the client is all data in the target data stripe, the file interface server stores the storage unit of the target data block. Sending a data block acquisition request, the storage server responds to the data block acquisition request to send the target data block and the corresponding data block version number to the file interface server, and when the data block version numbers corresponding to the respective target data blocks are the same, the file interface server sets each target The data block is sent to the client.

Referring to FIG. 5, FIG. 5 is a schematic flowchart of a file reading method based on a distributed system according to an embodiment of the present invention, where the file reading method based on the distributed system in the embodiment of the present invention is at least Can include:

S501. The client sends a file read request to the file interface server, where the file read request carries an offset address and a read data amount.

S502. The file interface server determines, according to the offset address and the amount of read data, at least one target data block that the client needs to read and a target data strip to which the at least one target data block belongs.

S503. When at least one target data block is part of data in the target data stripe, the file interface server sends a data stripe version number acquisition request for the target data stripe to the lock server.

S504. The lock server sends a data stripe version number to the file interface server in response to the data stripe version number acquisition request.

S505. The file interface server sends a data block acquisition request to a storage server that stores the target data block.

S506. The storage server sends the target data block and its corresponding data block version number to the file interface server in response to the data block acquisition request.

S507: When the data stripe version number and the data block version number corresponding to each target data block are not completely the same, the file interface server sends a data block acquisition request to the storage server of each of the storage target data strips.

S508. The storage server sends a data block to the file interface server in response to the data block acquisition request. The corresponding data block version number.

S509: When the data block version numbers corresponding to the respective data blocks are the same, the file interface server determines each target data block in the received data block.

In an optional embodiment, when the data block version numbers corresponding to the data blocks are the same, the file interface server may use the data block version number corresponding to the data block as the data stripe version number of the target data stripe, and divide the data into The strip version number is sent to the lock server. When the lock server does not store the data stripe version number of the target data stripe, the lock server may cache the data stripe version number and the data stripe identifier corresponding to the target data stripe; when the lock server When the data stripe version number of the target data stripe has been stored, the lock server can update the data stripe version number of the target data stripe.

S510. The file interface server sends each target data block to the client.

In the distributed system-based file reading method shown in FIG. 5, when the data stripe version number and the data block version number corresponding to each target data block are not completely the same, the file interface server branches into each storage target data. The storage server of the data block sends a data block acquisition request, and the storage server sends a data block and a corresponding data block version number to the file interface server according to the data block acquisition request. When the data block version number corresponding to each data block is the same, the file interface server Each target data block is determined in the received data block, and each target data block is sent to the client.

Referring to FIG. 6, FIG. 6 is a schematic flowchart of a file reading method based on a distributed system according to an embodiment of the present invention, where the file reading method based on the distributed system in the embodiment of the present invention is at least Can include:

S601. The client sends a file write request to the file interface server, where the file write request carries a file handle, an offset address, a write data, and a write data amount.

S602. The file interface server obtains a data stripe version number of the file corresponding to the file handle by using a version number allocator.

After receiving the file write request sent by the client, the file interface server may obtain the data stripe version number of the file corresponding to the file handle by using the version number allocator. For example, the current data stripe version number of the file maintained by the version number allocator is 10, and the preset monotonically increasing index is 2, and the file interface server allocates the file after the file interface server receives the file write request. The data striping version number is 12. It should be noted that the monotonically increasing index includes but is not limited to 2, for example, monotonically increasing The number may be 1 or 3, etc., and is not specifically limited by the embodiment of the present invention.

S603. The file interface server determines, according to the offset address and the amount of write data, a data block to which the write data belongs and a data strip to which the data block belongs.

In a specific implementation, the file interface server may obtain the file attribute of the file corresponding to the file handle in the local cache according to the file handle, and the file attribute may include: total file data amount, data EC protection level, and storage path of each data block in the file. In turn, the file interface server can determine the data block to which the write data belongs according to the offset address, the amount of write data, the total amount of data of the file, and the level of data EC protection. The number of data blocks to which the write data belongs may be at least one.

For example, the file interface server determines that the total file size of the file is 30 MB, and the data EC protection level is 3+2. If the data volume of each data block is 512 KB, the offset address is 5 MB, and the write data volume is 512 KB, the file interface server It can be determined that the data block to which the write data belongs is 4-2.

In an optional embodiment, the file interface server may obtain the data stripe identifier of the data strip to which the data block belongs according to the offset address and the stripe data amount of the data stripe, where the format of the data stripe identifier may be: file identifier Strip number, file ID can be the file ID of the file, and the strip number can be

Where X is the offset address and Y is the stripe data amount of the data stripe.

Further, the file interface server may send a write lock acquisition request to the lock server, and the write lock acquisition request carries the data stripe identifier, and the lock server may determine whether the other file interface server holds the data stripe corresponding to the data stripe identifier. The read lock or the write lock, when the other file interface server does not hold the read lock and the write lock of the data stripe corresponding to the data stripe identifier, the lock server may send the data score corresponding to the data stripe identifier to the file interface server. The write authorization message of the strip, so that the file interface server sends the write data to the storage server.

S604. The file interface server sends the write data to a storage server that stores the data block to which the write data belongs.

After the file interface server determines the data block to which the write data belongs, the write data may be sent to the storage server storing the data block to which the write data belongs according to the storage path of the data block. For example, the data block to which the write data belongs is 4-2, and the file interface server determines that the data block 4-2 is stored in the storage server 1 according to the storage path of the data block 4-2, and the file interface server can send the write data to the storage server 1 .

In an optional embodiment, the file interface server may process the write data by using an EC algorithm to obtain a check data block, and send the write data, the check data block, and the obtained data stripe version number to The storage server can update the data block to which the write data belongs, and update the check data block, and then update the data block version number corresponding to each data block in the data stripe to the data stripe version. number.

S605: When the write data is successfully sent, the file interface server sends the data stripe version number to the lock server.

When the write data is successfully sent, the file interface server may send the obtained data stripe version number to the lock server. Further, the file interface server may further send the data stripe identifier of the data strip to which the data block belongs to the lock server. For example, after the file interface server determines that the write data sent to the storage server is sent, the obtained data stripe version number may be sent to the lock server to notify the lock server to update the data stripe version number of the data stripe. For another example, after the file interface server sends the write data to the storage server, the storage server sends the write data to the file interface server to successfully receive the message, and the file interface server can successfully receive the message according to the write data to determine that the write data is successfully sent, and then the acquired data. The stripe version number is sent to the lock server.

S606. The lock server updates the data stripe version number of the data stripe.

In the distributed system-based file reading method shown in FIG. 6, the client sends a file write request to the file interface server, and the file write request carries the file handle, the offset address, the write data, and the write data amount, and the file interface server passes The version number allocator obtains the data stripe version number of the file corresponding to the file handle, and the file interface server determines, according to the offset address and the amount of write data, the data block to which the write data belongs and the data strip to which the data block belongs, and the file interface server writes The data is sent to the storage server storing the data block to which the write data belongs. When the write data is successfully sent, the file interface server sends the data stripe version number to the lock server, and the lock server updates the data stripe version number of the data strip to which the data block belongs. It ensures the validity of the data stripe version number stored by the lock server.

Referring to FIG. 7, FIG. 7 is a schematic structural diagram of a file interface server according to an embodiment of the present invention. As shown in FIG. 7, the file interface server may include a processor 701, a memory 702, a first network interface 703, a second network interface 704, and a third network interface 705. The processor 701 is connected to the memory 702, the first network interface 703, the second network interface 704, and the third network interface 705. For example, the processor 701 can be connected to the memory 702, the first network interface 703, the second network interface 704, and the Third network interface 705.

The processor 701 may be a central processing unit (CPU), a network processor (NP), or the like.

The memory 702 can be specifically configured to store a data block, a data block version number corresponding to the data block, and the like. The memory 702 may include a volatile memory such as a random-access memory (RAM); the memory may also include a non-volatile memory such as a read-only memory (read- Only memory, ROM), flash memory, hard disk drive (HDD) or solid-state drive (SSD); the memory may also include a combination of the above types of memory.

The first network interface 703 is configured to communicate with the client, for example, to receive a file read request sent by the client. The first network interface 703 can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), and the like.

The second network interface 704 is configured to communicate with the lock server, for example, send a data stripe version number acquisition request for the target data stripe to the lock server. The second network interface 704 can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), and the like.

The third network interface 705 is configured to communicate with the storage server, for example, send a data block acquisition request to the storage server. The third network interface 705 can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), and the like.

Specifically, the file interface server introduced in the embodiment of the present invention may be used to implement some or all of the processes in the embodiment of the distributed system-based file reading method introduced in conjunction with FIG. 3 to FIG.

Referring to FIG. 8, FIG. 8 is a schematic structural diagram of a file reading apparatus based on a distributed system according to an embodiment of the present invention, wherein the file reading apparatus based on the distributed system provided by the embodiment of the present invention can be combined with FIG. The processor 701 in the embodiment of the present invention, as shown in the figure, may include at least a request receiving unit 801, a data block determining unit 802, a request sending unit 803, a version number obtaining unit 804, A data block receiving unit 805 and a data block transmitting unit 806, wherein:

The request receiving unit 801 is configured to receive a file read request sent by the client, where the file read request carries an offset address and a read data amount.

a data block determining unit 802, configured to determine, according to the offset address and the amount of read data, the client needs At least one target data block to be read and a target data strip to which the at least one target data block belongs, one file includes at least one data stripe, and each data strip includes at least one data block.

The request sending unit 803 is configured to send, when the at least one target data block is part of the data in the target data stripe, a data stripe version number acquisition request for the target data stripe to the lock server.

The version number obtaining unit 804 is configured to receive a data stripe version number fed back by the lock server in response to the data stripe version number obtaining request.

The request sending unit 803 is further configured to send a data block acquisition request to each storage server storing the target data block.

The data block receiving unit 805 is configured to receive the target data block and the corresponding data block version number fed back by the respective storage server in response to the data block acquisition request.

The data block sending unit 806 is configured to send each target data block to the client when the data stripe version number and the data block version number corresponding to each target data block are the same.

In an optional embodiment, the request sending unit 803 is further configured to: when the at least one target data block is all the data in the target data stripe, send the data block obtaining request to the storage server of each storage target data block.

The data block receiving unit 805 is further configured to receive each target data block and each corresponding data block version number that are fed back by the respective storage server in response to the data block acquisition request.

The data block sending unit 806 is further configured to send each target data block to the client when the data block version numbers corresponding to the respective target data blocks are the same.

In an optional embodiment, the request sending unit 803 is further configured to: when the data stripe version number and the data block version number corresponding to each target data block are not completely the same, store the data block in each storage target data stripe. Send a data block get request.

The data block receiving unit 805 is further configured to receive each data block and each corresponding data block version number fed back by the respective storage server in response to the data block acquisition request.

The data block determining unit 802 is further configured to determine each target data block in the received data block when the data block version numbers corresponding to the respective data blocks are the same.

The data block sending unit 806 is further configured to send each target data block to the client.

In an optional embodiment, the request receiving unit 801 is further configured to receive a file write request sent by the client, where the file write request carries a file handle, an offset address, a write data, and a write data amount.

The version number obtaining unit 804 is further configured to obtain a data stripe version number of the file corresponding to the file handle by using the version number allocator.

The data block determining unit 802 is further configured to determine, according to the offset address and the amount of write data, a data block to which the write data belongs and a data strip to which the data block belongs.

Further, the file reading apparatus based on the distributed system in the embodiment of the present invention may further include:

The data transmitting unit 807 is configured to send the write data to a storage server that stores the data block to which the write data belongs.

The version number sending unit 808 is configured to send the data stripe version number to the lock server when the write data is successfully sent, to notify the lock server to update the data stripe version number of the data stripe.

In an alternative embodiment, the reading device of the file in the embodiment of the present invention may further include:

The version number sending unit 808 is configured to use the data block version number corresponding to the target data block as the data stripe version number of the target data strip when the data block version numbers corresponding to the target data blocks are the same, and the data stripe version The number is sent to the lock server to notify the lock server to update the data stripe version number of the target data strip.

Further, the version number sending unit 808 is specifically configured to:

Sending a lock release request to the lock server, the lock release request carrying the data stripe identifier and the data stripe version number, to notify the lock server to update the data stripe version number of the data stripe corresponding to the data stripe identifier.

The lock release server responds to the lock release response message fed back by the lock release request.

Further, the version number sending unit 808 is specifically configured to:

Receiving a lock recall request sent by the lock server, and the lock recall request carries a data stripe identifier.

Responding to the lock recall request, sending a lock recall response message to the lock server, the lock recall response message carrying the data stripe version number of the data stripe corresponding to the data stripe identifier, to notify the lock server to update the data stripe data corresponding to the data stripe identifier Striped version number.

In an optional embodiment, the request sending unit 803 sends a data stripe version number acquisition request for the target data striping to the lock server, specifically for:

A read lock acquisition request is sent to the lock server, and the read lock acquisition request carries the data stripe identifier of the target data strip.

The number of targets corresponding to the data stripe identifier fed back by the receiving lock server in response to the read lock acquisition request According to the striped read authorization response message, the read authorization response message carries the data stripe version number of the target data stripe.

In an optional embodiment, the file read request may further carry a file handle, and the request sending unit 803 sends a data block acquisition request to the storage server storing each target data block, specifically for:

The file attribute of the file is obtained locally according to the file handle, and the file attribute includes the storage path of each data block in the file.

A storage server that stores each target data block is determined according to a storage path of each target data block.

A data block acquisition request is sent to each of the determined storage servers.

Further, the request sending unit 803 is further configured to: when the data block version numbers corresponding to the respective target data blocks are not completely the same, send the check data block to the storage server of the check data block in each storage target data stripe. request.

The data block receiving unit 805 is further configured to receive the check data block and the corresponding data block version number fed back by the respective storage server in response to the check data block acquisition request.

The data block generating unit 809 is configured to process each check data block by an EC algorithm when the data block version numbers corresponding to the respective check data blocks are the same, to obtain each data block in the target data strip.

The data block sending unit 806 is further configured to send each target data block in the data block to the client.

Specifically, the distributed system-based file reading apparatus introduced in the embodiment of the present invention may be used to implement some or all of the processes in the distributed system-based file reading method embodiment introduced by the present invention in conjunction with FIG. 3 to FIG. .

In the description of the present specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" and the like means a specific feature described in connection with the embodiment or example. A structure, material or feature is included in at least one embodiment or example of the invention. In the present specification, the schematic representation of the above terms is not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. In addition, various embodiments or examples described in the specification, as well as features of various embodiments or examples, may be combined and combined.

Moreover, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, features defining "first" or "second" may include at least one of the features, either explicitly or implicitly. In the description of the present invention, the meaning of "a plurality" is at least two, such as two, three, etc., unless specifically defined otherwise.

The logic and/or steps represented in the flowchart or otherwise described herein, for example, a list of programs that can be considered as executable instructions for implementing logical functions, can be embodied in any computer readable medium, Used by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions and execute instructions from an instruction execution system, apparatus, or device) Used for equipment. For the purposes of this specification, a "computer-readable medium" can be any apparatus that can include, store, communicate, propagate, or transport a program for use in an instruction execution system, apparatus, or device, or in conjunction with such an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections (electronic devices) having one or more wires, portable computer disk cartridges (magnetic devices), random access memory, read only memory , erasable editable read-only memory, fiber optic devices, and portable optical disk read-only memory. In addition, the computer readable medium may even be a paper or other suitable medium on which the program can be printed, as it may be optically scanned, for example by paper or other medium, followed by editing, interpretation or, if appropriate, other suitable The method is processed to obtain the program electronically and then stored in computer memory.

It should be understood that portions of the invention may be implemented in hardware, software, firmware or a combination thereof. In the above-described embodiments, multiple steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or combination of the following techniques well known in the art: having logic gates for implementing logic functions on data signals. Discrete logic circuits, application specific integrated circuits with suitable combinational logic gates, programmable gate arrays, field programmable gate arrays, and the like.

In addition, the modules in the various embodiments of the present invention may be implemented in the form of hardware or in the form of software functional modules. An integrated module can also be stored in a computer readable storage medium if it is implemented as a software functional module and sold or used as a standalone product.

Although the embodiments of the present invention have been shown and described, it is understood that the above-described embodiments are illustrative and are not to be construed as limiting the scope of the invention. The embodiments are subject to variations, modifications, substitutions and variations.

Claims

A file reading method based on a distributed system, the method is applied to a file interface server, and the file interface server communicates with a client, a lock server, and a storage server, and the method includes:

Receiving a file read request sent by the client, where the file read request carries an offset address and a read data amount;

Determining, according to the offset address and the read data amount, at least one target data block that the client needs to read and a target data strip to which the at least one target data block belongs, and one file includes at least one data segment a strip, each of the data strips including at least one data block;

And when the at least one target data block is part of the target data stripe, sending a data stripe version number acquisition request for the target data stripe to the lock server, and receiving the lock server response The data stripe version number obtains a data stripe version number that is requested by the request;

And sending, to the storage server that stores the target data block, a data block acquisition request, and receiving, by the storage server, the target data block and the corresponding data block version number that are fed back in response to the data block acquisition request;

And when the data stripe version number is the same as the data block version number corresponding to each of the target data blocks, each of the target data blocks is sent to the client.
The method according to claim 1, wherein said determining at least one target data block and said at least one target data that said client needs to read according to said offset address and said amount of read data After the target data strip to which the block belongs, it also includes:

And when the at least one target data block is all the data in the target data stripe, sending the data block acquisition request to each storage server storing the target data block;

Receiving, by the respective storage server, the target data block and the corresponding data block version number that are fed back in response to the data block acquisition request;

When each of the target data blocks corresponds to a data block version number, each of the target data blocks is sent to the client.
The method of claim 1, wherein after receiving the target data block and the corresponding data block version number that are fed back by the respective storage server in response to the data block acquisition request, the method further includes:

And when the data stripe version number and the data block version number corresponding to each of the target data blocks are not completely the same, the data block obtaining request is sent to each storage server that stores the data block in the target data stripe;

Receiving, by each of the storage servers, each of the data blocks and corresponding data block version numbers that are fed back in response to the data block acquisition request;

Determining each of the target data blocks in the received data block when the data block version numbers corresponding to the data blocks are the same;

Each of the target data blocks is sent to the client.
The method according to any one of claims 1 to 3, wherein the method further comprises:

Receiving a file write request sent by the client, where the file write request carries a file handle, an offset address, a write data, and a write data amount;

Obtaining, by using a version number allocator, a data stripe version number of the file corresponding to the file handle;

Determining, according to the offset address and the amount of write data, a data block to which the write data belongs and a data strip to which the data block belongs;

Transmitting the write data to a storage server storing a data block to which the write data belongs;

When the write data is successfully sent, the data stripe version number is sent to the lock server to notify the lock server to update the data stripe version number of the data stripe.
The method according to claim 2, wherein after receiving the target data block and the corresponding data block version number that are fed back by the respective storage server in response to the data block acquisition request, the method further includes:

When the data block version numbers corresponding to the target data blocks are the same, the data block version number corresponding to the target data block is used as the data stripe version number of the target data stripe;

Sending the data stripe version number to the lock server to notify the lock server to update the data stripe version number of the target data stripe.
The method of claim 5, wherein the sending the data stripe version number to the lock server comprises:

Sending a lock release request to the lock server, where the lock release request carries a data stripe identifier and a data stripe version number, to notify the lock server to update a data stripe version of the data stripe corresponding to the data stripe identifier number;

Receiving a lock release response message fed back by the lock server in response to the lock release request.
The method of claim 5, wherein the sending the data stripe version number to the lock server comprises:

Receiving a lock recall request sent by the lock server, where the lock recall request carries a data stripe identifier;

Sending a lock recall response message to the lock server in response to the lock recall request, the lock recall response message carrying a data stripe version number of the data stripe corresponding to the data stripe identifier to notify the lock server to update the location The data stripe version number of the data stripe corresponding to the data stripe identifier.
The method according to any one of claims 1 to 7, wherein the sending a data stripe version number acquisition request for the target data stripe to the lock server comprises:

Sending a read lock acquisition request to the lock server, where the read lock acquisition request carries a data stripe identifier of the target data stripe;

Receiving a read authorization response message of the target data stripe corresponding to the data stripe identifier fed back by the lock server in response to the read lock acquisition request, where the read grant response message carries the data score of the target data stripe Strip version number.
The method according to claim 2, wherein after receiving the target data block and the corresponding data block version number that are fed back by the respective storage server in response to the data block acquisition request, the method further includes:

When the version numbers of the data blocks corresponding to the target data blocks are not completely the same, And storing, by the storage server storing the check data block in the target data stripe, a check data block acquisition request;

Receiving, by the respective storage server, the check data block and the corresponding data block version number fed back by the check data block obtaining request;

When the data block version numbers corresponding to the respective check data blocks are the same, each of the check data blocks is processed by a data check erase code EC algorithm to obtain each of the target data blocks;

Each of the target data blocks is sent to the client.
The method according to any one of claims 1 to 9, wherein the file read request further carries a file handle;

And sending the data block acquisition request to each storage server that stores the target data block, including:

Obtaining, according to the file handle, a file attribute of the file locally, where the file attribute includes a storage path of each data block in the file;

Determining, according to a storage path of each of the target data blocks, a storage server that stores the target data block;

The data block acquisition request is sent to each of the determined storage servers.
A file reading device based on a distributed system, comprising:

a request receiving unit, configured to receive a file read request sent by the client, where the file read request carries an offset address and a read data amount;

a data block determining unit, configured to determine, according to the offset address and the read data amount, at least one target data block that the client needs to read and a target data strip to which the at least one target data block belongs, A file includes at least one data stripe, each of the data strips including at least one data block;

a request sending unit, configured to: when the at least one target data block is part of the target data stripe, send a data stripe version number acquisition request for the target data stripe to the lock server;

a version number obtaining unit, configured to receive, by the lock server, a version number of the data stripe Take the data stripe version number that the request feeds back;

The request sending unit is further configured to send a data block acquisition request to each storage server that stores the target data block;

a data block receiving unit, configured to receive, by the respective storage server, the target data block and the corresponding data block version number that are fed back in response to the data block acquisition request;

And a data block sending unit, configured to send each of the target data blocks to the client when the data stripe version number and the data block version number corresponding to each of the target data blocks are the same.
The device of claim 11 wherein:

The request sending unit is further configured to: when the at least one target data block is all the data in the target data stripe, send the data block obtaining request to each storage server that stores the target data block;

The data block receiving unit is further configured to receive, by the respective storage server, the target data block and the corresponding data block version number that are fed back in response to the data block acquisition request;

The data block sending unit is further configured to send each of the target data blocks to the client when the data block version numbers corresponding to the target data blocks are the same.
The device of claim 11 wherein:

The request sending unit is further configured to: when the data stripe version number and the data block version number corresponding to each of the target data blocks are not completely the same, to each storage server that stores the data block in the target data stripe Sending the data block acquisition request;

The data block receiving unit is further configured to receive each of the data blocks and corresponding data block version numbers that are fed back by each of the storage servers in response to the data block acquisition request;

The data block determining unit is further configured to: when each of the data blocks corresponds to a data block version number, determine each of the target data blocks in the received data block;

The data block sending unit is further configured to send each of the target data blocks to the client.
Apparatus according to any one of claims 11 to 13 wherein:

The request receiving unit is further configured to receive a file write request sent by the client, where the file write request carries a file handle, an offset address, a write data, and a write data amount;

The version number obtaining unit is further configured to obtain, by using a version number allocator, a data stripe version number of the file corresponding to the file handle;

The data block determining unit is further configured to determine, according to the offset address and the amount of write data, a data block to which the write data belongs and a data strip to which the data block belongs;

The device also includes:

a data sending unit, configured to send the write data to a storage server that stores a data block to which the write data belongs;

The version number sending unit is configured to send the data stripe version number to the lock server when the write data is successfully sent, to notify the lock server to update the data stripe version number of the data stripe.
The device of claim 12, wherein the device further comprises:

a version number sending unit, configured to: when a data block version number corresponding to each of the target data blocks is the same, use a data block version number corresponding to the target data block as a data stripe version number of the target data stripe, and Sending the data stripe version number to the lock server to notify the lock server to update the data stripe version number of the target data stripe.
The device according to claim 15, wherein the version number sending unit is specifically configured to:

Sending a lock release request to the lock server, where the lock release request carries a data stripe identifier and a data stripe version number, to notify the lock server to update a data stripe version of the data stripe corresponding to the data stripe identifier number;

Receiving a lock release response message fed back by the lock server in response to the lock release request.
The device according to claim 15, wherein the version number sending unit is specifically configured to:

Receiving a lock recall request sent by the lock server, where the lock recall request carries data strips Identification

Sending a lock recall response message to the lock server in response to the lock recall request, the lock recall response message carrying a data stripe version number of the data stripe corresponding to the data stripe identifier to notify the lock server to update the location The data stripe version number of the data stripe corresponding to the data stripe identifier.
The device according to any one of claims 11 to 17, wherein the request sending unit sends a data stripe version number acquisition request for the target data stripe to the lock server, specifically for:

Sending a read lock acquisition request to the lock server, where the read lock acquisition request carries a data stripe identifier of the target data stripe;

Receiving a read authorization response message of the target data stripe corresponding to the data stripe identifier fed back by the lock server in response to the read lock acquisition request, where the read grant response message carries the data score of the target data stripe Strip version number.
The device of claim 12 wherein:

The request sending unit is further configured to: when each of the target data blocks corresponds to a data block version number that is not completely the same, send a check data block to each storage server that stores the check data block in the target data stripe. request;

The data block receiving unit is further configured to receive a check data block that is fed back by each of the storage servers in response to the check data block acquisition request and a corresponding data block version number thereof;

The device also includes:

a data block generating unit, configured to process each of the check data blocks by an EC algorithm to obtain each of the target data blocks when the data block version numbers corresponding to the check data blocks are the same;

The data block sending unit is further configured to send each of the target data blocks to the client.
Apparatus according to any one of claims 11 to 19, wherein said file is read The request also carries a file handle;

The request sending unit sends the data block obtaining request to each storage server that stores the target data block, specifically for:

Obtaining, according to the file handle, a file attribute of the file locally, where the file attribute includes a storage path of each data block in the file;

Determining, according to a storage path of each of the target data blocks, a storage server that stores the target data block;

The data block acquisition request is sent to each of the determined storage servers.