WO2014153931A1

WO2014153931A1 - File storage method and device, access client and metadata server system

Info

Publication number: WO2014153931A1
Application number: PCT/CN2013/083689
Authority: WO
Inventors: 胡剑华; 朱鹏; 俞超
Original assignee: 中兴通讯股份有限公司
Priority date: 2013-03-27
Filing date: 2013-09-17
Publication date: 2014-10-02
Also published as: CN104079600A; CN104079600B

Abstract

Provided are a file storage method and device, an access client and a metadata server system. The method comprises: determining storage location information about at least two files, the storage location information comprising respective offset location information about the at least two files in a copy and location information about the copy; and sending the storage location information to a file access client to enable the file access client to interact with a file access server according to the storage location information and enable the file access server to respectively store the at least two files in the respective offset locations of the at least two files in the copy. The embodiments of the present invention avoid the unnecessary increase of fragments of a storage medium of the file access server.

Description

File storage method, device, access client and metadata server system

Embodiments of the present invention relate to the field of cloud storage, and in particular, to a file storage method, apparatus, access client, and metadata server system. Background technique

DFS (Distributed File System) is to divide files into several CHUNKs and store multiple copies on different servers. In order to be applicable to a wider range of applications in the cloud storage field, DFS must be able to Support for the storage of files of various sizes, as small as a few bytes of files, up to tens of gigabytes should be supported, and storage performance should not be different, but according to the existing mechanism, no matter how small the file The file access server stores it separately in a single copy on disk, which unnecessarily increases disk fragmentation when the file access server has more than one such file. Summary of the invention

In view of this, it is an object of embodiments of the present invention to provide a file storage method, apparatus, access client, and metadata server system to avoid unnecessary addition of files to the server storage medium.

To solve the above technical problem, the solution provided by the embodiment of the present invention is as follows:

The embodiment of the invention provides a file storage method, which is applied to a metadata server system, and the method includes:

Determining storage location information of at least two files; wherein the storage location information includes offset location information and replica location information of each of the at least two files in one copy;

Sending the storage location information to the file access client, so that the file access client interacts with the file access server according to the storage location information, so that the file access server will The at least two files are respectively stored to offset positions of the at least two files in the copy. The offset positions of the at least two files in the copy respectively include:

Enabling the file access server to write the at least two files to a cache block and, after the at least two files are written, writing data in the cache block into the copy, from the The offset position in the copy.

Preferably, the method further includes:

Recording a correspondence between a file name of each of the at least two files, the copy location information, and offset position information of each of the at least two files in the copy;

Receiving, by the file accessing client, a file name of the to-be-read file in the at least two files read by the request sent by the client;

Determining first information according to the correspondence relationship and a file name of the file to be read; wherein the first information includes the copy location information and an offset position of the file to be read in the copy Information

Sending the first information to the file access client, causing the file access client to interact with the file access server according to the first information, and reading a corresponding first file from the file access server. .

The embodiment of the present invention further provides a file storage method, which is applied to a file access client, and the method includes:

Receiving storage location information of at least two files sent by the metadata server system; wherein the storage location information includes offset location information and the replica location information of each of the at least two files in one copy; the storing Position information is determined by the metadata server system; interacting with the file access server according to the storage location information, causing the file access server to store the at least two files to the at least two files respectively in the In the copy Offset position. The offset positions of the at least two files in the copy respectively include:

Preferably, the interacting with the file access server according to the storage location information, so that the offset location in the copy includes:

Writing, for each of the at least two files, a shared memory page in which each of the files is cached according to the copy location information and offset location information of each of the files in the replica The file accessing a cache block of the server, enabling the file access server to write data in the cache block to the copy after the shared memory page is written, and after the data is written Each file is stored at an offset location in the copy of each of the files.

Preferably, the method further includes:

Sending a request to the metadata server system, where the file name of the file to be read is received; receiving the first information sent by the metadata server system; wherein the first information includes the copy location information and the Reading offset position information of the file in the copy; the first information is used by the metadata server system according to a file name of each of the at least two files, the copy location information, and the at least Corresponding relationship between the offset position information of the two files in the copy and the file name of the file to be read; the correspondence relationship is recorded by the metadata server system;

And interacting with the file access server according to the first information, and reading a corresponding first file from the file access server. The embodiment of the invention further provides a file storage device, which is applied to a metadata server system, and the device includes:

a determining module, configured to determine storage location information of the at least two files; wherein the storage location information includes offset location information and the replica location information of each of the at least two files in one copy;

The sending module is configured to send the storage location information to the file access client, so that the file access client interacts with the file access server according to the storage location information to make an offset position in the copy.

Preferably, the sending module includes:

a sending unit, configured to send storage location information to the file access client, so that the file access client interacts with the file access server according to the storage location information, so that the file access server sets the at least two files Write a cache block and write the data in the cache block to the copy after the file is written, and set the data after the write is completed.

The embodiment of the present invention further provides a file storage device, which is applied to a file access client, and the device includes:

a receiving module, configured to receive storage location information of at least two files sent by the metadata server system, where the storage location information includes offset location information and the replica location of each of the at least two files in one copy Information; the storage location information is determined by the metadata server system;

The interaction module is configured to interact with the file access server according to the storage location information, each offset position in the copy.

Preferably, the interaction module includes: An interaction unit configured to: for each of the at least two files, according to the copy location information and offset location information of each file in the replica, each file is cached Writing a shared memory page to the cache block of the file access server, enabling the file access server to write data in the cache block to the copy and the data after the shared memory page is written Each file described after the completion of writing is stored at an offset position of each of the files in the copy.

Embodiments of the present invention also provide a metadata server system including the file storage device described above.

Embodiments of the present invention also provide a file access client including the file storage device described above.

As can be seen from the above, the embodiment of the present invention has at least the following beneficial effects: Supporting different files to be stored in the same copy, thereby avoiding file access server storage compared with different files in the prior art that cannot be stored in the same copy. An unnecessary increase in the fragmentation of the medium. DRAWINGS

1 is a flow chart of steps of a file storage method according to Embodiment 1 of the present invention; FIG. 2 is a flowchart of steps of another file storage method according to Embodiment 2 of the present invention; FIG. 3 is a small file of a preferred embodiment. Schematic diagram of polymerization;

4 is a schematic flow chart of a small file aggregation write of a preferred embodiment;

FIG. 5 is a schematic flow chart of a small file aggregation read of a preferred embodiment. detailed description

The embodiments of the present invention will be described in detail below with reference to the drawings and specific embodiments.

A first embodiment of the present invention provides a file storage method, where the method includes the following steps: Step 101: Determine storage location information of at least two files, where the storage location information includes offset location information and replica location information of each of the at least two files in one copy; Step 102, send a storage location to the file access client The information causes the file access client to interact with the file access server according to the storage location information, so that the file access server stores the at least two files respectively to the offset positions of the two files in the copy.

This method is applied to a metadata server system.

It can be seen that, in the above manner, different files are stored in the same copy, which avoids unnecessary increase of the fragmentation of the file access server storage medium compared with the prior art that different files cannot be stored in the same copy.

The copy is located on the storage medium of the file access server. The storage medium is, for example, a magnetic disk. The copy location information includes: File access server ID and storage media ID in the file access server. Wherein, the file access server identifier is, for example: IP of the file access server: 10.47.107.111; storage medium identifier such as: IP of the file access server: 10.47.107.111.

In the first embodiment of the present invention, the sum of the sizes of the at least two files is not greater than the copy size. Preferably, each of the at least two files is no larger than the shared cache page (PAGE) size in the file access client.

In the first embodiment of the present invention, the copy location corresponds to the copy handle. Specifically, the copy location is determined by the metadata server system according to the copy handle; the copy handle is the file name of the first file in the at least two files by the metadata server system. Assigned. The first file is any one of at least two files; or, in consideration of the file access client sending an open file request to each of the at least two files to the metadata server system, the first file is metadata The file corresponding to the open file request received by the server system from the file access client for the first time.

Preferably, the metadata server system comprises a metadata management server and a plurality of metadata storage servers, wherein the copy handle is specifically allocated by the metadata management server, and the copy location is specifically determined by the first metadata storage server of the plurality of metadata storage servers. Determining, according to the copy handle, the first metadata storage server is from the plurality of elements by the metadata management server according to the file name of the first file The data storage server is selected.

In the first embodiment of the present invention, at least two files have different offset positions in the copy. The offset position here refers to the position where the file is stored in the copy, and the starting position in the position is coincident or offset from the starting position of the copy. The offset position information of each of the at least two files in the copy includes: offset size information between the start position of each file stored in the copy and the start position of the copy, for example, the number of bytes or The number of bits, the size and number of PAGEs, or the number of PAGEs in which the size of each file in at least two files is not larger than the size of the PAGE, and so on.

The offset position information of each of the at least two files in the copy can be determined as follows:

The first method is a timing-based approach.

In the first method, considering that the metadata server system sequentially receives the request for opening the file from the file access client, the number of these open file requests is the same as the number of at least two files, and the open file requests and at least two files— - Correspondingly, the metadata server system assigns an offset position to the corresponding file in order from the start position of the copy position in the order in which the file request is received.

Specifically, for example, the offset position of the corresponding file allocation for the first received open file is the starting position of the copy position, and the offset position and copy of the corresponding file allocation are requested for the second received open file request. The starting position differs by a shared cache page size, and the offset position of the corresponding file allocation for the third received open file request differs from the starting position of the copy by two shared cache page sizes, and so on.

Way two

The second method is based on the serial number in the file name.

In the second method, the respective offset positions of the plurality of files are assigned according to the sequence number included in the file name in the predetermined size order.

For example, the metadata server system receives three open file requests from the file access client, Corresponding to the file with the file name 010, the file with the file name 001, and the file with the file name 003, the offset positions are assigned to the three files in the order of the serial number, which is the file with the file name 001. The assigned offset position is the start position of the copy position, the offset position assigned to the file with the file name 003 differs from the start position of the copy by a shared cache page size, and the offset position assigned to the file with the file name 010 There are two shared cache page sizes that differ from the starting position of the copy.

For another example, the metadata server system receives three open file requests from the file access client, corresponding to the file with the file name 112, the file with the file name 111, and the file with the file name 113, according to the serial number from large to small. The order of the three files is offset, the offset position assigned to the file with the file name 113 is the start position of the copy position, the offset position assigned to the file with the file name 112, and the start position of the copy. The difference is a shared cache page size, and the offset position assigned to the file with the file name 111 differs from the start position of the copy by two shared cache page sizes.

Mode 2 is more suitable for scenes with fixed file names. The file name has a series of consecutive serial numbers. For example, the system can put 3 files into one copy. File 001, file 002, and file 003 will be put together, and the offset positions are respectively For no offset, one shared cache page size, two shared cache page sizes; for example, file 010, file 011, and file 012 are also in the same copy, offset positions are no offset, one shared cache page size, Two shared cache page sizes.

In the first embodiment of the present invention, in order to reduce the unnecessary consumption of resources of the storage medium IOPS (Input/Output Operations Per Second), the file access server stores at least two files respectively. The offset locations of at least two files in the copy include:

Enabling the file access server to write at least two files to one cache block and writing the data in the cache block to the copy after the at least two files are written, and after the data is written, in the embodiment of the present invention One, in order to access the client to file access service with existing files The file writing mechanism of the device is compatible, thereby reducing the implementation cost and complexity of the embodiment of the present invention, and interacting with the file access server according to the storage location information, so that the file access server stores at least two files to at least two files respectively. The offset locations in the copy include:

For each of the at least two files, according to the copy location information and the offset location information of each file in the copy, the shared memory page in which each file is cached is written to the cache block of the file access server, so that the file is accessed. After the server finishes writing, the data in the cache block is written into the copy, and after the data is written, each file is stored at the offset position of each file in the copy.

Wherein, considering that the writing times of different files in at least two files may be different, there are: a cache block in which shared memory pages having different files are written is different, and different cache blocks are written in the copy. The time is different. Alternatively, in order to reduce the unnecessary consumption of the storage medium IOPS resources, the cache blocks in which the shared memory pages in which different files are cached are written into the same cache block.

In the first embodiment of the present invention, in order to support reading of a file in a copy, the method further includes: recording a file name, a copy location information, and an offset of each of the at least two files in each of the at least two files Correspondence of location information;

Receiving a file access name of a file to be read in at least two files read by a request sent by the client;

Determining, according to the correspondence relationship and the file name of the file to be read, the first information; wherein, the first information includes the copy location information and the offset location information of the file to be read in the copy;

Sending the first information to the file access client, so that the file access client interacts with the file access server according to the first information, and reads the first file from the file access server.

A file storage method according to the first embodiment of the present invention includes the following steps:

Step 201: Receive storage location information of at least two files sent by the metadata server system, where the storage location information includes offset location information and replica location information of each of the at least two files in one copy; storing location information by metadata Server system determination;

Step 202, interacting with the file access server according to the storage location information, so that the file access service The server stores at least two files in an offset position of each of the at least two files in the copy. This method is applied to the file access client.

In the second embodiment of the present invention, in order to reduce unnecessary consumption of the storage medium IOPS resources, the file access server stores the at least two files separately to the offset positions of the at least two files in the copy respectively:

Enabling the file access server to write at least two files to one cache block and after the at least two files are written, the data in the cache block is written into the copy, and at least two files are respectively stored after the data is written. The offset position in at least two files in the copy.

In the second embodiment of the present invention, in order to be compatible with the file writing mechanism of the existing file access client to the file access server, thereby reducing the implementation cost and complexity of the embodiment of the present invention, according to the storage location information, and the file access server Interacting, causing the file access server to store at least two files separately to offset positions of at least two files in the copy respectively:

For each of the at least two files, according to the copy location information and the offset location information of each file in the copy, the shared memory page in which each file is cached is written to the cache block of the file access server, so that the file is accessed. After the above-mentioned shared memory page write is completed, the server writes the data in the cache block to the copy and each file is stored in the offset position of each file in the copy after the data is written.

In the second embodiment of the present invention, in order to support the reading of the file in the copy, the method further includes: sending a request to the metadata server system, where the file name of the file to be read is included; and receiving the first information sent by the metadata server system Wherein, the first information includes copy location information and offset location information of the file to be read in the copy; the first information is determined by the metadata server system according to file name, copy location information, and at least file name of each of the at least two files The correspondence between the offset position information of each of the two files in the copy and the file name of the file to be read are determined; The correspondence is recorded by the metadata server system;

According to the first information, interacting with the file access server, reading the corresponding first file from the file access server.

It should be noted that, because another file storage method provided by the second embodiment of the present invention corresponds to a file storage method provided by the first embodiment of the present invention, the content described above for the other file storage method is involved. For the specific meanings of the terms and technical means, reference may be made to the specific meanings of the terms and technical means set forth in the first embodiment of the present invention, and the details are not described herein.

In order to clarify the two file storage methods corresponding to the first embodiment of the present invention and the two embodiments of the present invention, a preferred embodiment of the two file storage methods is provided below: A multi-copy distributed file Small file aggregation implementation of the system (DFS).

In the preferred embodiment, the File Access Client (FAC): The DFS-oriented application provides an interface calling service similar to the standard file system, and the read and write data for the application layer is the page (PAGE) size. Managed.

Metadata server system: It is responsible for managing the metadata information such as file name and copy information of all DFS files, exists in the database, and provides metadata write and query operations to the file access client. The metadata server system includes a metadata management server and a plurality of metadata storage servers.

File Access Server (FAS): Responsible for interacting with its own storage medium in the cache block unit, and performing read and write operations on the cache block. The FAS manages the data in the size of the cache block (BLK); The file accesses the client's data read and write request, reads the data from the storage medium and returns it to the file access client; reads the data from the file access client and writes the storage medium;

Storage media: Generally, it is a normal SCIC disk or SATA disk. Where CHUNK is actually stored, CHUNK is a minimum of BLK size, the maximum value can be set, and the size is increased by BLK size.

Each of the at least two files is a small file. The small file here means that the size is not greater than A PAGE size file. The size of the PAGE can be set. The degree of aggregation of a CHUNK is BLK size/PAGE size. For example, PAGE size=32K, BLK size=1024Κ, the number of files aggregated in one CHUNK is 1024/32=32.

In the prior art, a small file corresponds to a PAGE, a BLK, corresponding to a copy on the disk; and in the preferred embodiment, after the aggregation, multiple PAGEs may correspond to the same BLK, and multiple PAGEs correspond to the same copy.

The offset size information between the starting position where the file is stored in the copy and the starting position of the copy is the number of PAGEs.

The following takes the degree of aggregation as 3 as an example, and illustrates the small file aggregation process in conjunction with FIG. As shown in FIG. 3, at least two files include a file FILE#001, a file FILE#002, and a file FILE#003.

When creating the file FILE#001, the FAC writes the PAGE#1 carrying FILE#001 to the first 1/3 buffer space of a BLK (FAS_BLK#1) of the FAS; after the PAGE#1 is written, When the file FILE#002 is created, the FAC writes the PAGE#2 carrying FILE#002 to the second 1/3 buffer space of FAS_BLK#1; after the PAGE#2 is written, when the file FILE#003 is created, The FAC writes PAGE#3 carrying FILE#003 to the third 1/3 buffer space of FAS_BLK#1.

After the PAGE#3 write is completed, the FAS brushes its own cache block FAS_BLK#1 (ie, the data in FAS_BLK#1 is written to a copy on the disk FAS_BLK#1-CHKFILE). After the brushing, the file FILE#001, the file FILE#002 and the file FILE#003 are stored in the respective offset positions in the copy, wherein the file FILE#001 is stored in the copy at the starting position of the copy. The starting position, the corresponding offset position is 0; the starting position of the file FILE#002 in the copy is different from the starting position of the copy by a PAGE size, and the corresponding offset position is 1; File FILE#003 The starting position stored in the copy differs from the starting position of the copy by two PAGE sizes, and the corresponding offset position is 2.

It can be seen that the preferred embodiment stores a certain number of files smaller than one PAGE size on the same BLK on the file access server, and stores the same on the disk. In the case of CHUNK, on the one hand, compared with the prior art, the number of CHUNKs stored on the disk is greatly reduced, the disk fragmentation is effectively reduced, and a certain disk space is saved. The overall read and write performance of the disk is improved; on the other hand, the reading and writing of multiple small files at the application layer only needs to occur once for the disk 10, which reduces the limited IOPS capability of the disk compared with the disk 10 that occurs in the prior art. The limitation of IOPS on the application layer reads and writes, which in turn increases the IOPS of the application layer. In addition, the copy location information of the CHUNK file in the metadata, that is, the file access server where the copy is located and the volume information of the file are also greatly reduced.

In the preferred embodiment, specifically, the bitmap information in the metadata storage server may be added with a bitmap of the degree of aggregation, and the PAGE number information corresponding to the file is recorded. For example, for the above example with a degree of aggregation of 3, the offset of the file FILE#001 is 0, the bitmap is 001 (binary); the offset of the file FILE#002 is 1, and the bitmap is 010 (binary); file FILE# The offset of 002 is 2 and the bitmap is 100 (binary). Taking the degree of polymerization as 8 as an example, the offset of the file corresponding to the number of PAGEs is 0, and the bitmap is 00000001 (binary); the offset of the file corresponding to the number of PAGEs is 3, and the bitmap is 00001000 (binary) ; The offset of the file corresponding to the number of PAGEs is 7 and the bitmap is 10000000 (binary).

Take the aggregation degree as 8 as an example, the page size is 32k, and the blk size is 256k. For example, a file file003 is newly written, and the size is lk, which satisfies the condition of small file aggregation (the size is smaller than one page size), and the metadata management server opens the file. At the time, the offset is confirmed based on the first few small files.

The following describes the process of small file aggregation by taking the file writing process and the reading process as examples. 4 is a schematic flowchart of a small file aggregation write according to a preferred embodiment. Referring to FIG. 4, a small file aggregation write process includes:

Step 401: The application layer initiates an open file request to the file access client.

Step 402: The file access client initiates an open file request (with a file name and a creation flag) to the metadata management server, and the database determines, according to the file name, the metadata storage server to which the file belongs and the copy handle to which the file belongs and the copy Offset position information. Multiple files within the same copy belong to the same metadata storage server. Step 403: After the file access client receives the response from the metadata management server, if the given copy handle is not 0, the copy handle and the offset location information are recorded in the file management global structure of the file access client.

Step 404: The file access client opens the file to the corresponding metadata storage server, and the metadata storage server replies to the file access client, and the file access client responds to the application layer after receiving the response.

Step 405: The application layer sends a write request to the file access client. After receiving the write request, the file access client first determines whether the copy handle recorded in the global structure of the file management is 0. If not, the copy handle is used to The metadata storage server sends a request for obtaining a copy location, and if it is 0, the request for obtaining the copy location is sent to the metadata storage server by using the file identifier generated by the file identifier plus the calculated copy number of the written page.

Step 406: After receiving the request, the metadata storage server obtains the copy location from the database and returns it to the file access client.

Step 407: After receiving the copy location of the metadata storage server, the file access client writes the data of the application layer to the shared memory page of the file access client, and responds to the application layer.

Step 408, the file access client's write thread writes the shared memory page to the file access server according to the copy handle and the offset location information.

Among them, since the file is a small file and can be cached in a PAGE, the file access client can reuse the existing mechanism for writing file data to the file access server to write different small file data. Specifically, the FAC only needs to write data of different small files as data buffered by the same file in different PAGEs. Therefore, the preferred embodiment does not need to modify the existing file access server, which saves the upgrade cost of the cloud service system.

For the file access server, if the cache block is missed, a new cache block is requested, and the data page is written into the buffer block. The data of the multiple files is written to the same cache block before being flushed into the disk, and finally stored in the buffer block. In the same copy on the disk.

In step 409, the application layer sends a close request, and the file access client responds. FIG. 5 is a schematic flowchart of a small file aggregation read according to a preferred embodiment. Referring to FIG. 5, the small file aggregation read process includes the following steps:

Step 501: The application layer initiates an open file request to the file access client.

Step 502: The file access client initiates an open file request to the metadata management server, and the database determines, according to the file name, the metadata storage server to which the file belongs, and the copy to which the file belongs and the offset location information in the copy. Multiple files within the same copy belong to the same metadata storage server.

Step 503: After the file access client receives the response from the metadata storage server, if the given copy handle is not 0, the copy handle and the offset are recorded in the file management global structure.

Step 504: The file access client opens the file on the metadata storage server, and the metadata storage server responds to the file access client, and the file access client responds to the application layer after receiving the response.

Step 505: The application layer sends a read request to the file access client. After receiving the read request, the file access client first determines whether the copy handle recorded in the global structure of the file management is 0. If not, the copy handle is used to The metadata storage server sends a request for obtaining a copy location, and if it is 0, the request for obtaining the copy location is sent to the metadata storage server by using the file identifier generated by the file identifier plus the calculated copy number of the written page.

Step 506: After receiving the request, the metadata storage server obtains the copy location information from the database and returns the file to the file access client.

Step 507: After receiving the copy location information of the metadata storage server, the file access client first reads the shared memory of the file access client according to the copy location information and the offset location information, and determines whether there is any corresponding to the offset location information. PAGE, hit returns the file in the PAGE to the application layer, otherwise, continue to read to the file access server cache block, hit returns, otherwise, go to disk to read the corresponding copy, a copy of the data stored in multiple files, read One cache block data, the next time you read another file, you may directly hit the cache, no need to go to disk read, where the read data to the cache block is always read at the beginning of the cache block. Step 508: The file access client reads the cache block data read by the disk into the cache of the file access server, and reads the PAGE corresponding to the offset location information into the shared memory of the file access client, and returns it to the application layer.

It can be seen that, in the preferred embodiment, since the file access client reads the small file in the manner of reading the PAGE, the existing file access server does not need to be modified, which saves the upgrade cost of the cloud service system.

Step 509, the application layer sends a close request, and the file access client responds.

Preferably, the preferred embodiment is for an application scenario in which small files are stored more or only small files are stored.

It should be noted that the preferred embodiment is also applicable to other distributed file systems based on files for file storage.

A third embodiment of the present invention provides a file storage device, which is applied to a metadata server system, where the device includes:

a determining module, configured to determine storage location information of at least two files; wherein the storage location information includes offset location information and replica location information of each of the at least two files in one copy;

The sending module is configured to send the storage location information to the file access client, so that the file access client interacts with the file access server according to the storage location information, so that the file access server stores at least two files respectively into at least two files respectively in the copy The offset position in .

In the third embodiment of the present invention, the sending module includes:

The sending unit is configured to send the storage location information to the file access client, so that the file access client interacts with the file access server according to the storage location information, so that the file access server writes at least two files into one cache block and at least two After the file is written, the cache block will be The data in the write is written in the copy, and at least two files are respectively stored in the offset position of each of the at least two files in the copy after the data is written.

It should be noted that the third embodiment of the present invention is an apparatus embodiment, which corresponds to the first embodiment of the present invention (which is a method embodiment), and the part that is not described in detail in the third embodiment of the present invention is related to the first embodiment of the present invention. Part of the description can be, in order to save space, no longer repeat them here. In addition, the determining module in the third embodiment of the present invention may be implemented by a central processing unit of the metadata server system.

(CPU, Central Processing Unit), processor (MPU, Micro Processing Unit) or digital signal processor (DSP); the transmitting module and the transmitting unit in the transmitting module are available in the metadata server system The external communication function of the chip is implemented.

The fourth embodiment of the present invention provides another file storage device, which is applied to a file access client, and the device includes:

a receiving module, configured to receive storage location information of at least two files sent by the metadata server system, where the storage location information includes offset location information and copy location information of each of the at least two files in one copy; The metadata server system determines; the interaction module is configured to interact with the file access server according to the storage location information, so that the file access server stores the at least two files separately to an offset position of each of the at least two files in the copy.

In the fourth embodiment of the present invention, the interaction module includes:

The interaction unit is configured to write, to each of the at least two files, the shared memory page in which each file is cached to the file access server cache according to the copy location information and the offset location information of each file in the copy Block, after the file access server writes the shared memory page, writes the data in the cache block to the copy and stores each file in each file after the data is written. The offset position of the file in the copy.

It should be noted that the fourth embodiment of the present invention is an apparatus embodiment, and corresponds to the second embodiment of the present invention (which is a method embodiment), and the parts that are not described in detail in the fourth embodiment of the present invention refer to the first and second embodiments of the present invention. The description of the relevant part is not repeated here. The determining module in the fourth embodiment of the present invention may be implemented by the file accessing the CPU, the MPU or the DSP of the client; the interaction module and the interaction unit in the sending module may be implemented by the chip having the interactive function in the file access client.

A fifth embodiment of the present invention provides a metadata server system. The metadata server system includes a file storage device according to Embodiment 3 of the present invention.

The sixth embodiment of the present invention provides a file access client, and the file access client includes another file storage device provided in Embodiment 4 of the present invention.

The above is only an embodiment of the embodiments of the present invention. It should be noted that those skilled in the art can also make several improvements and retouchings without departing from the principles of the embodiments of the present invention. It should be considered as the scope of protection of the embodiments of the present invention.

Claims

Claim

A file storage method, applied to a metadata server system, the method comprising: determining storage location information of at least two files; wherein the storage location information includes each of the at least two files in one copy Offset location information and copy location information; sending the storage location information to a file access client, causing the file access client to interact with the file access server according to the storage location information, so that the file access server will At least two files are respectively stored to offset positions of the at least two files in the copy.

2. The method according to claim 1, wherein the causing the file access server to store the at least two files separately to an offset position of each of the at least two files in the copy comprises: After the at least two files are written, the data in the cache block is written into the copy, each offset position in the copy.

3. The method according to claim 1, wherein the method further comprises:

Recording a correspondence between a file name of each of the at least two files, the copy location information, and offset location information of each of the at least two files in the copy;

Receiving, by the file accessing client, a file name of the file to be read in the at least two files read by the request sent by the client;

Sending the first information to the file access client, causing the file access client to interact with the file access server according to the first information, and access the service from the file The device reads out the corresponding first file.

A file storage method, which is applied to a file access client, the method comprising: receiving storage location information of at least two files sent by a metadata server system; wherein the storage location information includes the at least two files Offset location information and copy location information in a copy; the storage location information is determined by the metadata server system;

And interacting with the file access server according to the storage location information, causing the file access server to separately store the at least two files to an offset position of each of the at least two files in the copy.

5. The method according to claim 4, wherein the causing the file access server to store the at least two files separately to an offset position of each of the at least two files in the copy comprises: After the at least two files are written, the data in the cache block is written into the copy, each offset position in the copy.

The method according to claim 4, wherein the interacting with the file access server according to the storage location information causes the file access server to separately store the at least two files to the at least two files The respective offset positions in the copy include:

Writing, for each of the at least two files, a shared memory page in which each of the files is cached according to the copy location information and offset location information of each of the files in the replica The file accessing a cache block of the server, after the file access server writes the data in the cache block into the copy after the writing of the shared memory page is completed, and the data is written after the writing is completed. Each file is stored at an offset location in the copy of each of the files.

The method of claim 4, wherein the method further comprises: Sending a request to the metadata server system, where the file name of the file to be read is received; receiving the first information sent by the metadata server system; wherein the first information includes the copy location information and the Reading offset position information of the file in the copy; the first information is used by the metadata server system according to a file name of each of the at least two files, the copy location information, and the at least Corresponding relationship between the offset position information of the two files in the copy and the file name of the file to be read; the correspondence relationship is recorded by the metadata server system;

And interacting with the file access server according to the first information, and reading a corresponding first file from the file access server.

A file storage device, which is applied to a metadata server system, the device comprising: a determining module configured to determine storage location information of at least two files; wherein the storage location information includes each of the at least two files Offset location information and copy location information in one copy;

And a sending module, configured to send the storage location information to the file access client, so that the file access client interacts with the file access server according to the storage location information to make an offset position in the copy.

9. The device of claim 8, wherein the sending module comprises:

a sending unit, configured to send storage location information to the file access client, so that the file access client interacts with the file access server according to the storage location information, so that the file access server sets the at least two files Writing a cache block and writing data in the cache block to the copy and the offset position in the copy after the at least two files are written.

A file storage device, applied to a file access client, the device comprising: a receiving module configured to receive at least two files sent by a metadata server system Storage location information; wherein the storage location information includes offset location information and the replica location information of each of the at least two files in one copy; the storage location information is determined by the metadata server system;

And an interaction module configured to interact with the file access server according to the storage location information, and each of the pieces is offset from the copy.

11. The apparatus according to claim 10, wherein the interaction module comprises: an interaction unit configured to, for each of the at least two files, according to the copy location information and each of the files Offset location information in the replica, writing a shared memory page in which each file is cached to a cache block of the file access server, so that the file access server writes the shared memory page after completion Each of the files in the cache block is written to the copy and the file is stored at an offset location in the copy of the file.

A metadata server system comprising the file storage device of claim 8 or 9.

A file access client comprising the file storage device of claim 10 or 11.