WO2017036308A1

WO2017036308A1 - File uploading method, access method, device and apparatus

Info

Publication number: WO2017036308A1
Application number: PCT/CN2016/095856
Authority: WO
Inventors: 王银瓶; 蔡瀛
Original assignee: 阿里巴巴集团控股有限公司; 王银瓶; 蔡瀛
Priority date: 2015-08-31
Filing date: 2016-08-18
Publication date: 2017-03-09
Also published as: CN106487855A; CN106487855B

Abstract

The present application discloses a file uploading method, access method, device and apparatus. The file uploading method comprises: receiving target file data of a target file; packaging the target file data as sub-files; writing at least one of the sub-files in a data file of the target file in a distributed file system; and appending update meta-information of the data file to said at least one sub-file, wherein the update meta-information of the data file comprises attribute information of all sub-files of the target file that are written. In an embodiment of the present application, only one piece of update meta-information comprising attribute information of all sub-files is maintained during a file upload, thereby preventing separate maintenance operation of meta-information for each sub-file, and saving a storage resource and maintenance resource of the system accordingly. In addition, synthesizing sub-files into a target file is not necessary, thus allowing a client device to read file data of a target file in a distributed file system at any time.

Description

File uploading method, access method, device and device

Technical field

The present application relates to the field of data processing technologies, and in particular, to a file uploading method, an access method, an apparatus, and a device.

Background technique

In the cloud storage network architecture, an uploading device for uploading real-time files and a distributed file system for saving files are included, and the distributed file system includes multiple storage servers, and the user terminal can be in a distributed file system. The file is accessed. When uploading a device to upload a target file to a distributed file system, it is necessary to add meta (meta) information at the end of the target file, and when the user terminal accesses the target file from the distributed file system, by reading The meta information parses the target file.

In the prior art, in order to satisfy the purpose that the user terminal can access the file data of the target file as soon as possible, the uploading device may divide the file data into several sub-files, and add corresponding meta information at the end of each sub-file. The number of files in the distributed file system is increased, and a large amount of meta information needs to be maintained in the distributed file system. Subsequently, the subfiles in the distributed file system are synthesized into target files, and the user cannot access the synthetic operation. To file data, and because of the need to frequently read and write sub-files, it consumes a lot of system processing resources.

Summary of the invention

The application provides a file uploading method, an access method, a device and a device, so as to solve the problem that the existing file uploading method increases the number of files in the system, and the synthetic file consumes system processing resources, which is inconvenient for access.

According to a first aspect of the embodiments of the present application, a file uploading method is provided, where the method is applied to an uploading device that uploads a file to a distributed file system in real time, and the method includes:

Obtain target file data of the target file;

Encapsulating the target file data into a subfile;

Writing at least one of the subfiles to a data file of the target file in the distributed file system;

The update meta information of the data file is written after the at least one subfile, and the update meta information of the data file includes attribute information of all subfiles of the target file that have been written.

According to a second aspect of the embodiments of the present application, a file access method is provided, and the method is used to access a file uploaded to a distributed file system by using the foregoing file uploading method, where the method includes:

Receiving a file access command of the data file sent by the client device, where the file access command includes target attribute information of the file data to be accessed;

Obtaining, according to the target attribute information and the updated meta information of the data file, a target subfile corresponding to the to-be-accessed file data in the data file;

The file data to be accessed read from the target subfile is provided to the client device.

According to a third aspect of the embodiments of the present application, a file uploading apparatus is provided, where the apparatus is applied to an uploading device that uploads a file to a distributed file system in real time, and the apparatus includes:

An obtaining unit, configured to acquire target file data of the target file;

a packaging unit, configured to encapsulate the target file data into a subfile;

a uploading unit, configured to write at least one of the subfiles to a data file of the target file in the distributed file system, and write update meta information of the data file after the at least one subfile The update meta information of the data file includes attribute information of all subfiles of the target file that have been written.

According to a fourth aspect of the embodiments of the present application, there is provided a file accessing device, the device for accessing a file uploaded to a distributed file system by using the foregoing file uploading device, the device comprising:

a receiving unit, configured to receive a file access command of the data file sent by the client device, where the file access command includes target attribute information of the file data to be accessed;

a obtaining unit, configured to obtain, according to the target attribute information and the updated meta information of the data file, a target subfile corresponding to the to-be-accessed file data in the data file;

And an access unit, configured to provide the to-be-accessed file data read from the target sub-file to the client device.

According to a fifth aspect of the embodiments of the present application, a file uploading apparatus is provided, including:

a processor; a memory for storing the processor executable instructions;

Wherein the processor is configured to:

Obtain target file data of the target file;

Encapsulating the target file data into a subfile;

According to a sixth aspect of the embodiments of the present application, a storage server is provided, including:

a processor; a memory for storing the processor executable instructions;

Wherein the processor is configured to:

When the file is uploaded in the embodiment of the present application, after the file data of the target file is encapsulated into a sub-file, the sub-file is not directly uploaded to the distributed file system, but the sub-file is written to the data file of the target file. After the subfile is written to the update meta information of the data file, since the update meta information includes attribute information of all subfiles of the target file that has been written to the distributed file system, only one update meta information needs to be maintained. Yes, avoid the separate dimension of the meta information of each subfile Protection, there is no need to synthesize the sub-file into the target file, thereby saving the storage resources and maintenance resources of the system; based on the data file of the foregoing upload target file, the client device can directly access any of the data files according to the updated metadata information of the current data file. The subfile of the uploaded target file, so that the client device can read the file data of the target file in the distributed file system at any time.

The above general description and the following detailed description are intended to be illustrative and not restrictive.

DRAWINGS

The drawings herein are incorporated in and constitute a part of the specification,

1 is a schematic structural diagram of a system for implementing file uploading and accessing by applying an embodiment of the present application;

2A is a flowchart of an embodiment of a method for uploading a file according to the present application;

2B is a schematic structural diagram of a data file uploaded by applying an embodiment of the file uploading method of the present application;

FIG. 3 is a flowchart of an embodiment of a method for accessing a file according to the present application; FIG.

4A is a hardware structural diagram of a device where an embodiment of a file uploading apparatus is located;

4B is a hardware structural diagram of a device where an embodiment of a file access device of the present application is located;

Figure 5 is a block diagram of an embodiment of the file uploading apparatus of the present application;

FIG. 6 is a block diagram of an embodiment of a file access device of the present application.

detailed description

The terminology used in the present application is for the purpose of describing particular embodiments, and is not intended to be limiting. The singular forms "a", "the" and "the" It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used to describe various information in this application, such information should not be limited to these terms. These terms are only used to refer to the same type of information. This area is separate. For example, the first information may also be referred to as the second information without departing from the scope of the present application. Similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "when" or "in response to a determination."

Referring to FIG. 1 , a schematic diagram of a system architecture for implementing file uploading and accessing by applying the embodiment of the present application:

The system shown in FIG. 1 includes: a file uploading device, a distributed file system including N storage servers, and a client device. The file uploading device is configured to upload the target file to the distributed file system, and the client device is configured to access the uploaded target file from the distributed file system.

In the embodiment of the present application, when uploading the target file, the file uploading device encapsulates the file data of the target file into a subfile, and then writes the subfile to the data file of the target file, and then writes the data file after the subfile. Update metadata information, which includes attribute information of all subfiles of the target file that have been written to the distributed file system, that is, each time the current subfile is written and the current subfile is included on the basis of the original data file. Attribute information of all uploaded sub-files; when the client device wants to access the target file, it can directly access the sub-file of any uploaded target file according to the update meta-information of the current data file, thereby ensuring the client The device reads the file data of the target file in the distributed file system at any time. The application will be described in detail below in conjunction with specific embodiments.

2A is a flowchart of an embodiment of a method for uploading a file according to the present application. The embodiment may be applied to an uploading device for uploading files to a distributed file system in real time, including the following steps:

Step 201: Acquire target file data of the target file.

In this embodiment, the uploading device can obtain the file data of the target file through the real-time data channel, and the file data of each target file cannot be obtained in one time, so the time period can be set in advance, and the file data is obtained according to the time period, and each The file data acquired during the time period is called the target file data, and the target file data is usually saved in the memory of the uploading device.

Therefore, in an optional implementation manner, the uploading device may determine whether a preset time period is reached. If the time period is reached, the file data received during the time may be obtained from the memory as the target file data of the target file. . For example, the target file is the electronic novel "Deer" to be uploaded. Ding Ji, the electronic novel has 508 pages, the file data is all the bytes corresponding to the text content of the electronic novel; assuming the preset time period is 1 minute, the uploading device obtains 10 pages of text content in 1 minute. , all the bytes corresponding to the 10 pages of text content are the target file data acquired by the uploading device within 1 minute.

Step 202: Encapsulate the target file data into a subfile.

In this step, after obtaining the target file data in any time period, the saved target file data may be packaged in the sub-file encapsulation format in the memory, and the corresponding meta information is added after the target file data, thereby generating a sub- A file in which meta information of a subfile is information for parsing a subfile. The specific process of encapsulating the object file into a sub-file is consistent with the prior art, and details are not described herein again.

Step 203: Write at least one subfile to the data file of the target file in the distributed file system.

For the convenience of description, the entire sub-file of the uploaded target file is referred to as the data file of the target file, and when the sub-file of the target file is uploaded, at least one sub-fold is uploaded each time according to the generation order of the sub-file. The file, therefore, the data file of the target file is a file that dynamically changes as the subfile is uploaded (this data file can also be referred to as an otar file in this embodiment). Each time at least one subfile is written, the at least one subfile is written to the end of the data file of the target file in the distributed file system, and the end of the data file includes data before the at least one subfile is written Meta-information of a file including attribute information of all sub-files of the target file that have been written to the data file before the at least one sub-file.

In an optional example, when uploading at least one subfile, the append command in the java language may be used, and the append command may be used to insert the specified content at the end of the selected element. In this embodiment, the append command may be used. At least one subfile is inserted at the end of the data file, and subsequent update meta information of the data file can be inserted after the at least one subfile by the append command.

Step 204: Write update meta information of the data file after the at least one subfile, and the update meta information of the data file includes attribute information of all subfiles of the written target file.

In this step, the historical meta information of the data file before the at least one subfile is written to the data file, and the historical meta information includes attributes of all subfiles of the target file written before the at least one subfile. Information; then generating attribute information of the uploaded at least one subfile in the data file, and generating update metadata information of the data file according to the attribute information of the subfile included in the historical meta information and the attribute information of the at least one subfile, and finally The generated update meta information is written after the at least one subfile in the data file, that is, the meta information of the data file written after the at least one subfile includes attribute information of all subfiles of the written target file. . For example, if the history information of the data file includes the attribute information of the n subfiles of the subfile 0 to the subfile n, after the subfile n+1 is written, the update meta information written at the end of the data file is included. Attribute information of sub-file 0 to n+1 sub-files of sub-file n+1.

In this embodiment, the attribute information of the subfile may include: offset information of the subfile in the data file (File offset in otar), length of the subfile (File size in otar), and number of data rows included in the subfile (Record) Count), and other attribute information (Other info) of the subfile, for example, statistical information, or custom information.

2B is a schematic diagram of a structure of a data file uploaded by applying an embodiment of the file uploading method of the present application. The structure of the data file is described below in conjunction with a sub-file uploading process.

In this embodiment, the file structure of each sub-file encapsulated by the uploading device is consistent with the prior art, and includes file data of the sub-file and meta-information of the sub-file, and the sub-file encapsulation process is not further described later, and the encapsulated sub-process is directly The file is described.

First upload (append): Upload the device to complete the first subfile 0 of the target file, upload the subfile 0 to the distributed file system, and add all subfiles of the uploaded target file at the end of subfile 0. The attribute information, at this time, only the attribute information of the subfile 0 is included as the update meta information of the data file 1;

Second upload (append): uploads the second subfile 1 of the device package completion target file, uploads the subfile 1 to the data file 1 in the distributed file system, and adds the uploaded target at the end of the subfile 1. The attribute information of all the subfiles of the file, at this time including the attribute information of the subfile 0 and the subfile 1, as the update meta information of the data file 2;

The third upload (append): upload the device to complete the third subfile 2 and the fourth subfile 3 of the target file, upload the subfile 2 and the subfile 3 to the data file 2 in the distributed file system, and then The attribute information of all the subfiles of the uploaded target file is added at the end of the file 3, and the attribute information of the subfile 0, the subfile 1, the subfile 2, and the subfile 3 is included as the update meta information of the data file 3.

As shown in FIG. 2B, the meta information of the data file of the dynamically generated target file includes attribute information of all uploaded subfiles, and the attribute information may include subfile offset, subfile length, and subfile data. The number of rows, other attribute information of the subfile, and so on.

As can be seen from the above embodiment, after the file data of the target file is encapsulated into a subfile, the embodiment does not directly upload the subfile to the distributed file system, but writes the subfile to the data file of the target file. After the subfile is written to the update meta information of the data file, since the update meta information includes attribute information of all subfiles of the target file that has been written to the distributed file system, only one update meta information needs to be maintained. Yes, the meta information of each subfile is separately maintained, and the subfiles are not synthesized into the target file, thereby saving the storage resources and maintenance resources of the system.

3 is a flowchart of an embodiment of a file access method of the present application. The embodiment may be applied to a storage server of a distributed file system, and used to implement uploading and distributing the file uploading method embodiment shown in FIG. 2A. File access in the file system:

Step 301: Receive a file access command of a data file sent by the client device, where the file access command includes target attribute information of the file data to be accessed.

In this step, when the client device accesses the data file of the target file, the file access command of the data file is sent to the distributed file system, and the file access command may include the attribute information of the file data to be accessed, which is referred to in this embodiment. For the target attribute information, for example, the target attribute information may be the number of lines of the file data to be accessed in the target file.

Step 302: Obtain a target sub-file corresponding to the file data to be accessed in the data file according to the target attribute information and the update meta-information of the data file.

2A and 2B, the update meta-information at the end of the data file of the target file includes attribute information of all uploaded sub-files, and thus can be based on the target genus. The sexual information is sequentially matched with the attribute information of each subfile in the updated meta information. For example, when the target attribute information is the number of rows, the row number information in the attribute information of each subfile can be matched, when from each subfile. When the attribute information is matched to the target attribute information, the subfile corresponding to the target attribute information may be determined as the target subfile.

Step 303: Provide the to-be-accessed file data read from the target sub-file to the client device.

In this step, after obtaining the target subfile, the file data of the target subfile may be parsed according to the meta information of the target subfile, and the file data to be accessed is read from the parsed file data according to the target attribute information, and then the file is accessed to the client. The device transmits the read file data to be accessed.

With reference to the example of FIG. 2B, assuming that the target attribute information included in the received file access command is the target line number information, the target line number information and the meta information of the data file 3 (Otar Meta(File0, File1, File2, File3)) The number of lines in the attribute information of the neutron file 0 to the subfile 3 is matched, and the subfile to which the target line number information belongs is determined as the target subfile, and if the target subfile is the subfile 1, the further According to the offset information and the length information in the attribute information of the subfile 1, the subfile 1 is obtained from the data file 3, and the subfile 1 is parsed according to the meta information (File1Meta) of the subfile 1, so that the subfile is obtained from the subfile. The file data to be accessed is read from the file data of 1.

It can be seen from the above embodiment that, based on the data file of the target file uploaded in the distributed file system, the client device can directly access the subfile of any uploaded target file according to the updated meta information of the current data file, thereby ensuring the client. The end device reads the file data of the target file in the distributed file system at any time.

The embodiment of the file uploading apparatus can be applied to a file uploading device, and an embodiment of the file accessing apparatus can be applied to a storage server. The device embodiment may be implemented by software, or may be implemented by hardware or a combination of hardware and software. Taking the software implementation as an example, as a logical means, the processor of the device in which it is located reads the corresponding computer program instructions in the non-volatile memory into the memory. From the hardware level, as shown in FIG. 4A and FIG. 4B, respectively, the hardware structure diagram of the device where the file uploading device and the file access device are located, except for the processor, the memory, the network interface, and the non-display shown in FIG. 4A and FIG. 4B. Beyond volatile memory, real The device in which the device is located in the embodiment may also include other hardware according to the actual function of the device, which is not shown in FIG. 4A and FIG. 4B.

Referring to FIG. 5, it is a block diagram of an embodiment of a file uploading apparatus of the present application, which may be applied to an uploading device that uploads files in real time to a distributed file system:

The file uploading device includes an obtaining unit 510, a packaging unit 520, and an uploading unit 530.

The obtaining unit 510 is configured to acquire target file data of the target file.

a packaging unit 520, configured to encapsulate the target file data into a subfile;

The uploading unit 530 is configured to write at least one of the subfiles to a data file of the target file in the distributed file system, and write an update element of the data file after the at least one subfile Information, the update meta information of the data file includes attribute information of all subfiles of the target file that have been written.

In an optional implementation:

The obtaining unit 510 may include (not shown in FIG. 5):

a period determining subunit, configured to determine whether a preset time period is reached;

And a data obtaining subunit, configured to acquire file data of the target file received in the time period if the time period is reached, and use the acquired file data as the target file data.

In another alternative implementation:

The package unit 520 can include (not shown in FIG. 5):

a data writing subunit, configured to package the target file data that has been written into the memory according to a subfile encapsulation format;

The meta information adding subunit is configured to generate the subfile after adding the meta information of the target file data after the target file data.

In another alternative implementation:

The uploading unit 530 can include (not shown in FIG. 5):

a historical meta information obtaining subunit, configured to acquire historical meta information of the data file before the at least one subfile is written into the data file, where the historical meta information is included in the at least Attribute information of a subfile of the target file written before a subfile;

An attribute information generating subunit, configured to generate attribute information of the at least one subfile in the data file;

An update meta information generating subunit, configured to generate updated meta information of the data file according to attribute information of the subfile included in the historical meta information and attribute information of the at least one subfile;

The update meta information is written to the subunit for writing the update meta information after at least one subfile in the data file.

In another alternative implementation:

The attribute information of the subfile may include:

The offset information of the subfile in the data file, the length of the subfile, and the number of data rows included in the subfile.

Referring to FIG. 6, a block diagram of an embodiment of a file access device for accessing a file uploaded to a distributed file system using the device shown in FIG. 5 is used:

The file access device includes a receiving unit 610, an obtaining unit 620, and an access unit 630.

The receiving unit 610 is configured to receive a file access command of the data file sent by the client device, where the file access command includes target attribute information of the file data to be accessed.

The obtaining unit 620 is configured to obtain, according to the target attribute information and the updated meta information of the data file, a target subfile corresponding to the to-be-accessed file data in the data file;

The access unit 630 is configured to provide the to-be-accessed file data read from the target sub-file to the client device.

In an optional implementation:

The obtaining unit 620 may include (not shown in FIG. 6):

An attribute information matching subunit, configured to match attribute information of each subfile in the update meta information according to the target attribute information;

The subfile determining subunit is configured to determine, as the target subfile, a subfile corresponding to the target attribute information when the target attribute information is matched from the attribute information of each of the subfiles.

In another alternative implementation:

The access unit 630 can include (not shown in FIG. 6):

a subfile parsing subunit, configured to parse file data of the target subfile according to meta information of the target subfile;

a file data reading subunit, configured to read the to-be-accessed file data from the parsed file data according to the target attribute information;

a file data transmission subunit, configured to transmit the to-be-accessed file data to the client device.

The implementation process of the function and the function of each unit in the foregoing device is specifically described in the implementation process of the corresponding steps in the foregoing method, and details are not described herein again.

For the device embodiment, since it basically corresponds to the method embodiment, reference may be made to the partial description of the method embodiment. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, ie may be located A place, or it can be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the objectives of the present application. Those of ordinary skill in the art can understand and implement without any creative effort.

It can be seen from the above embodiment that when the file is uploaded in the embodiment of the present application, after the file data of the target file is encapsulated into a subfile, the subfile is not directly uploaded to the distributed file system, but the subfile is written to the target file. After the data file is written, the update meta information of the data file is written after the subfile, and since the update meta information contains the attribute information of all the subfiles of the target file that has been written to the distributed file system, only maintenance is required. The one piece updates the meta information, avoids maintaining the meta information of each subfile separately, and does not need to synthesize the subfile into the target file, thereby saving the storage resource and maintenance resources of the system; based on the data file of the foregoing upload target file, due to the client The end device can directly access the subfile of any uploaded target file according to the update meta information of the current data file, so that the client device can read the file data of the target file in the distributed file system at any time.

Those skilled in the art will readily appreciate the present invention after considering the specification and practicing the invention disclosed herein. Other embodiments of the application. The application is intended to cover any variations, uses, or adaptations of the application, which are in accordance with the general principles of the application and include common general knowledge or common technical means in the art that are not disclosed herein. . The specification and examples are to be regarded as illustrative only,

It is to be understood that the invention is not limited to the details of the details and The scope of the present application is limited only by the accompanying claims.

Claims

A method for uploading a file, the method is applied to an uploading device that uploads a file to a distributed file system in real time, and the method includes:

Obtain target file data of the target file;

Encapsulating the target file data into a subfile;

Writing at least one of the subfiles to a data file of the target file in the distributed file system;

The update meta information of the data file is written after the at least one subfile, and the update meta information of the data file includes attribute information of all subfiles of the target file that have been written.
The method according to claim 1, wherein the obtaining target file data of the target file comprises:

Determining whether a preset time period has elapsed;

If the time period is reached, the file data of the target file received in the time period is acquired, and the acquired file data is used as the target file data.
The method according to claim 1, wherein the encapsulating the target file data into a subfile comprises:

Packaging the target file data that has been written into the memory according to the subfile encapsulation format;

After the meta information of the target file data is added after the target file data, the subfile is generated.
The method according to claim 1, wherein the writing of the updated meta information of the data file after the at least one subfile comprises:

Acquiring historical meta information of the data file before the at least one subfile is written to the data file, where the historical meta information includes a subfile of the target file written before the at least one subfile Attribute information;

Generating attribute information of the at least one subfile in the data file;

And attribute information of the subfile included in the historical meta information and the at least one subfile The attribute information generates updated meta information of the data file;

Writing the update meta information after the at least one subfile in the data file.
The method according to any one of claims 1 to 4, characterized in that the attribute information of the subfile comprises:

The offset information of the subfile in the data file, the length of the subfile, and the number of data rows included in the subfile.
A file access method, characterized in that the method is used for accessing a file uploaded to a distributed file system by using the method of any one of claims 1 to 5, the method comprising:

Receiving a file access command of the data file sent by the client device, where the file access command includes target attribute information of the file data to be accessed;

Obtaining, according to the target attribute information and the updated meta information of the data file, a target subfile corresponding to the to-be-accessed file data in the data file;

The file data to be accessed read from the target subfile is provided to the client device.
The method according to claim 6, wherein the obtaining, according to the target attribute information and the updated meta information of the data file, the target subfile corresponding to the file data to be accessed in the data file, include:

Matching attribute information of each subfile in the update meta information according to the target attribute information;

When the target attribute information is matched from the attribute information of each of the subfiles, the subfile corresponding to the target attribute information is determined as the target subfile.
The method according to claim 6, wherein the providing the file data to be accessed read from the target subfile to the client device comprises:

Parsing file data of the target subfile according to meta information of the target subfile;

Reading the to-be-accessed file data from the parsed file data according to the target attribute information;

Transmitting the file data to be accessed to the client device.
A file uploading device, wherein the device is applied to an uploading device that uploads a file in real time to a distributed file system, the device comprising:

An obtaining unit, configured to acquire target file data of the target file;

a packaging unit, configured to encapsulate the target file data into a subfile;

a uploading unit, configured to write at least one of the subfiles to a data file of the target file in the distributed file system, and write update meta information of the data file after the at least one subfile The update meta information of the data file includes attribute information of all subfiles of the target file that have been written.
The device according to claim 9, wherein the obtaining unit comprises:

a period determining subunit, configured to determine whether a preset time period is reached;

And a data obtaining subunit, configured to acquire file data of the target file received in the time period if the time period is reached, and use the acquired file data as the target file data.
The device according to claim 9, wherein the package unit comprises:

a data writing subunit, configured to package the target file data that has been written into the memory according to a subfile encapsulation format;

The meta information adding subunit is configured to generate the subfile after adding the meta information of the target file data after the target file data.
The device according to claim 9, wherein the uploading unit comprises:

a historical meta information obtaining subunit, configured to acquire historical meta information of the data file before the at least one subfile is written to the data file, where the historical meta information includes writing before the at least one subfile Attribute information of the subfile of the target file that is entered;

An attribute information generating subunit, configured to generate attribute information of the at least one subfile in the data file;

An update meta information generating subunit, configured to generate updated meta information of the data file according to attribute information of the subfile included in the historical meta information and attribute information of the at least one subfile;

The update meta information is written to the subunit for writing the update meta information after at least one subfile in the data file.
Apparatus according to any one of claims 9 to 12, characterized in that the attributes of the subfiles Information includes:

The offset information of the subfile in the data file, the length of the subfile, and the number of data rows included in the subfile.
A file access device, wherein the device is configured to access a file uploaded to a distributed file system by using the device of any one of claims 9 to 13, the device comprising:

a receiving unit, configured to receive a file access command of the data file sent by the client device, where the file access command includes target attribute information of the file data to be accessed;

a obtaining unit, configured to obtain, according to the target attribute information and the updated meta information of the data file, a target subfile corresponding to the to-be-accessed file data in the data file;

And an access unit, configured to provide the to-be-accessed file data read from the target sub-file to the client device.
The apparatus according to claim 14, wherein said obtaining unit comprises:

An attribute information matching subunit, configured to match attribute information of each subfile in the update meta information according to the target attribute information;

The subfile determining subunit is configured to determine, as the target subfile, a subfile corresponding to the target attribute information when the target attribute information is matched from the attribute information of each of the subfiles.
The device according to claim 14, wherein the access unit comprises:

a subfile parsing subunit, configured to parse file data of the target subfile according to meta information of the target subfile;

a file data reading subunit, configured to read the to-be-accessed file data from the parsed file data according to the target attribute information;

a file data transmission subunit, configured to transmit the to-be-accessed file data to the client device.
A file uploading device, comprising:

a processor; a memory for storing the processor executable instructions;

Wherein the processor is configured to:

Obtain target file data of the target file;

Encapsulating the target file data into a subfile;

Writing at least one of the subfiles to a data file of the target file in the distributed file system;

The update meta information of the data file is written after the at least one subfile, and the update meta information of the data file includes attribute information of all subfiles of the target file that have been written.
A storage server, comprising:

a processor; a memory for storing the processor executable instructions;

Wherein the processor is configured to:

Receiving a file access command of the data file sent by the client device, where the file access command includes target attribute information of the file data to be accessed;

Obtaining, according to the target attribute information and the updated meta information of the data file, a target subfile corresponding to the to-be-accessed file data in the data file;

The file data to be accessed read from the target subfile is provided to the client device.