WO2017177752A1

WO2017177752A1 - File storage method and device

Info

Publication number: WO2017177752A1
Application number: PCT/CN2017/073080
Authority: WO
Inventors: 丁智勇; 汪渭春; 陈伟; 曹力
Original assignee: 杭州海康威视数字技术股份有限公司
Priority date: 2016-04-14
Filing date: 2017-02-08
Publication date: 2017-10-19
Also published as: CN107301177B; CN107301177A

Abstract

A file storage method and device, the method comprising: receiving a storage request for a file to be stored (S101); obtaining a first file identifier of the file to be stored and a data amount of the file to be stored, and determining a target key used for storing the file to be stored (S102); according to the target key, obtaining metadata to be stored when the file to be stored is stored according to a preset storage format (S103); according to the preset storage format, storing the first file identifier, the data amount and the file to be stored in a value of the target key, and storing the metadata to be stored in a preset metadata library (S104). The method and device effectively ensure metadata recovery, thereby preventing file loss.

Description

File storage method and device

The present application claims priority to Chinese Patent Application No. 20161023061, the entire disclosure of which is incorporated herein in

Technical field

The present application relates to the field of data processing, and in particular, to a file storage method and apparatus.

Background technique

Currently, when storing a file, it is necessary to store the metadata of the file and the file itself. In the prior art, the metadata of the file is generally stored in a metabase of the storage terminal, and the file itself is stored in another location of the storage terminal. When there is a data access requirement, the storage terminal determines the storage location of the file according to the metadata stored in the metadata database, thereby obtaining the file. At this time, if the metadata stored in the metabase is lost, even if the file data corresponding to the lost metadata is read, it cannot be determined that the file data read belongs to those files, thereby causing file loss.

Summary of the invention

The embodiment of the present application discloses a file storage method and device, which ensures metadata recovery and avoids file loss.

To achieve the above objective, the embodiment of the present application discloses a file storage method, which is applied to a storage terminal, and the method includes:

Receiving a storage request for a file to be stored;

Obtaining a first file identifier of the file to be stored and a data amount of the file to be stored, and determining a target key for storing the file to be stored;

Obtaining, according to the target key, metadata to be stored when the file to be stored is stored according to a preset storage format;

And storing, according to the preset storage format, the first file identifier, the data amount, and the to-be-stored file into a value of the target key, and storing the to-be-stored metadata to a preset In the metabase.

Optionally, the first file identifier, the data volume, and the to-be-stored file are stored in the value of the target key according to the preset storage format, including:

And storing, in the order of the first keyword, the file identifier length, and the first file identifier, the first keyword, the file identifier length, and the first file identifier into a value of the target key, where The first keyword is a keyword that is identified by the file, and the file identifier length is a number of bytes occupied by the first file identifier.

And storing, in the order of the second keyword, the data amount, and the file to be stored, the second keyword, the data amount, and the file to be stored in a value of the target key, where The second keyword is a keyword for the file.

Optionally, the method further includes:

In the case that the preset metabase has metadata loss, the preset metabase is updated according to the first keyword and the second keyword stored in the values of all the keys.

Optionally, the updating the preset metadata database according to the first keyword and the second keyword stored in values of all the keys includes:

Obtaining a second file identifier of the file in which the metadata is lost;

Obtaining, according to the second file identifier, a target data segment that includes the second file identifier from values of all keys;

Updating the preset metabase according to the second file identifier and the data amount stored in the target data segment.

Optionally, the obtaining, according to the second file identifier, the target data segment that includes the second file identifier from the values of all the keys, including:

Obtaining, according to the second file identifier, a key K including the identifier of the second file;

Determining whether the first keyword is stored after the second file identifier in the value of the key K;

If yes, obtaining a first keyword W1 that is closest to the second file identifier after the second file identifier in the value of the key K, and determining the first keyword W2 and the first keyword W1 The data segment is a target data segment, wherein the first keyword W2 is: the second of the values of the key K a first keyword that is identified by the file and that is closest to the second file identifier;

If not, it is determined that the data segment between the first keyword W2 and the storage location at the end of the value of the key K is the target data segment.

Optionally, the obtaining the metadata to be stored when the file to be stored is stored according to the preset storage format according to the target key includes:

Obtaining a first keyword and a second keyword stored in a value of the target key;

Calculating an offset of the file to be stored in a value of the target key according to the obtained first keyword and the obtained second keyword;

The metadata to be stored of the file to be stored is determined according to the calculated offset.

To achieve the above objective, the embodiment of the present application further discloses a file storage device, which is applied to a storage terminal, and the device includes: a storage request receiving module, an information determining module, a metadata obtaining module, and a data storage module;

The storage request receiving module is configured to receive a storage request for a file to be stored;

The information determining module is configured to obtain a first file identifier of the file to be stored and a data amount of the file to be stored, and determine a target key for storing the file to be stored;

The metadata obtaining module is configured to obtain, according to the target key, metadata to be stored when the file to be stored is stored according to a preset storage format;

The data storage module is configured to store the first file identifier, the data amount, and the to-be-stored file into a value of the target key according to the preset storage format, and store the The storage metadata is stored in a preset metabase.

Optionally, the data storage module includes:

a file storage submodule, configured to store the first file identifier, the data amount, and the to-be-stored file into a value of the target key according to the preset storage format;

a metadata storage submodule, configured to store the to-be-stored metadata into a preset metadata database;

The file storage submodule includes: a file identifier storage unit and a file storage unit;

The file identifier storage unit is configured to: the first keyword, the file identifier length, and the first file identifier according to a first keyword, a file identifier length, and an order of the first file identifier And storing, in the value of the target key, where the first keyword is a keyword for a file identifier, where the file identifier length is a number of bytes occupied by the first file identifier;

The file storage unit, configured to store the second keyword, the data amount, and the file to be stored to the target according to a second keyword, the data amount, and an order of the file to be stored The value of the key, wherein the second keyword is a keyword for a file.

Optionally, the device further includes: a metabase update module, configured to:

Optionally, the metabase update module includes: a file identifier obtaining submodule, a target data segment determining submodule, and a metabase updating submodule;

The file identifier obtaining submodule is configured to obtain a second file identifier of the file with the metadata lost in the case that the metadata of the preset metabase is lost;

The target data segment determining submodule is configured to obtain, according to the second file identifier, a target data segment that includes the second file identifier from values of all keys;

The meta-database update sub-module is configured to update the preset meta-database according to the second file identifier and the data amount stored in the target data segment.

Optionally, the target data segment determining submodule includes: a key obtaining unit, a keyword determining unit, a first target data segment determining unit, and a second target data segment determining unit;

The key obtaining unit, configured to obtain, according to the second file identifier, a key K that includes the second file identifier;

The keyword determining unit is configured to determine whether the first keyword is stored after the second file identifier in the value of the key K, and if yes, triggering the first target data segment determining unit, if No, the second target data segment determining unit is triggered;

The first target data segment determining unit is configured to obtain a first keyword W1 that is closest to the second file identifier after the second file identifier in the value of the key K, and determine the first keyword W2 and Place The data segment between the first keywords W1 is a target data segment, wherein the first keyword W2 is: the value of the key K is before the second file identifier and is closest to the second file identifier. First keyword;

The second target data segment determining unit is configured to determine a data segment between the first keyword W2 and a storage location at the end of the value of the key K as a target data segment.

Optionally, the metadata obtaining module includes: a keyword obtaining submodule, an offset calculating submodule, and a metadata determining submodule;

The keyword obtaining submodule, configured to obtain a first keyword and a second keyword stored in a value of the target key;

The offset calculation submodule is configured to calculate, according to the obtained first keyword and the obtained second keyword, an offset of the file to be stored in a value of the target key;

The metadata determining submodule is configured to determine metadata to be stored of the file to be stored according to the calculated offset.

To achieve the above objective, an embodiment of the present application further discloses a storage terminal, where the storage terminal includes: a casing, a processor, a memory, a circuit board, and a power supply circuit, wherein the circuit board is disposed in the casing Inside the space, the processor and the memory are disposed on the circuit board; the power circuit is configured to supply power to each circuit or device of the storage terminal; and the memory is used to store executable program code The processor executes the following steps by running executable program code stored in the memory:

Receiving a storage request for a file to be stored;

To achieve the above objective, an embodiment of the present application further discloses an executable program code, where the executable program code is configured to perform the following steps at runtime:

Receiving a storage request for a file to be stored;

To achieve the above objective, the embodiment of the present application further discloses a storage medium for storing executable program code, and the executable program code is executed to perform the following steps:

Receiving a storage request for a file to be stored;

It can be seen that, in the embodiment of the present application, after receiving the storage request for the file to be stored, the storage terminal obtains the first file identifier of the file to be stored and the data amount of the file to be stored, and determines to store the file to be stored. The target key is obtained according to the target key, and the metadata to be stored when the file to be stored is stored according to the preset storage format is obtained, and the first file identifier, the data volume, and the file to be stored are stored in the target key according to the preset storage format. And storing the to-be-stored metadata in a preset metabase. In this way, in the case that the metadata loss of the metadata database, the first file identifier, the data amount, and the stored text of the stored file in the value of the key may be determined according to a preset storage format. The offset of the value in the key, combined with the name of the key, determines the metadata of the stored file, and then restores the metadata in the metadata database according to the metadata, thereby ensuring recovery of the metadata and avoiding Loss of documents.

DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings to be used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present application, and other drawings can be obtained according to the drawings without any creative work for those skilled in the art.

FIG. 1 is a schematic flowchart diagram of a file storage method according to an embodiment of the present application;

2 is a schematic structural diagram of a TLV storage format;

FIG. 3 is a schematic flowchart diagram of another file storage method according to an embodiment of the present disclosure;

FIG. 4 is a schematic flowchart diagram of another file storage method according to an embodiment of the present disclosure;

FIG. 5 is a schematic flowchart diagram of another file storage method according to an embodiment of the present disclosure;

FIG. 6 is a schematic diagram of a file storage arrangement according to an embodiment of the present application;

FIG. 7 is a schematic flowchart diagram of another file storage method according to an embodiment of the present disclosure;

FIG. 8 is a schematic structural diagram of a file storage device according to an embodiment of the present disclosure;

FIG. 9 is a schematic structural diagram of another file storage device according to an embodiment of the present disclosure;

FIG. 10 is a schematic structural diagram of another file storage device according to an embodiment of the present disclosure;

FIG. 11 is a schematic structural diagram of another file storage device according to an embodiment of the present disclosure;

FIG. 12 is a schematic structural diagram of another file storage device according to an embodiment of the present disclosure.

detailed description

The technical solutions in the embodiments of the present application are clearly and completely described in the following with reference to the drawings in the embodiments of the present application. It is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.

The present application will be described in detail below through specific embodiments.

Referring to FIG. 1 , FIG. 1 is a schematic flowchart of a file storage method according to an embodiment of the present disclosure, which is applied to a storage terminal, and the method may include the following steps:

S101: Receive a storage request for a file to be stored;

Here, the storage request includes a file to be stored, and may further include an identifier of the file to be stored, so that the storage terminal can quickly obtain the identifier of the file to be stored, and then store the metadata of the file to be stored; or, the storage request may also be The storage terminal does not include the identifier of the file to be stored, and the storage terminal can parse the identifier of the file to be stored in the file to be stored, so that the storage request is simpler and clearer.

S102: Obtain a first file identifier of the file to be stored and a data amount of the file to be stored, and determine a target key for storing the file to be stored;

In an embodiment of the present application, the first file identifier and the data amount of the file to be stored are part of the metadata of the file to be stored, and may be directly obtained when the storage request is obtained, and therefore, the storage is to be stored. Before the file, the first file identifier and the amount of data of the file to be stored are obtained first.

In one embodiment of the present application, the massive data storage with high reliability and scalability is a huge challenge for the current storage terminal, and the traditional database is often difficult to meet the requirement. In this case, the relationship is used. Databases will make storing and reading data inefficient, and scaling databases will be a big problem. In such cases, using key storage would be a good choice. Key storage has the following advantages:

1. Availability availability;

2. Scalability scalability;

3, Failover failure recovery;

4, Performance high performance.

The key consists of the name Key and the value Value, and a Key corresponds to a Value.

S103: Obtain metadata to be stored when the file to be stored is stored according to a preset storage format according to the target key.

Generally, the metadata of the file to be stored generally includes metadata information such as an identifier of the file to be stored in the target key, an amount of data, and an offset from a start storage location of the target key.

In one embodiment of the present application, S103 can be implemented by the following process:

Because the target key and the storage format of the file to be stored are determined, the storage location of the file to be stored in the target key may be determined according to the storage format. At this time, it may be determined that the file to be stored is stored relative to the target key. The offset of the location, such that the metadata of the file to be stored, that is, the metadata to be stored, can be obtained by combining the obtained first file identifier, the amount of data, the offset, and the name of the target key.

In an embodiment of the present application, the preset storage format may be a format of TLV (TAG_LENGTH_VALUE, mark_length_value), as shown in FIG. 2, and FIG. 2 is a schematic structural diagram of a TLV storage format. For the sake of simplicity.

S104: Store the first file identifier of the file to be stored, the data volume of the file to be stored, and the file to be stored in the value of the target key according to the preset storage format, and store the metadata to be stored in the preset metabase. in.

The preset metabase is used to store metadata of the file.

In an embodiment of the present application, the storage terminal may store a plurality of files having a small capacity relative to the value of the key, such as: the value of the key has a capacity of 1 Mbytes, and the size of the file to be stored is 100 Kbytes. At this time, if the value of a key is Storing only one file in it will easily cause a waste of storage space, so you can consider storing multiple complete files in the value of one key.

When storing files, if you only store the metadata of the file in the metabase, even if the file is stored as a key, whether a full file is stored in the value of a key, multiple complete files are stored. Once the metadata of a file in the metabase is lost, even if the data of the file whose metadata is lost is read from the value of the corresponding key, due to the lack of metadata information, it is impossible to know that the read data is the metadata lost. The data of the file.

In an embodiment of the present application, the storage device obtains the first file identifier of the file to be stored, the data amount of the file to be stored, and the file to be stored, and stores the value in the target key according to a preset storage format, so that If the meta-database has metadata loss, the data belonging to the same file may be determined in the value of the key according to the storage format, and according to the storage format, the obtained data is obtained. The metadata information such as the file identifier and the data amount of the file is stored, and the metadata stored in the metadata database is restored.

Generally, after the storage terminal determines the target key to which the data to be stored is to be stored, metadata information such as the offset amount in the value of the target key when the data to be stored is stored according to the preset storage format can be obtained, and integrated. The data identifier of the file to be stored may be determined by the first file identifier and the data volume of the file to be stored, and the metadata to be stored is stored in a preset metadata database. In this way, when data access is performed, the storage terminal can perform data access according to the metadata in the metadata database.

In an implementation manner of the present application, after successfully storing the file to be stored, the storage terminal stores the metadata of the file to be stored, thereby ensuring the accuracy of the metadata in the metadata database, and avoiding unsuccessful storage to be stored. The file stores the metadata of the file to be stored, and the storage terminal cannot correctly read the metadata stored in the metadata database, and the storage terminal library crashes.

It should be noted that the foregoing step of storing the file to be stored may be performed simultaneously with the step of storing the metadata to be stored, or may be performed after storing the metadata to be stored before S102, or may be performed after storing the metadata to be stored, this application This is not limited.

Applying the embodiment shown in FIG. 1 , after receiving the storage request for the file to be stored, the storage terminal obtains the first file identifier of the file to be stored and the data amount of the file to be stored, and determines a target key for storing the file to be stored. According to the target key, the metadata to be stored when the file to be stored is stored according to the preset storage format is obtained, and the first file identifier, the data volume, and the file to be stored are stored in the target key according to the preset storage format, and The metadata to be stored is stored in a preset metabase. In this way, in the case that the metadata is lost in the metadata database, when the data is read from a key, the first file identifier, the data amount, and the amount of the stored file in the value of the key can be determined according to a preset storage format. The offset of the stored file in the value of the key, combined with the name of the key, determines the metadata of the stored file, and then restores the metadata in the metadata database according to the metadata, thereby ensuring recovery of the metadata. Avoid the loss of files.

Referring to FIG. 3, FIG. 3 is a schematic flowchart of another file storage method according to an embodiment of the present disclosure, which is applied to a storage terminal. In the method, step S104 may include the following steps:

S1041: According to the first keyword, the file identifier length, and the first file identifier of the file to be stored Sequence, storing the first keyword, the file identifier length, and the first file identifier of the file to be stored into the value of the target key;

The first keyword is a keyword for the file identifier, and the file identifier length is the number of bytes occupied by the first file identifier.

In FIG. 2, the TAG may be used to store the first keyword, and the first keyword may be TAG_FILE_NAME, and other characters may be used to represent the first keyword, which is not limited in this embodiment of the present application. After the file is stored in the TLV format, if the storage terminal reads TAG_FILE_NAME, it indicates that the length of the file identifier stored in LENGTH is 4, for example, 4 stored in LENGTH, and 4 when stored in LENGTH. VALUE then takes up 4 bytes, which in turn can read a file identifier that occupies 4 bytes in VALUE.

S1042: Store the second keyword, the data volume, and the file to be stored in the value of the target key according to the second keyword, the data volume of the file to be stored, and the order of the file to be stored;

The second keyword is a keyword for the file.

Generally, in FIG. 3, the TAG may also be used to store the second keyword, and the second keyword may be TAG_FILE_DATA, and the other keywords may be used to represent the second keyword, which is not limited in this embodiment of the present application. After the file is stored in the TLV format, if the storage terminal reads TAG_FILE_DATA, it indicates the amount of data stored in the LENGTH file, for example, 200 stored in LENGTH, and 200 when stored in LENGTH. It means that the VALUE takes up 200 bytes, and then it can read the file that takes up 200 bytes in VALUE.

S1043: Store the metadata to be stored into a preset metabase.

Applying the embodiment shown in FIG. 3, the storage terminal stores the first keyword, the file identifier length, and the first file identifier of the file to be stored according to the first keyword, the file identifier length, and the first file identifier of the file to be stored. Up to the value of the target key, and storing the second keyword, the data amount, and the file to be stored in the value of the target key according to the second keyword, the data amount of the file to be stored, and the order of the file to be stored, to be stored The metadata is stored in a preset metabase. In this way, in the case that the metadata loss of the metadata database, it is still possible to determine which file the read data belongs to according to the identifier and the data amount of the file stored in the value of the key according to the preset storage format, effectively Avoided the text The loss of pieces.

Referring to FIG. 4, FIG. 4 is a schematic flowchart of another file storage method according to an embodiment of the present disclosure, which is applied to a storage terminal, and the method may include the following steps:

S105: In a case where the metadata of the preset metabase is lost, the preset metabase is updated according to the first keyword and the second keyword stored in the values of all the keys.

In an embodiment of the present application, each time the metadata database is updated, a first check value is obtained, and a second check value may be obtained again before the next time the metadata is stored, when the first check is performed. When the value is different from the second check value, it can be said that the metabase has metadata loss. Currently, the check value can be obtained according to a CRC (Cyclic Redundancy Check) check algorithm or other check algorithm.

In an embodiment of the present application, when it is determined that the metabase has metadata loss, the identifier and data of the stored file in the value of the key may be obtained according to the first keyword and the second keyword stored in the values of all the keys. The amount of metadata information, and then the metadata database is updated according to the obtained metadata information, which ensures the recovery of the metadata in the metadata database and avoids file loss.

It should be noted that step S105 may occur before any of steps S101, S102, S103, and S104, and may also occur after any of steps S101, S102, S103, and S104, and may also be followed by step S101. Any of the steps S102, S103, and S104 occur at the same time, which is not limited by the examples in the present application.

Applying the embodiment shown in FIG. 4, the storage terminal obtains the identifier of the stored file and the stored file according to the preset storage format when the metadata is lost from the preset key database. The amount of data, according to the first keyword and the second keyword stored in the values of all the keys, obtaining the offset of the stored file in the value of the key, and updating the preset metabase according to the obtained information, can effectively Restore metadata in the metabase to avoid file loss.

Referring to FIG. 5, FIG. 5 is a schematic flowchart of another file storage method according to an embodiment of the present disclosure, which is applied to a storage terminal. In the method, step S105 may include the following steps:

S1051: Obtain a second file identifier of the file in which the metadata is lost in the case that the metadata of the preset metabase is lost.

In one embodiment of the present application, the metadata lost in the metabase may be only a small part. In this case, if all the metadata stored in the metabase is updated, the workload of the storage terminal will be increased, and the metadata is The recovery speed is slow. If only the missing metadata in the metabase is updated, the workload of the storage terminal will be greatly reduced, and the metadata recovery speed will be fast.

Generally, when the storage terminal is unable to read the file for reading, the metadata of the file in the metadata database may be considered to be lost, thereby obtaining the second file identifier of the file. In one implementation of the present application, the second file identification of the file can be obtained from a read request when the file is read.

S1052: Obtain a target data segment that includes the second file identifier from the values of all the keys according to the second file identifier of the lost file;

In an embodiment of the present application, the obtaining, according to the second file identifier of the lost file, the target data segment that includes the second identifier from the file stored by the value of all the keys may include:

S1, according to the second file identifier, obtain a key K containing the second file identifier;

Generally, the first keyword is read in the value of all the keys, and then the corresponding file identifier length is determined according to the first keyword, and then the file identifier is determined according to the file identifier length, and the second file is obtained from the determined file identifier. The same file identifier and the key corresponding to the file identifier are identified, and the key obtained here is the key containing the second file identifier.

It should be noted that the above-mentioned key K may be a plurality of keys, or may be only one key. When the above-mentioned key K is a plurality of keys, it is explained that one file is stored in a plurality of keys.

S2, determining whether the first key is stored in the value of the key K after the second file identification, if yes, proceeding to step S3, if not, executing step S4;

Generally, the identifier of the file, the amount of data, and the order of the file itself may be: first store the identifier of the file, then store the data amount of the file and the file itself, and the first keyword is a data segment storing a file. Starting storage location. As shown in FIG. 6, FIG. 6 is a schematic diagram of a file storage arrangement according to an embodiment of the present application. As can be seen from FIG. 6, two files are stored in the value of the key named Key1, and file2 is stored after file1, and no file is stored after file2. Therefore, the first keyword A (TAG_FILE_NAME) corresponding to file1 and The space between the first key B (TAG_FILE_NAME) corresponding to file2 is the data segment storing file1, and the first key corresponding to file2 The space between the word B and the end of the value of the key named Key1 is the data segment in which file2 is stored. It can be seen from the above that only knowing whether the file identifier contains the first keyword, that is, only knowing whether the file is stored after the file, can know how to determine the data segment storing a file, and each file corresponds to a key Word, therefore, it is necessary to determine whether the first keyword is stored after the second file identification.

S3, obtaining a first keyword W1 that is the closest to the second file identifier after the second file identifier in the value of the key K, and determining that the data segment between the first keyword W2 and the first keyword W1 is the target data segment, where The first keyword W2 is: a first keyword in the value of the key K before the second file identifier and closest to the second file identifier;

Generally, the first keyword is a data segment starting storage location for storing a file, a file identifier corresponds to a first keyword, and a space between two adjacent first keywords is a data segment, where the file The identifier corresponding to the first keyword is: the first keyword before the file identifier and closest to the file identifier. After the target data segment is determined, the file identifier length may be obtained according to the first keyword stored in the target data segment, and then the file identifier is determined according to the file identifier length; and the second identifier may be stored according to the target data segment. The keyword gets the amount of data for the file and gets the file. Here, the file identifier and the data amount of the file are metadata, so that the metadata in the metadata database can be restored according to the metadata obtained in the value of the key.

S4. Determine a data segment between the first keyword W2 and the end storage location of the value of the key K as the target data segment.

Generally, after the file is identified, the first keyword does not exist, and the file is not stored after the file is deleted. Therefore, the first keyword W2 corresponding to the file identifier and the last storage location of the value of the key K where the file identifier is located may be determined. The data segment between them is the target data segment.

S1053: Update the preset metabase according to the second file identifier of the lost file stored in the target data segment and the data amount of the lost file.

Applying the embodiment shown in FIG. 5, the storage terminal only needs to obtain the second file identifier of the file whose metadata is lost, and obtain the second identifier from the file stored in the value of all the keys according to the second file identifier of the lost file. The target data segment is finally updated according to the second file identifier of the lost file stored in the target data segment and the data amount of the lost file, and the default metadata database is updated, which is reduced. The workload of the storage terminal improves the efficiency of the metadata database update.

Referring to FIG. 7, FIG. 7 is a schematic flowchart of another file storage method according to an embodiment of the present disclosure, which is applied to a storage terminal. In the method, step S103 may include the following steps:

S1031: Obtain a first keyword and a second keyword stored in a value of the target key;

In one embodiment of the present application, the number of bytes occupied by the TAG and LENGTH in the file stored in the TLV format is a fixed size, so only the first keyword and the second keyword stored in the value of the target key are required (also That is, TAG), it is determined according to the number of the first keyword and the second keyword and the file identifier length corresponding to the first keyword and the data amount corresponding to the second keyword (ie, LENGTH), to determine the to-be-stored The offset of the file in the value of the target key.

S1032: Calculate, according to the obtained first keyword and the obtained second keyword, an offset of the file to be stored in the value of the target key;

According to FIG. 6, for example, the number of bytes occupied by TAG and LENGTH is 4Kbytes, the identifier length of file1 is 10Kbytes, the data amount of file1 is 100Kbytes, the identification length of file2 is 12Kbytes, the data volume of file2 is 200Kbytes, file1 and The storage order of file2 is as follows: first store file1 and then store file2. The metadata of file1 and file2 obtained at this time are shown in Table 1.

Table 1

Generally, the offset of file1 and file2 in the value of the key is calculated by storing the file1 in the value of the target key named Key1, the value of the target key is empty, and at this time, it can be determined The starting storage location of the value of the target key has a first keyword and a second keyword between the file headers of the file1 file (this is the first keyword and the default storage format carried by the TLV itself) The obtained second keyword), that is, there are 2 TAGs, corresponding to 2 LENGTH, from LENGTH, the file length of file1 can be 10Kbytes, and the data amount of file1 is 100Kbytes. Therefore, the value of file1 at Key1 can be determined. The offset in the is: 2 * 4Kbytes (the number of bytes occupied by 2 TAG) + 2 * 4Kbytes (2 bytes occupied by LENGTH) +10Kbytes (the number of bytes occupied by the file1 identifier) = 8Kbytes + 8Kbytes + 10Kbytes = 26Kbytes; in addition, when storing file2 in the value of the target key named Key1, the target The file1 is stored in the value of the key, and one first keyword and one second keyword are obtained, and one first keyword and one second keyword are further included (this is the preset storage format TLV itself). The first keyword carried and the obtained second keyword), that is, there are 4 TAGs, corresponding to 4 LENGTH, and the file length of file1 is 10Kbytes, the data amount of file1 is 100Kbytes, and file2 is obtained from LENGTH. The identifier length is 12Kbytes, and the data volume of file2 is 200Kbytes. Therefore, it can be determined that the offset of file2 in the value of Key1 is: 4*4Kbytes (the number of bytes occupied by 2 TAGs) + 4*4Kbytes (2 LENGTH occupations) Bytes) +10Kbytes (the number of bytes occupied by the file1 identifier) +100Kbytes (the number of bytes occupied by file1) +12Kbytes (the number of bytes occupied by the identifier of file2)=16Kbytes+16Kbytes+10Kbytes+100Kbytes+12Kbytes= 154Kbytes.

S1033: Determine, according to the calculated offset, the metadata to be stored of the file to be stored;

In one embodiment of the present application, the offset of the file in the value of the key is also part of the metadata of the file, so that the storage terminal quickly obtains the storage location of the file according to the offset.

Generally, after obtaining the offset of the file to be stored in the value of the target key, the metadata of the file to be stored is determined in combination with the first file identifier and the data amount of the file to be stored.

Applying the embodiment shown in FIG. 7, the storage terminal obtains the first keyword and the second keyword stored in the value of the target key, and calculates the file to be stored in the target key according to the obtained first keyword and the obtained second keyword. The offset in the value, and based on the calculated offset, determine the metadata to be stored of the file to be stored. Here, the offset of the file is calculated according to the first keyword and the second keyword stored in the value of the target key, thereby determining the metadata to be stored of the file to be stored, without having to count the storage location and target of the file to be stored one by one. The amount of data between the starting storage locations of the values of the keys to determine the offset of the file, the determination of the metadata is faster and more convenient.

Referring to FIG. 8, FIG. 8 is a schematic structural diagram of a file storage device according to an embodiment of the present disclosure, which is applied to a storage terminal, and the device may include: a storage request receiving module 801, an information determining module 802, a metadata obtaining module 803, and data. Storage module 804;

The storage request receiving module 801 is configured to receive a storage request for a file to be stored;

The information determining module 802 is configured to obtain a first file identifier of the file to be stored and a file to be stored. The amount of data, and determine the target key for storing the file to be stored;

The metadata obtaining module 803 is configured to obtain, according to the target key, metadata to be stored when the file to be stored is stored according to a preset storage format;

The data storage module 804 is configured to store the first file identifier, the data amount, and the to-be-stored file into a value of the target key according to the preset storage format, and store the to-be-stored The metadata is stored in a preset metabase, wherein a preset metabase is used to store metadata of the file.

Applying the embodiment shown in FIG. 8 , after receiving the storage request for the file to be stored, the storage terminal obtains the first file identifier of the file to be stored and the data amount of the file to be stored, and determines a target key for storing the file to be stored. According to the target key, the metadata to be stored when the file to be stored is stored according to the preset storage format is obtained, and the first file identifier, the data volume, and the file to be stored are stored in the target key according to the preset storage format, and The metadata to be stored is stored in a preset metabase. In this way, in the case that the metadata is lost in the metadata database, when the data is read from a key, the first file identifier, the data amount, and the amount of the stored file in the value of the key can be determined according to a preset storage format. The offset of the stored file in the value of the key, combined with the name of the key, determines the metadata of the stored file, and then restores the metadata in the metadata database according to the metadata, thereby ensuring recovery of the metadata. Avoid the loss of documents.

Referring to FIG. 9, FIG. 9 is a schematic structural diagram of another file storage device according to an embodiment of the present disclosure, which is applied to a storage terminal. In the device, the data storage module 804 may include: a file identifier storage submodule 8041 and a metadata storage device. Module 8042;

a file storage sub-module 8041, configured to store the first file identifier, the data amount, and the to-be-stored file into a value of the target key according to the preset storage format;

a metadata storage sub-module 8042, configured to store the to-be-stored metadata into a preset metadata database;

The file storage submodule 8041 includes: a file identifier storage unit 8041a and a file storage unit 8041b;

The file identifier storage unit 8041a is configured to store the first keyword, the file identifier length, and the first file identifier to the target key according to the first keyword, the file identifier length, and the first file identifier. The value of the first keyword is a keyword for the file identifier, and the file identifier length is the number of bytes occupied by the first file identifier;

The file storage unit 8041b is configured to store the second keyword, the data amount, and the file to be stored in the value of the target key according to the second keyword, the data amount, and the order of the file to be stored, where the second keyword is for The keyword of the file.

Applying the embodiment shown in FIG. 9 , the storage terminal stores the first keyword, the file identifier length, and the first file identifier of the file to be stored according to the first keyword, the file identifier length, and the first file identifier of the file to be stored. Up to the value of the target key, and storing the second keyword, the data amount, and the file to be stored in the value of the target key according to the second keyword, the data amount of the file to be stored, and the order of the file to be stored, to be stored The metadata is stored in a preset metabase. In this way, in the case that the metadata loss of the metadata database, it is still possible to determine which file the read data belongs to according to the identifier and the data amount of the file stored in the value of the key according to the predetermined storage format, which effectively avoids Loss of documents.

FIG. 10 is a schematic structural diagram of another file storage device according to an embodiment of the present disclosure, which is applied to a storage terminal, and the device may further include: a metabase update module 805;

Here, the metabase update module 805 is configured to:

In the case where the metadata of the preset metabase is lost, the preset metabase is updated according to the first keyword and the second keyword stored in the values of all the keys.

Applying the embodiment shown in FIG. 10, the storage terminal obtains the identifier of the stored file and the stored file according to the preset storage format when the data is read from each key in the case that the metadata is lost in the preset metabase. The amount of data, according to the first keyword and the second keyword stored in the values of all the keys, determine the offset of each stored file in the value of each key, and update the preset metabase according to the obtained information, which can be effective Restore metadata in the metabase to avoid file loss.

Referring to FIG. 11 , FIG. 11 is a schematic structural diagram of another file storage device according to an embodiment of the present disclosure, which is applied to a storage terminal. In the device, the meta-database update module 805 may include: a file identifier obtaining sub-module 8051 and a target data segment. Determining submodule 8052 and metabase update submodule 8053;

The file identifier obtaining sub-module 8051 is configured to obtain a second file identifier of the file in which the metadata is lost if the metadata of the preset meta-database is lost.

a target data segment determining sub-module 8052, configured to obtain, according to the second file identifier, a target data segment that includes the second file identifier from values of all the keys;

The metabase update submodule 8053 is configured to update the preset metabase according to the second file identifier and the amount of data stored in the target data segment.

In an implementation manner of the present application, the target data segment determining submodule 8052 may include: a key obtaining unit, a keyword determining unit, a first target data segment determining unit, and a second target data segment determining unit (not shown in FIG. 11 show);

The key obtaining unit is configured to obtain, according to the second file identifier, a key K that includes the second file identifier;

a keyword determining unit, configured to determine whether a first keyword is stored after the second file identifier in the value of the key K, and if yes, triggering the first target data segment determining unit, and if not, triggering the second target data Segment determination unit;

a first target data segment determining unit, configured to obtain a first keyword W1 that is closest to the second file identifier after the second file identifier in the value of the key K, and determine data between the first keyword W2 and the first keyword W1 The segment is a target data segment, where the first keyword W2 is: a first keyword in the value of the key K before the second file identifier and closest to the second file identifier;

The second target data segment determining unit is configured to determine the data segment between the first keyword W2 and the storage location at the end of the value of the key K as the target data segment.

Applying the embodiment shown in FIG. 11, the storage terminal only needs to obtain the second file identifier of the file whose metadata is lost, and obtain the second identifier from the file stored in the value of all the keys according to the second file identifier of the lost file. The target data segment finally updates the preset metabase according to the second file identifier of the lost file stored in the target data segment and the data amount of the lost file, thereby reducing the workload of the storage terminal and improving the metabase. Updated efficiency.

Referring to FIG. 12, FIG. 12 is a schematic structural diagram of another file storage apparatus according to an embodiment of the present disclosure, which is applied to a storage terminal. In the apparatus, the metadata obtaining module 803 may include: a keyword obtaining submodule 8031, and an offset. a quantity calculation sub-module 8032, a metadata determination sub-module 8033;

The keyword obtaining submodule 8031 is configured to obtain the first stored in the value of the target key. Keyword and second keyword;

The offset calculation sub-module 8032 is configured to calculate, according to the obtained first keyword and the obtained second keyword, an offset of the file to be stored in the value of the target key;

The metadata determining sub-module 8033 is configured to determine metadata to be stored of the file to be stored according to the calculated offset.

Applying the embodiment shown in FIG. 12, the storage terminal obtains the first keyword and the second keyword stored in the value of the target key, and calculates the file to be stored in the target key according to the obtained first keyword and the obtained second keyword. The offset in the value, and based on the calculated offset, determine the metadata to be stored of the file to be stored. Here, the offset of the file is calculated according to the first keyword and the second keyword stored in the value of the target key, thereby determining the metadata to be stored of the file to be stored, without having to count the storage location and target of the file to be stored one by one. The amount of data between the starting storage locations of the values of the keys to determine the offset of the file, the determination of the metadata is faster and more convenient.

The embodiment of the present application provides a storage terminal, including: a casing, a processor, a memory, a circuit board, and a power supply circuit, wherein the circuit board is disposed inside a space enclosed by the casing, The processor and the memory are disposed on the circuit board; the power supply circuit is configured to supply power to each circuit or device of the storage terminal; the memory is configured to store executable program code; Executing executable program code stored in the memory to perform the following steps:

Receiving a storage request for a file to be stored;

Applying the embodiment of the present application, after receiving the storage request for the file to be stored, the storage terminal Obtaining a first file identifier of the file to be stored and a data volume of the file to be stored, and determining a target key for storing the file to be stored, and obtaining metadata to be stored when the file to be stored is stored according to a preset storage format according to the target key And storing the first file identifier, the data volume, and the file to be stored in the target key according to the preset storage format, and storing the to-be-stored metadata in a preset metabase. In this way, in the case that the metadata loss of the metadata database, the first file identifier, the data amount of the stored file in the value of the key, and the stored file in the value of the key may be determined according to a preset storage format. The offset, combined with the name of the key, determines the metadata of the stored file, and then restores the metadata in the metadata database according to the metadata, thereby ensuring recovery of the metadata and avoiding file loss.

The storage terminal exists in various forms including, but not limited to:

(1) Mobile communication devices: These devices are characterized by mobile communication functions and are mainly aimed at providing voice and data communication. Such terminals include: smart phones (such as iPhone), multimedia phones, functional phones, and low-end phones.

(2) Ultra-mobile personal computer equipment: This type of equipment belongs to the category of personal computers, has computing and processing functions, and generally has mobile Internet access. Such terminals include: PDAs, MIDs, and UMPC devices, such as the iPad.

(3) Portable entertainment devices: These devices can display and play multimedia content. Such devices include: audio, video players (such as iPod), handheld game consoles, e-books, and smart toys and portable car navigation devices.

(4) Server: A device that provides computing services. The server consists of a processor, a hard disk, a memory, a system bus, etc. The server is similar to a general-purpose computer architecture, but because of the need to provide highly reliable services, processing power and stability High reliability in terms of reliability, security, scalability, and manageability.

(5) Other electronic devices with data interaction functions.

An embodiment of the present application provides an executable program code for performing the following steps at runtime:

Receiving a storage request for a file to be stored;

Applying the storage request for the file to be stored, obtaining the first file identifier of the file to be stored and the data amount of the file to be stored, and determining a target key for storing the file to be stored, according to the target key. And obtaining the to-be-stored metadata when the file to be stored is stored according to the preset storage format, storing the first file identifier, the data volume, and the file to be stored in the target key according to the preset storage format, and storing the to-be-stored element The data is stored in a preset metabase. In this way, in the case that the metadata loss of the metadata database, the first file identifier, the data amount of the stored file in the value of the key, and the stored file in the value of the key may be determined according to a preset storage format. The offset, combined with the name of the key, determines the metadata of the stored file, and then restores the metadata in the metadata database according to the metadata, thereby ensuring recovery of the metadata and avoiding file loss.

Embodiments of the present application provide a storage medium for storing executable program code, the executable program code being executed to perform the following steps:

Receiving a storage request for a file to be stored;

After the storage request for the file to be stored is received, the application is obtained. And storing the first file identifier of the file and the data volume of the file to be stored, and determining a target key for storing the file to be stored, and obtaining the metadata to be stored when the file to be stored is stored according to the preset storage format according to the target key, according to the target key The preset storage format stores the first file identifier, the data volume, and the file to be stored in the target key, and stores the to-be-stored metadata in a preset metabase. In this way, in the case that the metadata loss of the metadata database, the first file identifier, the data amount of the stored file in the value of the key, and the stored file in the value of the key may be determined according to a preset storage format. The offset, combined with the name of the key, determines the metadata of the stored file, and then restores the metadata in the metadata database according to the metadata, thereby ensuring recovery of the metadata and avoiding file loss.

For the device, the storage terminal, the executable program code and the storage medium embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.

It should be noted that, in this context, relational terms such as first and second are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply such entities or operations. There is any such actual relationship or order between them. Furthermore, the term "comprises" or "comprises" or "comprises" or any other variations thereof is intended to encompass a non-exclusive inclusion, such that a process, method, article, or device that comprises a plurality of elements includes not only those elements but also Other elements, or elements that are inherent to such a process, method, item, or device. An element that is defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in the process, method, item, or device that comprises the element.

One of ordinary skill in the art can understand that all or part of the steps in implementing the above method embodiments can be completed by a program to instruct related hardware, and the program can be stored in a computer readable storage medium, which is referred to herein. Storage media such as ROM/RAM, disk, CD, etc.

The above description is only the preferred embodiment of the present application, and is not intended to limit the scope of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application are included in the scope of the present application.

Claims

A file storage method is applied to a storage terminal, and the method includes:

Receiving a storage request for a file to be stored;

Obtaining a first file identifier of the file to be stored and a data amount of the file to be stored, and determining a target key for storing the file to be stored;

Obtaining, according to the target key, metadata to be stored when the file to be stored is stored according to a preset storage format;

And storing, according to the preset storage format, the first file identifier, the data amount, and the to-be-stored file into a value of the target key, and storing the to-be-stored metadata to a preset In the metabase.
The method according to claim 1, wherein the storing the first file identifier, the data amount, and the file to be stored to a value of the target key according to the preset storage format Including:

And storing, in the order of the first keyword, the file identifier length, and the first file identifier, the first keyword, the file identifier length, and the first file identifier into a value of the target key, where The first keyword is a keyword that is identified by the file, and the file identifier length is a number of bytes occupied by the first file identifier.

And storing, in the order of the second keyword, the data amount, and the file to be stored, the second keyword, the data amount, and the file to be stored in a value of the target key, where The second keyword is a keyword for the file.
The method of claim 2, wherein the method further comprises:

In the case that the preset metabase has metadata loss, the preset metabase is updated according to the first keyword and the second keyword stored in the values of all the keys.
The method according to claim 3, wherein the updating the preset metadata database according to the first keyword and the second keyword stored in values of all the keys comprises:

Obtaining a second file identifier of the file in which the metadata is lost;

Obtaining, according to the second file identifier, a target data segment that includes the second file identifier from values of all keys;

Updating the preset metabase according to the second file identifier and the data amount stored in the target data segment.
The method according to claim 4, wherein the obtaining, according to the second file identifier, the target data segment including the second file identifier from the values of all the keys comprises:

Obtaining, according to the second file identifier, a key K including the identifier of the second file;

Determining whether the first keyword is stored after the second file identifier in the value of the key K;

If yes, obtaining a first keyword W1 that is closest to the second file identifier after the second file identifier in the value of the key K, and determining the first keyword W2 and the first keyword W1 The data segment is a target data segment, wherein the first keyword W2 is: a first keyword that is before the second file identifier and is closest to the second file identifier in the value of the key K;

If not, it is determined that the data segment between the first keyword W2 and the storage location at the end of the value of the key K is the target data segment.
The method according to claim 2, wherein the obtaining, according to the target key, the metadata to be stored when the file to be stored is stored according to a preset storage format comprises:

Obtaining a first keyword and a second keyword stored in a value of the target key;

Calculating an offset of the file to be stored in a value of the target key according to the obtained first keyword and the obtained second keyword;

The metadata to be stored of the file to be stored is determined according to the calculated offset.
A file storage device is applied to a storage terminal, and the device includes: a storage request receiving module, an information determining module, a metadata obtaining module, and a data storage module;

The storage request receiving module is configured to receive a storage request for a file to be stored;

The information determining module is configured to obtain a first file identifier of the file to be stored and a data amount of the file to be stored, and determine a target key for storing the file to be stored;

The metadata obtaining module is configured to obtain, according to the target key, the file to be stored according to the Metadata to be stored when the preset storage format is stored;

The data storage module is configured to store the first file identifier, the data amount, and the to-be-stored file into a value of the target key according to the preset storage format, and store the The storage metadata is stored in a preset metabase.
The device according to claim 7, wherein the data storage module comprises:

a file storage submodule, configured to store the first file identifier, the data amount, and the to-be-stored file into a value of the target key according to the preset storage format;

a metadata storage submodule, configured to store the to-be-stored metadata into a preset metadata database;

The file storage submodule includes: a file identifier storage unit and a file storage unit;

The file identifier storage unit is configured to: the first keyword, the file identifier length, and the first file identifier according to a first keyword, a file identifier length, and an order of the first file identifier And storing, in the value of the target key, where the first keyword is a keyword for a file identifier, where the file identifier length is a number of bytes occupied by the first file identifier;

The file storage unit, configured to store the second keyword, the data amount, and the file to be stored to the target according to a second keyword, the data amount, and an order of the file to be stored The value of the key, wherein the second keyword is a keyword for a file.
The device according to claim 8, wherein the device further comprises: a metabase update module, configured to:

In the case that the preset metabase has metadata loss, the preset metabase is updated according to the first keyword and the second keyword stored in the values of all the keys.
The apparatus according to claim 9, wherein the metabase update module comprises: a file identifier obtaining submodule, a target data segment determining submodule, and a metabase updating submodule;

The file identifier obtaining submodule is configured to obtain a second file identifier of the file with the metadata lost in the case that the metadata of the preset metabase is lost;

The target data segment determining submodule is configured to obtain, according to the second file identifier, a target data segment that includes the second file identifier from values of all keys;

The meta-database update sub-module is configured to update the preset meta-database according to the second file identifier and the data amount stored in the target data segment.
The apparatus according to claim 10, wherein the target data segment determining submodule comprises: a key obtaining unit, a keyword determining unit, a first target data segment determining unit, and a second target data segment determining unit;

The key obtaining unit, configured to obtain, according to the second file identifier, a key K that includes the second file identifier;

The keyword determining unit is configured to determine whether the first keyword is stored after the second file identifier in the value of the key K, and if yes, triggering the first target data segment determining unit, if No, the second target data segment determining unit is triggered;

The first target data segment determining unit is configured to obtain a first keyword W1 that is closest to the second file identifier after the second file identifier in the value of the key K, and determine the first keyword W2 and The data segment between the first keywords W1 is a target data segment, wherein the first keyword W2 is: the value of the key K is before the second file identifier and is closest to the second file identifier First keyword

The second target data segment determining unit is configured to determine a data segment between the first keyword W2 and a storage location at the end of the value of the key K as a target data segment.
The apparatus according to claim 8, wherein the metadata obtaining module comprises: a keyword obtaining submodule, an offset calculating submodule, and a metadata determining submodule;

The keyword obtaining submodule, configured to obtain a first keyword and a second keyword stored in a value of the target key;

The offset calculation submodule is configured to calculate, according to the obtained first keyword and the obtained second keyword, an offset of the file to be stored in a value of the target key;

The metadata determining submodule is configured to determine metadata to be stored of the file to be stored according to the calculated offset.
A storage terminal, comprising: a housing, a processor, a memory, a circuit board, and a power supply circuit, wherein the circuit board is disposed in a space enclosed by the housing Internally, the processor and the memory are disposed on the circuit board; the power supply circuit is configured to supply power to each circuit or device of the storage terminal; the memory is configured to store executable program code; The processor performs the following steps by running executable program code stored in the memory:

Receiving a storage request for a file to be stored;

Obtaining a first file identifier of the file to be stored and a data amount of the file to be stored, and determining a target key for storing the file to be stored;

Obtaining, according to the target key, metadata to be stored when the file to be stored is stored according to a preset storage format;

And storing, according to the preset storage format, the first file identifier, the data amount, and the to-be-stored file into a value of the target key, and storing the to-be-stored metadata to a preset In the metabase.
An executable program code, wherein the executable program code is configured to perform the following steps at runtime:

Receiving a storage request for a file to be stored;

Obtaining a first file identifier of the file to be stored and a data amount of the file to be stored, and determining a target key for storing the file to be stored;

Obtaining, according to the target key, metadata to be stored when the file to be stored is stored according to a preset storage format;

And storing, according to the preset storage format, the first file identifier, the data amount, and the to-be-stored file into a value of the target key, and storing the to-be-stored metadata to a preset In the metabase.
A storage medium, characterized in that the storage medium is for storing executable program code, the executable program code being executed to perform the following steps:

Receiving a storage request for a file to be stored;

Obtaining a first file identifier of the file to be stored and a data amount of the file to be stored, and determining a target key for storing the file to be stored;

Obtaining, according to the target key, metadata to be stored when the file to be stored is stored according to a preset storage format;

And storing, according to the preset storage format, the first file identifier, the data amount, and the to-be-stored file into a value of the target key, and storing the to-be-stored metadata to a preset In the metabase.