CN103699585B - Methods, devices and systems for file metadata storage and file recovery - Google Patents

Methods, devices and systems for file metadata storage and file recovery Download PDF

Info

Publication number
CN103699585B
CN103699585B CN201310656195.0A CN201310656195A CN103699585B CN 103699585 B CN103699585 B CN 103699585B CN 201310656195 A CN201310656195 A CN 201310656195A CN 103699585 B CN103699585 B CN 103699585B
Authority
CN
China
Prior art keywords
file
metadata
data
mark
storage device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201310656195.0A
Other languages
Chinese (zh)
Other versions
CN103699585A (en
Inventor
文海
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN201310656195.0A priority Critical patent/CN103699585B/en
Publication of CN103699585A publication Critical patent/CN103699585A/en
Application granted granted Critical
Publication of CN103699585B publication Critical patent/CN103699585B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/16File or folder operations, e.g. details of user interfaces specifically adapted to file systems
    • G06F16/164File meta data generation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/13File access structures, e.g. distributed indices

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiments of the invention provide methods, devices and systems for file metadata storage and file recovery. The method for file metadata storage comprises the steps: acquiring meta datum information respectively corresponding to each file datum according to storage spaces which are respectively allocated to one or more file data of a file, wherein the meta datum information comprises a file identifier of the file; respectively storing the one or more file data in the storage spaces in the form of first binary data; respectively calculating meta datum identifiers respectively corresponding to all the file data according to all the first binary data, wherein the meta datum identifiers are used for identifying the meta datum information; selecting any physical block with an idle storage space from a storage device for each meta datum, and storing the corresponding meta datum in the corresponding physical block, wherein the corresponding meta datum comprises meta datum information and a meta datum identifier for identifying the meta datum information. By adopting the methods, the devices and the systems, provided by the embodiments of the invention, the loss of users can be reduced.

Description

The metadata storage of file and the methods, devices and systems of file access pattern
Technical field
The present invention relates to the communications field, in particular, is the side of the metadata storage and file access pattern for being related to file Method, apparatus and system.
Background technology
File system is a most basic part in computer operating system, is widely used in computer realm for a long time.Text Part system includes metadata information.Assume that user needs to recover an impaired word document, needs are obtained from metadata information Positional information of the word document in storage device is obtained, and the word document correspondence is read from the relevant position of the storage device Binary data, and the binary data stored in storage device is entered according to the word document type in metadata information Row parsing, so as to obtain the file data in word document.
Memory area in storage device of the prior art includes that file data storage region domain and metadata information are concentrated Memory area, wherein, file data storage region domain can store corresponding binary data of above-mentioned word document etc., metadata Information centralized stores region is stored with the metadata information of file system All Files, the metadata information pair of different file types The data structure answered is different, and the metadata information of each different file type constitutes multiple directory branches, if file directory tree In a directory branches in a certain metadata information be damaged, then storage location be located at the metadata information after metadata Information will be lost.
Inventor had found during the invention is realized, as each metadata information in prior art can not be by Recognize respectively, so need each metadata information is stored in metadata information centralized stores region, for the ease of reading Each metadata information in metadata information centralized stores region, needs the association set up between each metadata information.By It is interrelated between each metadata information, so a certain metadata information is damaged, it is possible to cause storage location to be located at Metadata information after the metadata information will be lost.
The content of the invention
In view of this, the invention provides a kind of metadata storage of file and the method for file access pattern, device and being System, to overcome in prior art due to when the interface data of a directory branches in metadata information centralized stores region is damaged Bad when, all metadata informations for constituting this directory branches, will lose, these corresponding files of metadata information lost Data can also be lost, the problem of serious loss so as to cause the user.
For achieving the above object, the present invention provides following technical scheme:
In a first aspect, a kind of metadata storing method of file, including:
Obtained and each described number of files according to the memory space being respectively allocated for one or more file datas in file According to the corresponding metadata information of difference, the metadata information includes the file identification of the file;
Respectively one or more described file datas are stored to the memory space in the form of the first binary data;
First number corresponding with file data difference each described is calculated respectively according to each described first binary data According to mark, the metadata is identified for identifying the metadata information;
For each metadata, arbitrary physical block with idle storage space is selected from the storage device, by institute State metadata to store to the physical block, the metadata includes a metadata information and identifies a metadata information Metadata is identified.
In the first implementation of first aspect, select arbitrary to deposit with the free time from the storage device described The physical block in storage space, the metadata is stored to the physical block, also including the method for reading the file, is read The method of the file includes:
Obtain the file name and file path of the file;
According to the corresponding relation of file name, the file path and metadata mark for pre-setting, the file bag is obtained The All Files data for including distinguish corresponding metadata mark;
For each metadata is identified, the metadata is obtained from the storage device and identifies corresponding first number According to the first binary number of file data corresponding with the metadata is obtained from the storage device according to the metadata According to, the first check value is calculated according to first binary data, when first check value and the metadata mark phase Deng when, parse first binary data, to obtain file data corresponding with first binary data;
The file that acquisition is made up of each the described file data for parsing.
Second aspect, a kind of file access pattern method, the file include one or more file datas, the file data Corresponding metadata includes metadata information and identifies the metadata mark of the metadata information, and the metadata information is Obtained according to the memory space that distributes for the file data, the metadata mark be according to the first binary data and What preset algorithm was calculated, the file data is in the form of first binary data to be stored in the memory space In, the file data restoration methods include:
When receiving the request of recovery file, the metadata with metadata mark is obtained from the storage device;
For each metadata, relevant position in the storage device is read according to the metadata information in the metadata The first binary data, calculate check value according to first binary data and the preset algorithm, when the school When value is tested with the metadata identity equality, determine that the metadata effectively, parses first binary data;
Obtain the file being made up of the file data for parsing the first binary data acquisition.
The third aspect, a kind of metadata storage device of file, including:
First acquisition module, for being obtained according to the memory space being respectively allocated for one or more file datas in file With the corresponding metadata information of file data difference each described, the metadata information includes the file identification of the file;
Memory module, for respectively by one or more described file datas stored in the form of the first binary data to The memory space;
Computing module, it is right respectively with file data each described to be calculated according to each described first binary data respectively The metadata mark answered, the metadata are identified for identifying the metadata information;
First processing module, it is arbitrary with idle storage for for each metadata, selecting from the storage device The physical block in space, the metadata is stored to the physical block, and the metadata includes a metadata information and mark The metadata mark of one metadata information.
In the first implementation of the third aspect, the metadata storage device of the file also includes reading file Device, the device of the reading file include:
Second acquisition module, for obtaining the file name and file path of the file;
3rd acquisition module, for the corresponding pass identified according to the file name, file path for pre-setting and metadata Corresponding metadata mark is distinguished by system, the All Files data that obtaining the file includes;
Second processing module, for identifying for each metadata, obtains first number from the storage device According to corresponding metadata is identified, number of files corresponding with the metadata is obtained from the storage device according to the metadata According to the first binary data, the first check value is calculated according to first binary data, when first check value with During the metadata identity equality, first binary data is parsed, it is corresponding with first binary data to obtain File data;
File module is obtained, for obtaining the file being made up of each the described file data for parsing.
Fourth aspect, a kind of file restoring device, the file include one or more file datas, the file data Corresponding metadata includes metadata information and identifies the metadata mark of the metadata information, and the metadata information is Obtained according to the memory space that distributes for the file data, the metadata mark be according to the first binary data and What preset algorithm was calculated, the file data is in the form of first binary data to be stored in the memory space In, the file data recovery device includes:
First acquisition module, for receiving during the request for recovering file, obtains with first number from the storage device According to the metadata of mark;
First processing module, for for each metadata, reading according to the metadata information in the metadata described First binary data of relevant position in storage device, calculates according to first binary data and the preset algorithm Go out check value, when the check value is with the metadata identity equality, determine the metadata effectively, parsing the described 1st Binary data;
First comprising modules, for obtaining the text being made up of the file data for parsing the first binary data acquisition Part.
Understand via above-mentioned technical scheme, compared with prior art, a kind of unit of file provided in an embodiment of the present invention In date storage method, a file potentially includes one or more file datas, and each file data corresponds to metadata, institute One or more metadata may be corresponded to a file, there is each metadata metadata to identify, as metadata mark is For identification metadata information, it is possible to learn in storage device which data is metadata according to metadata mark, institute So that no matter which position is metadata be stored in, can be identified by metadata it is identified, so need not be by first number It is believed that breath is centrally stored in a certain region.When metadata is stored, the physical block to storing metadata does not have particular/special requirement, only needs Arbitrary physical block with idle storage space, i.e. metadata is selected to be likely stored in the optional position of storage device, Each metadata does not have the restriction of memory area, the association that need not be set up between each metadata, so phase between each metadata Mutually independent, i.e., a certain metadata has been damaged and has had no effect on other metadata.
Description of the drawings
In order to be illustrated more clearly that the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing Accompanying drawing to be used needed for having technology description is briefly described, it should be apparent that, drawings in the following description are only this Inventive embodiment, for those of ordinary skill in the art, on the premise of not paying creative work, can be with basis The accompanying drawing of offer obtains other accompanying drawings.
Fig. 1 is to embodiments provide a kind of schematic flow sheet of the metadata storing method of file;
Fig. 2 is a kind of structural representation of implementation of metadata provided in an embodiment of the present invention;
Fig. 3 be file provided in an embodiment of the present invention metadata storing method in each number of files in metadata and file According to memory space position view;
Fig. 4 be file provided in an embodiment of the present invention metadata storing method in each number of files in metadata and file According to memory space position view;
Fig. 5 be file provided in an embodiment of the present invention metadata storing method in the metadata of the first file data is deposited Store up a kind of method flow schematic diagram of the implementation to the first physical block;
Fig. 6 be file provided in an embodiment of the present invention metadata storing method in metadata memory space and number of files According to memory space position structural representation;
Fig. 7 is that a kind of flow process of another implementation method of the metadata storing method of file provided in an embodiment of the present invention is shown It is intended to;
Fig. 8 is a kind of schematic flow sheet for reading document method provided in an embodiment of the present invention;
Fig. 9 is a kind of method flow schematic diagram of file access pattern method provided in an embodiment of the present invention;
Figure 10 is a kind of structural representation of the metadata storage device of file provided in an embodiment of the present invention;
Figure 11 is a kind of structural representation of device for reading file provided in an embodiment of the present invention;
Figure 12 is a kind of structural representation of file restoring device provided in an embodiment of the present invention;
Figure 13 is a kind of apparatus structure schematic diagram of another embodiment of file restoring device provided in an embodiment of the present invention;
Figure 14 is a kind of structural representation of the metadata storage system of file provided in an embodiment of the present invention.
Specific embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete Site preparation is described, it is clear that described embodiment is only a part of embodiment of the invention, rather than the embodiment of whole.It is based on Embodiment in the present invention, it is every other that those of ordinary skill in the art are obtained under the premise of creative work is not made Embodiment, belongs to the scope of protection of the invention.
Accompanying drawing 1 is referred to, to embodiments provide a kind of schematic flow sheet of the metadata storing method of file, The method includes:
Step S101:Obtained and each institute according to the memory space being respectively allocated for one or more file datas in file State file data and distinguish corresponding metadata information, the metadata information includes the file identification of the file.
Metadata information includes the file identification of the file.Metadata information can also be being deposited including above-mentioned file data The form of positional information and above-mentioned file data in storage equipment.
Have in practical application many application scenarios can for file data distribute memory space, the embodiment of the present invention provide but not It is limited to following application scenarios, scene one:User's input data in a file, the data are first stored in a temporary file or interior In depositing, after user clicks on save button, can generate the instruction of the data storage to storage device, and be to be somebody's turn to do in storage device One memory space of data distribution.Scene two:User arranges the Automatic Save Every of storage file(Automatic Save Every Can be 5 seconds, or other, the embodiment of the present invention is not especially limited to this), user is in Automatic Save Every Between in section the data of input first can be stored in a temporary file or internal memory, when Automatic Save Every arrives, generation The instruction of the data is stored, and is one memory space of data distribution in storage device.The text referred in the embodiment of the present invention Number of packages evidence can be the data stored into temporary file or internal memory, or store the data into temporary file or internal memory A part, the above-mentioned data stored into temporary file or internal memory can also be above-mentioned file data a part.
It is understood that word document, pdf document or txt file can be two file of above-mentioned scene one and scene, this The file data that inventive embodiments are proposed can correspond to the part number in all data, or file in a file According to.
Above-mentioned file, is exactly in the electronic device, for the purpose of the partial function for realizing certain function or certain software One unit of definition.File in electronic equipment can be document, program, shortcut and equipment.File is by filename Constitute with icon, a type of file has identical icon, and filename is no more than 255 characters(Including space).Text Part can also refer to the set of the data being stored on external agency.
Above-mentioned file data may correspond to a metadata, it is also possible to the multiple metadata of correspondence.Assume above-mentioned file data Including N number of metadata, i.e. the first metadata to N metadata, N is the positive integer more than or equal to 1, and corresponding file data includes First sub-file data is to N sub-file datas, wherein the i-th sub-file data the i-th sub- metadata of correspondence, the i-th sub- metadata letter Can include in breath:Store the start offset address of the physical block of the i-th sub-file data, the byte length of the i-th sub-file data, I-th sub-file data is located at the positional information of above-mentioned file data, the modification time of the i-th sub-file data, the i-th sub-file data The file format of the byte length of place physical block and the i-th sub-file data.Above-mentioned i is more than or equal to 1, less than or equal to N just Integer.
The physical address and start offset address, the i-th Ziwen number of packages of the physical block of the i-th sub-file data of storage can be utilized According to byte length, the corresponding binary data of the i-th sub-file data is read out from corresponding physical block, because file data Stored with binary form in memory space.
I-th sub-file data is located at the positional information of above-mentioned file data and may refer to the i-th sub-file data in above-mentioned text Which section of which section or which page of number of packages evidence etc..Because file data may have multiple paragraghs, it needs to be determined that Which section of which sub-file data in above-mentioned file data in storage device gone out.
User may carry out multiple modification to the i-th sub-file data, it is possible that have multiple i-th sub-file datas, but Be in general, it is nearest apart from current time, it should to be that user just changes recently, when user opens a certain file When, the sub-file data for constituting this document should be included apart from the i-th nearest sub-file data of current time, and can not include I-th sub-file data of the modification time for other times.
The file format of the i-th sub-file data can be the suffix of the title of above-mentioned file data, or above-mentioned filename Last 8 bytes for claiming.Last 8 bytes of file name have usually contained the suffix name of file data, can according to suffix name To judge the type of file data.For example " .doc " suffix name represents word document, and " .rar " suffix name represents rar compressed formats File.File system generally supports that long filenames, common file system support the filename length of 256 bytes, if Retain the space of 256 bytes inside metadata preserving filename, then memory space can be caused than larger in the space for consuming Waste, in order to avoid the waste of memory space, can be with the last a few bytes of document retaining title, it is preferred that can be text Last 8 bytes of part title are remained, for differentiating file type.The file name of such as above-mentioned file data is " Shen Please file 12345678.doc ", and the types of variables of file name is character type, then last 8 bytes of file name are referred to “5678.doc”.Corresponding first binary number of the i-th sub-file data of File Format Analysis of the i-th sub-file data can be utilized According to, and corresponding application program is called, to be shown to user.
I-th sub- metadata information can also include retaining space, naturally it is also possible to not include.
The above-mentioned explanation to the i-th sub- metadata information is also applied for any piece of metadata information that the embodiment of the present invention is referred to.
Step S102:Respectively one or more described file datas are stored to described in the form of the first binary data Memory space.
File data is to be stored in memory space in the form of binary data, in order to follow-up binary data Make a distinction, storage form of the corresponding file data in memory space in above-mentioned file is referred to as into the first binary number here According to.
Step S103:It is right respectively with file data each described to be calculated according to each described first binary data respectively The metadata mark answered.
The metadata is identified for identifying the metadata information.
Metadata mark can be calculated according to the first binary data, such as by the first binary system by preset algorithm The cyclic redundancy check value of data identifies as the metadata in the embodiment of the present invention, the first binary data is carried out MD5 (Message Digest Algorithm, Message Digest 5)The value for obtaining afterwards, identifies as metadata, or the one or two is entered Data processed carry out SHA(Secure Hash Algorithm, SHA)After calculating, the value for obtaining is used as metadata mark Know, the embodiment of the present invention is not especially limited to the computational methods of the corresponding metadata mark of the first binary data.
Fig. 2 is referred to, is a kind of structural representation of implementation of metadata provided in an embodiment of the present invention.
As can be seen from Figure 2 there can be 10 fields in each metadata, altogether 64 bytes, it is to be understood that In metadata can no retaining space the two fields, with only one of which retaining space field, or can have in metadata Two or more retaining space fields, the embodiment of the present invention are not especially limited to this, so metadata can also be 48 Byte, naturally it is also possible to which the byte length of each field is set to other numerical value, for example, be set to 5 words by retaining space Metadata mark is set to 6 bytes, so the embodiment of the present invention is not especially limited to this by section.To sum up, the word of metadata Section length can be more than less than or equal to 64 bytes.Metadata in the embodiment of the present invention can include other fields, and differ Surely it is above-mentioned 10 fields, for example, can also includes file name field, the embodiment of the present invention is to the field number in metadata And species is not especially limited.
From figure 2 it can be seen that each field all corresponds to a types of variables, the embodiment of the present invention is provided but is not limited to Fig. 2 Shown in types of variables, the types of variables of such as file identification can also be unsigned char.
Storage device can be hard disk, CD, USB flash disk, tape etc..
The embodiment of the present invention each metadata information of metadata mark for marking, it is possible to the foundation from storage device Metadata mark obtains each metadata information.
Step S104:For each metadata, arbitrary thing with idle storage space is selected from the storage device Reason block, the metadata is stored to the physical block.
The metadata includes a metadata information and identifies the metadata mark of a metadata information.
Each metadata information and identify the metadata information metadata mark need to be stored in Same Physical block In, but different metadata can be stored in different physical blocks, and when each metadata is stored, physical block can be It is randomly selected, so being different from metadata information is centrally stored in metadata information centralized stores region in prior art , each metadata in the embodiment of the present invention can not be centrally stored in memory area in metadata set, i.e., the present invention is implemented Can no metadata information centralized stores region in storage device in example.
For identical file, one or more metadata can be corresponded to, can be to be stored separately between each metadata , and it is separate, for different files, can also be stored separately between the corresponding metadata of different files , and it is separate.
The embodiment of the present invention does not have particular/special requirement to the physical block for storing metadata, so the first number in the embodiment of the present invention According to may be located in different physical blocks, i.e., metadata may be located at different regions, different from prior art by file Metadata is stored in memory block in the metadata set in storage device.
In the metadata storing method of file provided in an embodiment of the present invention, there is metadata metadata to identify, due to unit Data Identification is for identification metadata, it is possible to learn which data is first number in storage device according to metadata mark According to, and when metadata is stored, the physical block to storing metadata does not have particular/special requirement, it is only necessary to which physical block was deposited with the free time Storage space, so metadata is likely stored in the optional position of storage device, that is, each file data is corresponded to respectively Storage location of the metadata in storage device can be arbitrary, with each file data in prior art metadata set In to be stored in metadata information centralized stores region be different and not existing between each metadata in the embodiment of the present invention Interface function in technology, thus each file data in the embodiment of the present invention corresponding metadata is separate respectively , if a certain metadata is damaged, the corresponding file data of the impaired metadata can be lost, and other metadata are simultaneously Will not lose, and the corresponding file data of other metadata will not be lost, so as to reduce the loss of user.
Further, in prior art, the metadata of All Files data is all centrally stored in metadata information centralized stores Region, when the metadata information centralized stores region is damaged, all metadata informations will be lost, and cause the user is tight The loss of weight, in the embodiment of the present invention, each metadata can be not stored in the region of a concentration, so if because depositing During a certain partial destruction of storage equipment, also it is that metadata in the part is damaged, has no effect on storage device other positions The effectiveness of metadata.
Further, usually need to arrange multiple metadata informations with identical content in prior art, so when which In metadata information when being damaged, can be to reply to file data according to other metadata informations, but this is more Synchronization is needed between individual metadata information, such as after user changes file data, deleting file data or increases file data, Need renewal to be synchronized to the metadata information of corresponding file data in each metadata information, multiple metadata are carried out Synchronized update can cause file system performance to reduce, and metadata information need not be synchronized in the embodiment of the present invention, from And improve the performance of file system.
It is understood that file data has many with storage mode of the metadata in storage device in actual applications Kind, the embodiment of the present invention is provided but is not limited to following several storage modes.
First, file data and metadata are stored in Same Physical block.
In file, each file data can be stored in Same Physical block with metadata, and file can include the first file Data, metadata corresponding with the first file data is the first metadata accordingly, and the storage for being set to the distribution of the first file data is empty Between be the first physical block, the byte length of the first file data is not more than the first byte number and deducts difference obtained by the second byte number, First byte number refers to the byte number of the first physical block idle storage space before the first file data is stored, the second byte number The byte length of the first metadata, in the metadata storing method of file provided in an embodiment of the present invention by above-mentioned file Metadata is stored to above-mentioned physical block:The metadata of the first file data is stored to the first physical block.
The byte length of the first file data can be equal to the first byte number and deduct difference obtained by the second byte number, refer to Fig. 3, be file provided in an embodiment of the present invention metadata storing method in metadata deposit with each file data in file The position view in storage space.
As can be seen from Figure 3, file can include the first file data, the second file data and the 3rd file data.With first It is the second metadata and that the corresponding metadata of file data is the first metadata metadata corresponding with the second file data The corresponding metadata of three file datas is trinary data, store the first file data for the first physical block, storage second is literary Number of packages evidence for the second physical block, store the 3rd file data for the 3rd physical block.As can be seen from Figure 3 it is metadata is attached It is added to behind corresponding physical block.
In Fig. 3, also enumerate 6 examples, the size of physical block be respectively 4096 bytes, 8192 bytes, 16384 bytes, 32768 bytes, 65536 bytes and X bytes, then file data be stored in the byte length in each physical block can respectively For:4032 bytes, 8128 bytes, 16320 bytes, 32704 bytes, 65472 bytes and X-64 bytes, and in each physical block Metadata be 64 bytes(By taking the metadata structure in Fig. 2 as an example).
The storage mode of file data and metadata shown in Fig. 3, after a certain metadata is obtained from storage device, by Be stored with a certain metadata the byte length information of corresponding file data(Hypothesis byte length is 1024 words Section), it is possible to front 1024 bytes of a certain metadata are known for the corresponding file data of a certain metadata, Ke Yizhi The corresponding file data of the reading a certain metadata is connect, is eliminated and text corresponding with a certain metadata is searched from storage device The time of number of packages evidence, so as to improve the speed for reading and recovering file data.
The byte length of file may be very big in actual applications, if at this moment file only corresponds to a metadata, when When the metadata is damaged, whole file cannot just recover, it is possible to arrange the density of metadata, i.e., often by preset length File data when storing to storage device, be generated as a metadata corresponding with the file data of the preset length, so i.e. Be damaged one of them or several metadata, it is also possible to partial document to be recovered according to other unspoiled metadata, use The loss at family is reduced, and the byte length of the first file data can be preset length, refer to Fig. 4, be that the embodiment of the present invention is carried For file metadata storing method in metadata and file the memory space of each file data position view.
As can be seen from Figure 4, file data can include the first file data, the second file data and the 3rd file data.With The corresponding metadata of first file data be the first metadata metadata corresponding with the second file data be the second metadata, Metadata corresponding with the 3rd file data is trinary data.
Above-mentioned file data can be stored in multiple physical blocks, it is also possible to be stored in a physical block, such as Fig. 4 institutes Show, the start offset address of the first metadata is the start offset address of the first file data, the byte long of the first file data Degree and 1 sum, i.e. the first metadata are close to the storage location of the first file data.Second metadata and trinary data are same Reason.
Refer to Fig. 5, be file provided in an embodiment of the present invention metadata storing method in by the first file data Metadata stores a kind of method flow schematic diagram of the implementation to the first physical block, and the method includes:
Step S501:Obtain the first file data to store to the first start offset address of the first physical block.
Step S502:By the first start offset address, the byte length of the first file data and 1 sum, it is defined as first The metadata of file data is stored to the second start offset address of the first physical block.
Step S503:With the second start offset address as start offset address, the first metadata is stored.
2nd, each file data in file is stored in different physical blocks from metadata.
In actual applications, the physical block of the physical block of storage file data and storage metadata can be distinguished, can To be understood by, it is assumed that after physical block stores the first file data, remaining idle bytes are if less than the first metadata Byte number, the first metadata can be stored in another physical block.
The address realm of the physical block of storage metadata can be the first preset address scope, then the embodiment of the present invention is provided File metadata storing method in select from storage device with the physical block of idle storage space be:From storage device In belong to the physical block of the first preset address scope in select with idle storage space physical block.Fig. 6 is referred to, is the present invention The position of the memory space of the memory space and file data of metadata in the metadata storing method of the file that embodiment is provided Structural representation.
Here the first preset address scope not implied that and divided memory area in a metadata set for metadata, and this first Address in preset address scope can be the discontinuous address of interruption, naturally it is also possible to for continuation address.
As can be seen from Figure 6 file data can include:First file data, the second file data and the 3rd number of files According to store the first file data can be the first physical block, and store the second file data can be the second physical block, store 3rd file data can be the 3rd physical block, store the physical block of the first metadata, the second metadata and trinary data It can be the first physical block 4.
In Fig. 6, it is the first preset address scope due to storing the range of physical addresses of physical block of metadata, so looking into When looking for metadata, it is not necessary to travel through whole storage device, it is only necessary to travel through the metadata of the first preset address scope, so as to improve Search the speed of metadata.
3rd, may include in storage device that file data and the metadata of file are stored in the storage side in Same Physical block Formula, it is also possible to be stored in the storage mode in different physical blocks from the metadata of file including file data.
In actual applications, it is assumed that after certain physical block stores the first file data, if remaining idle bytes are little In the byte number of the first metadata, the first metadata can be stored in another physical block, if to store first literary for physical block According to afterwards, remaining idle bytes are more than or equal to the byte number of the first metadata to number of packages, then the first metadata can be stored in In the physical block, so the storage mode that can have both had in same storage device in Fig. 6, it is possibility to have the storage side in Fig. 3 Formula.
In prior art, as the corresponding metadata information of All Files data is stored in same metadata centralized stores Region, so when memory area is damaged in metadata set, all of file data will be lost, will bring huge to user Big loss, on the basis of the metadata information that the embodiment of the present invention can be in the prior art, increases the metadata information Metadata, due to only storing data hereof, can just generate metadata when storing to storage device, so needing Metadata of the prior art is write in a file, Fig. 7 is referred to, is a kind of unit of file provided in an embodiment of the present invention The schematic flow sheet of another implementation method of date storage method, the method include:
Step S701:Corresponding metadata information is stored to first the All Files data that the file is included respectively In file.
First file is a kind of file of the special format for storing the corresponding metadata information of All Files data.
Can be stored with first file the metadata information of multiple files.
Step S702:According to the storage device for one or more the first file data distribution in first file In memory space, obtain and the corresponding file metadata information of the first file data difference each described.
The file metadata information includes the file identification of first file.
The description to the i-th sub- metadata is can be found in the description of file metadata information, is no longer repeated one by one here.
First file can correspond to one or more file metadatas.
Step S703:Respectively by one or more described first file datas stored in the form of the second binary data to The memory space.
First file data is to be stored in memory space in the form of binary data, in order to above-mentioned file data Corresponding first binary data makes a distinction, and the storage form by the first file data in memory space is referred to as second here Binary data.
Step S704:According to the second binary system that each described first file data is stored in the storage device respectively Data, calculate the file unit of mark corresponding with the first file data difference each the described file metadata information respectively Data Identification.
Step S705:For each file metadata, select arbitrary with idle storage space from the storage device Physical block, the file metadata is stored into the physical block.
The file metadata includes a file metadata information and the text with mark one file metadata information Part metadata is identified.
The File metadata of each File metadata information and mark this document file metadata information Mark needs to be stored in Same Physical block, but different file metadatas can be stored in different physical blocks, and When storing each file metadata, physical block can be it is randomly selected, so with prior art in by File metadata It is different that information is centrally stored in File metadata information centralized stores region, each file in the embodiment of the present invention Metadata can not be centrally stored in file metadata centralized stores region, i.e. storage device in the embodiment of the present invention can be with Without File metadata information centralized stores region.
For same first file, one or more file metadatas can be corresponded to, between each file metadata be It is stored separately, and is separate, for the first different files, the corresponding file metadata of different first files Between can also be stored separately, it is and separate.
In the embodiment of the present invention, there is file metadata file metadata to identify, due to file metadata mark be for Mark file metadata, it is possible to learn which data is file unit number in storage device according to file metadata mark According to, and in storage file metadata, there is no particular/special requirement to the physical block of storage file metadata, it is only necessary to physical block has Available free memory space, so file metadata is likely stored in the optional position of storage device, that is, the first file Storage location of the corresponding file metadata in storage device can be arbitrary, when in metadata set, memory area is damaged When, the corresponding file metadata of the first file may not be damaged, it is possible to recover the first file according to file metadata, so Afterwards again according to the first file access pattern file data.
Above-mentioned steps S701 can be combined with embodiment described in step S101 to step S104 to step S705, step After S701 may be located at step S101 to step S705, the data in such storage device just have double shield, work as file When data are damaged, first file can be recovered according to file metadata, then according to the first file access pattern file data, File data can be recovered according to the metadata in above-described embodiment with respective meta-data mark.
It is understood that the metadata mark in the metadata storing method embodiment of any of the above-described file can be From default metadata mark scope unassigned mark is selected to identify as metadata, or from the text of default file data The corresponding relation of part title, the file path of file data and metadata mark, obtains the metadata mark of above-mentioned file data Know.
Refer to Fig. 8, be it is provided in an embodiment of the present invention it is a kind of read document method schematic flow sheet, the reading file The storage method of each metadata in method is the metadata storing method of above-mentioned file, and the reading document method includes:
Step S801:Obtain the file name and file path of the file.
Step S802:According to the corresponding relation of file name, the file path and metadata mark for pre-setting, institute is obtained The All Files data that stating file includes distinguish corresponding metadata mark.
Step S803:For each metadata is identified, the metadata mark is obtained from the storage device right The metadata answered, obtains the first of file data corresponding with the metadata from the storage device according to the metadata Binary data, calculates the first check value according to first binary data, when first check value and first number During according to identity equality, first binary data is parsed, to obtain file data corresponding with first binary data.
Step S804:The file that acquisition is made up of each the described file data for parsing.
In reading document method provided in an embodiment of the present invention, due to pre-setting file name, file path and unit The corresponding relation of Data Identification, so when needing to open a certain file, can directly obtain according to above-mentioned corresponding relation and wait out The corresponding metadata mark of Octride part, when the metadata with the metadata mark is searched from storage device, it is only necessary to right Identify than the metadata of each metadata, the particular content of each metadata need not be read, and utilized in prior art During metadata information file opening in metadata information centralized stores region, need to travel through in metadata set in memory area Each metadata information, till finding metadata information corresponding with file to be opened, so the embodiment of the present invention is carried For file data read method improve from storage device the speed for obtaining the corresponding metadata of above-mentioned file, so as to improve The speed that file reads.
In above-mentioned reading document method embodiment, metadata information can include the file identification of above-mentioned file, according to pre- The corresponding relation of the file name, file path and metadata mark that first arrange, obtains the corresponding metadata mark of above-mentioned file Implementation method have various, the embodiment of the present invention provide but be not limited to following methods:
According to the corresponding relation of the file name, file path and file identification for pre-setting, the text of the file is obtained Part is identified;According to the file identification for pre-setting and the corresponding relation of metadata mark, the corresponding metadata of above-mentioned file is obtained Mark.
Fig. 9 is referred to, is a kind of method flow schematic diagram of file access pattern method provided in an embodiment of the present invention, file bag Include one or more file datas, the corresponding metadata of the file data includes metadata information and identifies the metadata The metadata mark of information, the metadata information are obtained according to the memory space for file data distribution, described Metadata mark is calculated according to the first binary data and preset algorithm, and the file data is with described first The form of binary data is stored in the memory space, the storage of the corresponding metadata of the file in the embodiment of the present invention Method is consistent with the metadata storing method of file described in above-described embodiment, and this document data reconstruction method includes:
Step S901:When receiving the request of recovery file, obtain from the storage device with metadata mark Metadata.
File to be restored may correspond to one or more metadata, i.e., file to be restored may correspond to one or more Metadata is identified.
Step S902:For each metadata, the storage device is read according to the metadata information in the metadata First binary data of middle relevant position, calculates verification according to first binary data and the preset algorithm Value, when the check value is with the metadata identity equality, determines that the metadata effectively, parses first binary number According to.
It is understood that some or multiple metadata are likely to be broken, it is now extensive according to the metadata after damage The data appeared again are probably a pile mess code, so before parsing metadata the first binary data of correspondence, can first detect unit Whether data are damaged, that is, detect the effectiveness of metadata.
Check value can be calculated according to the first binary data, such as by the first binary data by preset algorithm Cyclic redundancy check value carry out MD5 as the test value in the embodiment of the present invention, by the first binary data(Message Digest Algorithm, Message Digest 5)The value of acquisition is used as the check value in the embodiment of the present invention, or the one or two is entered Data processed carry out SHA(Secure Hash Algorithm, SHA), the value of acquisition is used as in the embodiment of the present invention Check value, the embodiment of the present invention is not especially limited to the computational methods of the corresponding check value of the first binary data.
The preset algorithm of Computing Meta Data Identification is identical with the algorithm for calculating check value, so just can compare both Compared with.
Assume the N number of metadata of above-mentioned file correspondence, respectively the first metadata is entered to N metadata, the corresponding 1st Data processed include:First metadata the first binary data 1 of correspondence and N metadata the first binary data N of correspondence, N are Natural number more than or equal to 1.At this moment need to parse the first binary data 1 and the first binary data N respectively, obtain 1 corresponding first data of the first binary data after must parsing, and the corresponding Nth datas of the first binary data N.
Step S903:Obtain the file data that the data obtained by the first binary data of parsing are constituted.
Above-mentioned first data and Nth data can be with composing document data.In the embodiment of the present invention, if the first metadata quilt Damage, then only the first data correctly can not be parsed, other data having no effect in presents data.To sum up, if M metadata can be read, it is possible to recover M data, M is the natural number more than or equal to 1 less than or equal to N.
All there is metadata to identify for file access pattern method provided in an embodiment of the present invention, each metadata, and metadata mark Knowledge is, for identification metadata, can to learn according to metadata mark which data in memory space are metadata, this No interface function of the prior art in bright embodiment, even and if a certain metadata be damaged, do not interfere with other yuan of number yet According to that is, each metadata is separate, i.e., several metadata are not damaged, and can just recover these metadata corresponding Data, so as to reduce the loss of user.
In any of the above-described file access pattern embodiment of the method, the side of the corresponding metadata of above-mentioned file is obtained from storage device Method can have various, obtain with above-mentioned metadata mark in file access pattern method provided in an embodiment of the present invention from storage device A kind of implementation method of the metadata of knowledge includes:Obtain the file name and file path of above-mentioned file.According to pre-setting File name, file path and metadata mark corresponding relation, obtain corresponding with above-mentioned file data metadata mark. The metadata with above-mentioned file identification is obtained from storage device.
Metadata information can include file identification, file name, file path and first number that above-mentioned basis pre-sets According to the corresponding relation of mark, obtaining metadata mark corresponding with above-mentioned file data can include:According to the text for pre-setting The corresponding relation of part title, file path and file identification, obtains the file identification of above-mentioned file data, according to what is pre-set File identification and the corresponding relation of metadata mark, obtain the corresponding metadata mark of above-mentioned file.
Metadata information can include file identification, from storage device in file access pattern method provided in an embodiment of the present invention The middle another kind of implementation method for obtaining the metadata with above-mentioned metadata mark includes:Obtain the file name of above-mentioned file with And file path.According to the corresponding relation of the file name, file path and file identification for pre-setting, obtain and above-mentioned file The file identification of data.The metadata with above-mentioned file identification is obtained from storage device.
It is understood that metadata can be stored in Same Physical block with file, it is also possible to be stored separately with file, Assume storage metadata physical block address scope be the first preset address scope, file access pattern side provided in an embodiment of the present invention A kind of implementation method of metadata with above-mentioned metadata mark is obtained from storage device in method includes:In scanning storage device Range of physical addresses belongs to the physical block of the first preset address scope.Belong to the first preset address scope from range of physical addresses The metadata with above-mentioned metadata mark is obtained in physical block.
The embodiment of the present invention additionally provides a kind of flowage structure schematic diagram of file data restoration methods, and this document data are extensive Storage device in compound recipe method is also stored with file metadata, and the file metadata includes file metadata information and mark The file metadata mark of the file metadata information, the file metadata is the corresponding metadata of the first file, described First file is stored with the corresponding all metadata of the file, and the method includes:Obtain from the storage device with institute State the file metadata of file metadata mark;For each file metadata, according to the file unit in the file metadata Data message reads the second binary data of relevant position in the storage device, is calculated according to second binary data Go out the second check value, when second check value is with the file metadata identity equality, parse second binary number According to obtain the first file data corresponding with second binary data;What acquisition was made up of first file data First file;The file according to first file access pattern.
Method is described in detail in the invention described above disclosed embodiment, for the method for the present invention can take various forms Device realize that therefore the invention also discloses various devices, are given below specific embodiment and are described in detail.
Figure 10 is referred to, is a kind of structural representation of the metadata storage device of file provided in an embodiment of the present invention, The metadata storage device of this document includes:First acquisition module 1001, the first memory module 1002, computing module 1003 and First processing module 1004, wherein:
First acquisition module 1001, for according to the memory space being respectively allocated for one or more file datas in file Obtain and the corresponding metadata information of file data difference each described.
The metadata information includes the file identification of the file.
Description to the first acquisition module 1001 can be found in the description to step S101, no longer be repeated one by one herein.
First memory module 1002, for respectively by one or more described file datas with the shape of the first binary data Formula is stored to the memory space.
Description to the first memory module 1002 can be found in the description to step S102, no longer be repeated one by one herein.
Computing module 1003, for being calculated respectively and number of files each described according to each described first binary data According to the corresponding metadata mark of difference, the metadata is identified for identifying the metadata information.
First processing module 1004, for for each metadata, selecting arbitrary with the free time from the storage device The physical block of memory space, the metadata is stored to the physical block, the metadata include a metadata information and Identify the metadata mark of a metadata information.
In the metadata storage device of file provided in an embodiment of the present invention, a file potentially includes one or more texts Number of packages evidence, each file data correspond to metadata, so a file may correspond to one or more metadata, each first number According to identifying with metadata, as metadata mark is for identification metadata information, it is possible to identify according to metadata In learning storage device, which data is metadata, so no matter which position is metadata be stored in, can be by first number It is identified according to identifying, so metadata information need not be centrally stored in a certain region.First processing module 1004 exists During storage metadata, the physical block to storing metadata does not have particular/special requirement, it is only necessary to select arbitrary with idle storage space Physical block, i.e. metadata is likely stored in the optional position of storage device, and each metadata does not have the limit of memory area System, the association that need not be set up between each metadata, so separate between each metadata, i.e., a certain metadata is damaged Have no effect on other metadata.
It is understood that each file data and storage of the metadata in storage device in file in actual applications Mode has various, and the embodiment of the present invention is provided but is not limited to following several storage modes.
File data is stored in Same Physical block with the metadata of file.Above-mentioned file can include the first number of files According to metadata corresponding with the first file data is the first metadata, is that the memory space of the first file data distribution is the first thing Reason block, byte length no more than first byte number of the first file data deduct the difference obtained by the second byte number, above-mentioned first word Joint number refers to the byte number of the first physical block idle storage space before the first file data is stored, and above-mentioned second byte number is The byte length of above-mentioned first metadata, above-mentioned first memory module include:
Memory element, for the metadata of above-mentioned first file data is stored to the first physical block.
Metadata of the file data with file is stored in Same Physical block, then when a certain metadata is detected, The physical block address of the corresponding file data of a certain metadata is obtained just, it is possible to save locating file data corresponding The time of physical block address, so as to improve the efficiency for reading and recovering file data.The embodiment of the present invention additionally provides one kind A kind of implementation of the memory element in the metadata storage device of file, the memory element include:Obtain subelement, determine Subelement and storing sub-units, wherein:Subelement is obtained, is stored to first physical block for obtaining the first file data The first start offset address.Determination subelement, for by the word of first start offset address, first file data Section length and 1 sum, the metadata for being defined as first file data store inclined to the second starting of first physical block Move address.Storing sub-units, for second start offset address as start offset address, storing first yuan of number According to.
In the embodiment of the present invention, as storing sub-units are when the first metadata is stored, determined with determination subelement First start offset address, the byte length of the first file data and 1 sum are start offset address, so recovering or reading It is when taking this document data, after the first metadata is obtained from storage device, right with which due to being stored with the first metadata The byte length information of the file data answered(Hypothesis byte length is 1024 bytes), it is possible to know the first metadata Front 1024 bytes are corresponding first file data of first metadata, can directly read the first file data, eliminate The time of the first file data is searched from storage device, so as to improve the speed for reading and recovering file data.
The address realm for storing the physical block of above-mentioned metadata can be for the first preset address scope, the first memory module tool Body is used for:Select with idle storage space physical block in belonging to the physical block of the first preset address scope from storage device. As the range of physical addresses of the physical block for storing metadata is the first preset address scope, so when metadata is searched, no Need to travel through whole storage device, it is only necessary to travel through the metadata of the first preset address scope, so as to improve lookup metadata Speed.
The embodiment of the present invention additionally provides a kind of another embodiment of the metadata storage device of file, and the device includes: First memory module, the 4th acquisition module, the second memory module, the first computing module and the 3rd processing module, wherein:First Memory module, for the All Files data that include the file, corresponding metadata information is stored to the first file respectively In.4th acquisition module, for according to the storage for one or more the first file data distribution in first file Memory space in equipment, obtains and the corresponding file metadata information of the first file data difference each described.The file Metadata information includes the file identification of first file.Second memory module, for respectively by one or more described One file data is stored to the memory space in the form of the second binary data.First computing module, for according to each The second binary data that first file data is stored in the storage device respectively, calculate respectively with described in each The file metadata mark of the first file data difference corresponding mark file metadata information.3rd processing module, uses In for each file metadata, arbitrary physical block with idle storage space is selected from the storage device, will be described File metadata is stored into the physical block, and the file metadata includes a file metadata information and described with mark The file metadata mark of one file metadata information.
In the embodiment of the present invention when the first file is damaged, the first file can be carried out according to file metadata extensive It is multiple, according to recovery after the first file be obtained with a file data so that the data in storage device are safer.
The embodiment of the present invention can be combined with Figure 10 shown device embodiments, and the data in such storage device just have Double shield, when file is damaged, can recover first file according to file metadata, then according to the first file access pattern File data, it is also possible to which the metadata according to having respective meta-data mark in above-described embodiment recovers file data.
It is understood that the dress of the acquisition metadata mark in the metadata storage device embodiment of any of the above-described file Putting to include:Mark unit is selected, for unassigned mark being selected as above-mentioned from default metadata mark scope Metadata is identified;Or determine mark unit, for the file name from default above-mentioned file data, the file of above-mentioned file data Path and the corresponding relation of the metadata mark of above-mentioned file data, determine the metadata mark of above-mentioned file data.
Refer to Figure 11, be it is provided in an embodiment of the present invention it is a kind of read file device structural representation, the reading The storage method of the metadata in the device of file is consistent with the metadata storing method of above-mentioned file, and above-mentioned file data reads Device includes:Second acquisition module 1101, the 3rd acquisition module 1102, Second processing module 1103 and acquisition file module 1104, wherein:
Second acquisition module 1101, for obtaining the file name and file path of the file.
3rd acquisition module 1102, for according to the right of the file name, file path and metadata mark for pre-setting Should be related to, the All Files data that obtaining the file includes distinguish corresponding metadata mark.
Second processing module 1103, for identifying for each metadata, obtains described from the storage device Metadata identifies corresponding metadata, obtains text corresponding with the metadata according to the metadata from the storage device First binary data of number of packages evidence, calculates the first check value according to first binary data, when the described first verification When value is with the metadata identity equality, first binary data is parsed, to obtain and first binary data pair The file data answered.
File module 1104 is obtained, for obtaining the file being made up of each the described file data for parsing.
In file data reading device provided in an embodiment of the present invention, as the 3rd acquisition module 1102 pre-sets text The corresponding relation of part title, file path and metadata mark, so when needing to open a certain file, the 3rd acquisition module 1102 can directly obtain the corresponding metadata mark of file to be opened according to above-mentioned corresponding relation, in Second processing module 1103 When the metadata with the metadata mark is searched from storage device, it is only necessary to which the metadata for contrasting each metadata is identified i.e. Can, the particular content of each metadata need not be read, and in prior art in using metadata information centralized stores region During metadata information file opening, need to travel through each metadata information in memory area in metadata set, until find with Till the corresponding metadata information of file to be opened, so file data read method provided in an embodiment of the present invention improves The speed of the corresponding metadata of file data to be opened is obtained from storage device, so as to improve the speed of file data reading.
It is understood that the device of above-mentioned reading file is applied in example, metadata information can include above-mentioned file data File identification, the structure of the 3rd acquisition module has various, and the embodiment of the present invention provides but be not limited to following construction, and the 3rd obtains Module can include:First obtains unit, for corresponding with file identification according to the file name, file path for pre-setting Relation, obtains the file identification of the file data;Second obtaining unit, for according to the file identification for pre-setting and first number According to the corresponding relation of mark, the corresponding metadata mark of the file data is obtained.
Figure 12 is referred to, is a kind of structural representation of file restoring device provided in an embodiment of the present invention, the file Including one or more file datas, the corresponding metadata of the file data includes metadata information and identifies first number It is believed that the metadata mark of breath, the metadata information is obtained according to the memory space for file data distribution, institute State metadata mark to be calculated according to the first binary data and preset algorithm, the file data is with described the The form of binary evidence is stored in the memory space, metadata in the file restoring device in the embodiment of the present invention Storage method it is consistent with the metadata storing method of above-mentioned file, above-mentioned file data recovery device includes:First obtains mould Block 1201, first processing module 1202 and the first comprising modules 1203, wherein:
First acquisition module 1201, for receiving during the request for recovering file, is had from the storage device The metadata of metadata mark.
File to be restored may correspond to one or more metadata, i.e., file to be restored may correspond to one or more Metadata is identified.
First processing module 1202, for for each metadata, reading according to the metadata information in the metadata First binary data of relevant position in the storage device, according to first binary data and the preset algorithm Check value is calculated, when the check value is with the metadata identity equality, the metadata is determined effectively, described the is parsed Binary evidence.
It is understood that some or multiple metadata are likely to be broken, it is now extensive according to the metadata after damage The data appeared again are probably a pile mess code, so before parsing metadata the first binary data of correspondence, can first detect unit Whether data are damaged, that is, detect the effectiveness of metadata.
Check value can be calculated according to the first binary data, such as by the first binary data by preset algorithm Cyclic redundancy check value carry out MD5 as the test value in the embodiment of the present invention, by the first binary data(Message Digest Algorithm, Message Digest 5)The value of acquisition is used as the check value in the embodiment of the present invention, or the one or two is entered Data processed carry out SHA(Secure Hash Algorithm, SHA), the value of acquisition is used as in the embodiment of the present invention Check value, the embodiment of the present invention is not especially limited to the computational methods of the corresponding check value of the first binary data.
Assume the N number of metadata of above-mentioned file correspondence, respectively the first metadata is entered to N metadata, the corresponding 1st Data processed include:First metadata the first binary data 1 of correspondence and N metadata the first binary data N of correspondence, N are Natural number more than or equal to 1.At this moment need to parse the first binary data 1 and the first binary data N respectively, obtain 1 corresponding first data of the first binary data after must parsing, and the corresponding Nth datas of the first binary data N.
First comprising modules 1203, for obtaining what is be made up of the file data for parsing the first binary data acquisition File.
Above-mentioned first data and Nth data can be with composing document data.If the first metadata of the embodiment of the present invention is damaged Bad, only the first data correctly can not be parsed, other data having no effect in presents data.To sum up, if can be with Read M metadata, it is possible to recover M data, M is the natural number more than or equal to 1 less than or equal to N.
All there is metadata to identify for file data recovery device provided in an embodiment of the present invention, each metadata, and first number It is for identification metadata, so the first acquisition module 1201 can be learnt in memory space according to metadata mark according to mark Which data be metadata, no interface function of the prior art in the embodiment of the present invention, even and if a certain metadata quilt Damage, also do not interfere with other metadata, i.e., each metadata is separate, i.e., several metadata are not damaged, with regard to energy The corresponding data of these metadata are recovered, so as to reduce the loss of user.
In a kind of file data recovery device provided in an embodiment of the present invention, the first acquisition module can include:First obtains Unit, the second obtaining unit and the 3rd obtaining unit, wherein:First obtains unit, for obtaining the filename of above-mentioned file Claim and file path.Second obtaining unit, for according to the file name, file path and metadata mark for pre-setting Corresponding relation, obtains metadata mark corresponding with above-mentioned file.3rd obtaining unit, for being had from storage device The metadata of above-mentioned file identification.
Metadata information can include file identification, and the second obtaining unit can include:First obtains subelement, for root According to the corresponding relation of the file name, file path and file identification for pre-setting, the file identification of above-mentioned file data is obtained; And second obtain subelement, for according to the file identification that pre-sets and the corresponding relation of metadata mark, obtaining above-mentioned The corresponding metadata mark of file data.
Metadata information can include file identification, in a kind of file data recovery device provided in an embodiment of the present invention also Can include:4th obtaining unit, the 5th obtaining unit and the 6th obtaining unit, wherein:4th obtaining unit, for obtaining The file name and file path of above-mentioned file.5th obtaining unit, for according to the file name, file road for pre-setting Footpath and the corresponding relation of file identification, obtain the file identification with above-mentioned file data.6th obtaining unit, for setting from storage Standby middle metadata of the acquisition with above-mentioned file identification.
In a kind of file data recovery device provided in an embodiment of the present invention, the first acquisition module can include:Scanning element And the 7th obtaining unit, wherein:Scanning element, for scan range of physical addresses in storage device belong to above-mentioned first preset The physical block of address realm.7th obtaining unit, for belonging to the thing of above-mentioned first preset address scope from range of physical addresses The metadata with above-mentioned metadata mark is obtained in reason block.
Figure 13 is referred to, is a kind of apparatus structure of another embodiment of file restoring device provided in an embodiment of the present invention Schematic diagram, the storage device in file restoring device are also stored with file metadata, and the file metadata includes file unit number It is believed that ceasing and identifying the file metadata mark of the file metadata information, the file metadata is the first file correspondence Metadata, first file is stored with the corresponding all metadata of the file, and above-mentioned file data recovery device includes: Second acquisition module 1301, Second processing module 1302, the second comprising modules 1303 and recovery module 1304, wherein:
Second acquisition module 1301, for the file with file metadata mark is obtained from the storage device Metadata.
Second processing module 1302, for for each file metadata, according to the file unit in the file metadata Data message reads the second binary data of relevant position in the storage device, is calculated according to second binary data Go out the second check value, when second check value is with the file metadata identity equality, parse second binary number According to obtain the first file data corresponding with second binary data.
Second comprising modules 1303, for obtaining the first file being made up of first file data.
Recovery module 1304, for according to each above-mentioned file of above-mentioned first file access pattern.
Figure 14 is referred to, is a kind of structural representation of the metadata storage system of file provided in an embodiment of the present invention, The metadata storage system of this document includes:Processor 1401, communication bus 1402 and storage device 1403, wherein:
Wherein processor 1401, memorizer 1403 complete mutual communication by communication bus 1402.
Processor 1401 is used for configuration processor.
Storage device 1403 is used to deposit program.
Program can include program code, and said procedure code includes computer-managed instruction.
The possibly central processor CPU of processor 1401, or specific integrated circuit ASIC(Application Specific Integrated Circuit), or be arranged to implement one or more integrated electricity of the embodiment of the present invention Road.
Storage device 1403 may include high-speed RAM memorizer, it is also possible to also including nonvolatile memory(non- volatile memory), for example, at least one disk memory.
Wherein said procedure is used for:
Obtained and each described number of files according to the memory space being respectively allocated for one or more file datas in file According to the corresponding metadata information of difference, the metadata information includes the file identification of the file;
Respectively one or more described file datas are stored to the memory space in the form of the first binary data;
First number corresponding with file data difference each described is calculated respectively according to each described first binary data According to mark, the metadata is identified for identifying the metadata information;
For each metadata, arbitrary physical block with idle storage space is selected from the storage device, by institute State metadata to store to the physical block, the metadata includes a metadata information and identifies a metadata information Metadata is identified.
Optionally, said procedure can include the functional module shown in Figure 10 to Figure 11.
The embodiment of the present invention additionally provides a kind of structural representation of file data recovery system, and the file includes one Or multiple file datas, the corresponding metadata of the file data includes metadata information and identifies the metadata information Metadata is identified, and the metadata information is obtained according to the memory space for file data distribution, the metadata Mark is calculated according to the first binary data and preset algorithm, and the file data is with first binary system The form of data is stored in the memory space, and this document data recovery system includes:Processor, storage device and logical Letter bus, wherein processor, storage device complete mutual communication by communication bus.
Processor is used for configuration processor.
Storage device is used to deposit program.
Program can include program code, and said procedure code includes computer-managed instruction.Processor is probably one Central processor CPU, or specific integrated circuit ASIC(Application Specific Integrated Circuit), or be arranged to implement one or more integrated circuits of the embodiment of the present invention.
Storage device may include high-speed RAM memorizer, it is also possible to also including nonvolatile memory(non-volatile memory), for example, at least one disk memory.
Wherein said procedure is used for:
When receiving the request of recovery file, the metadata with metadata mark is obtained from the storage device;
For each metadata, relevant position in the storage device is read according to the metadata information in the metadata The first binary data, calculate check value according to first binary data and the preset algorithm, when the school When value is tested with the metadata identity equality, determine that the metadata effectively, parses first binary data;
Obtain the file being made up of the file data for parsing the first binary data acquisition
Optionally, said procedure can include functional module shown in Figure 12 to Figure 13.
It should be noted that each embodiment in this specification is described by the way of progressive, each embodiment weight Point explanation is all difference with other embodiment, between each embodiment identical similar part mutually referring to. For device or system class embodiment, due to itself and embodiment of the method basic simlarity, so description is fairly simple, it is related Part is illustrated referring to the part of embodiment of the method.
Also, it should be noted that herein, such as first and second or the like relational terms are used merely to one Entity or operation are made a distinction with another entity or operation, and are not necessarily required or implied between these entities or operation There is any this actual relation or order.And, term " including ", "comprising" or its any other variant are intended to contain Lid nonexcludability is included, so that a series of process, method, article or equipment including key elements not only will including those Element, but also including other key elements being not expressly set out, or also include for this process, method, article or equipment Intrinsic key element.In the absence of more restrictions, the key element for being limited by sentence "including a ...", it is not excluded that Also there is other identical element in process, method, article or equipment including the key element.
The step of method described with reference to the embodiments described herein or algorithm, directly can be held with hardware, processor Capable software module, or the combination of the two is implementing.Software module can be placed in random access memory(RAM), internal memory, read-only deposit Reservoir(ROM), electrically programmable ROM, electrically erasable ROM, depositor, hard disk, moveable magnetic disc, CD-ROM or technology In any other form of storage medium well known in field.
The foregoing description of the disclosed embodiments, enables professional and technical personnel in the field to realize or using the present invention. Various modifications to these embodiments will be apparent for those skilled in the art, as defined herein General Principle can be realized without departing from the spirit or scope of the present invention in other embodiments.Therefore, the present invention The embodiments shown herein is not intended to be limited to, and is to fit to and principles disclosed herein and features of novelty phase one The most wide scope for causing.

Claims (12)

1. a kind of metadata storing method of file, it is characterised in that include:
Obtained and each described file data point according to the memory space being respectively allocated for one or more file datas in file Not corresponding metadata information, the metadata information include the file identification of the file;
Respectively one or more described file datas are stored to the memory space in the form of the first binary data;
Metadata mark corresponding with file data difference each described is calculated respectively according to each described first binary data Know, the metadata is identified for identifying the metadata information;
For each metadata, arbitrary physical block with idle storage space is selected from the storage device, by the unit To the physical block, the metadata includes a metadata information and identifies first number of a metadata information data storage According to mark.
2. the metadata storing method of file according to claim 1, it is characterised in that the file includes the first number of files According to metadata corresponding with first file data is the first metadata, is that the storage of the first file data distribution is empty Between be the first physical block, the byte length of first file data is not more than the first byte number and deducts obtained by the second byte number Difference, first byte number refer to the word of first physical block idle storage space before first file data is stored Joint number, second byte number are the byte numbers of first metadata, described the metadata to be stored to the physical block Including:
The metadata of first file data is stored to first physical block.
3. the metadata storing method of file according to claim 2, it is characterised in that described by first file data Metadata store to the first physical block and include:
Obtain the first file data to store to the first start offset address of first physical block;
By first start offset address, the byte length of first file data and 1 sum, it is defined as described first literary The metadata of number of packages evidence is stored to the second start offset address of first physical block;
With second start offset address as start offset address, first metadata is stored.
4. the metadata storing method of file according to claim 1, it is characterised in that it is described according to each described first After binary data calculates metadata mark corresponding with file data difference each described respectively, also include:
Corresponding metadata information is stored into the first file the All Files data that the file is included respectively;
According to the memory space in the storage device distributed for one or more first file datas in first file, Obtain and the corresponding file metadata information of the first file data difference each described, the file metadata information includes described The file identification of the first file;
Respectively one or more described first file datas are stored to the memory space in the form of the second binary data;
According to the second binary data that each described first file data is stored in the storage device respectively, calculate respectively Go out the file metadata mark of mark corresponding with the first file data difference each the described file metadata information;
For each file metadata, arbitrary physical block with idle storage space is selected from the storage device, by institute State file metadata to store into the physical block, the file metadata include a file metadata information and with mark institute State the file metadata mark of a file metadata information.
5. the metadata storing method of file according to claim 1, it is characterised in that described from the storage device Arbitrary physical block with idle storage space is selected, the metadata is stored to the physical block, also including reading The method of the file, the method for reading the file include:
Obtain the file name and file path of the file;
According to the corresponding relation of file name, the file path and metadata mark for pre-setting, obtain what the file included All Files data distinguish corresponding metadata mark;
For each metadata is identified, the metadata is obtained from the storage device and identifies corresponding metadata, according to The first binary data of file data corresponding with the metadata, root are obtained from the storage device according to the metadata The first check value is calculated according to first binary data, when first check value is with the metadata identity equality, First binary data is parsed, to obtain file data corresponding with first binary data;
The file that acquisition is made up of each the described file data for parsing.
6. a kind of file access pattern method, it is characterised in that the file includes one or more file datas, the file data Corresponding metadata includes metadata information and identifies the metadata mark of the metadata information, and the metadata information is Obtained according to the memory space that distributes for the file data, the metadata mark be according to the first binary data and What preset algorithm was calculated, the file data is in the form of first binary data to be stored in the memory space In, the file data restoration methods include:
When receiving the request of recovery file, the metadata with metadata mark is obtained from the storage device;
For each metadata, the of relevant position in the storage device is read according to the metadata information in the metadata Binary evidence, calculates check value according to first binary data and the preset algorithm, when the check value During with the metadata identity equality, determine that the metadata effectively, parses first binary data;
Obtain the file being made up of the file data for parsing the first binary data acquisition.
7. file access pattern method according to claim 6, it is characterised in that the storage device is also stored with file unit number According to the file metadata includes file metadata information and identifies the file metadata mark of the file metadata information Know, the file metadata is the corresponding metadata of the first file, and first file is stored with, and the file is corresponding to be owned Metadata, the file access pattern method also include:
The file metadata with file metadata mark is obtained from the storage device;
For each file metadata, read in the storage device according to the file metadata information in the file metadata Second binary data of relevant position, calculates the second check value according to second binary data, when second school When value is tested with the file metadata identity equality, parse second binary data, to obtain and second binary system Corresponding first file data of data;
The first file that acquisition is made up of first file data;
The file according to first file access pattern.
8. the metadata storage device of a kind of file, it is characterised in that include:
First acquisition module, for according to the memory space being respectively allocated for one or more file datas in file obtain with it is each The individual file data distinguishes corresponding metadata information, and the metadata information includes the file identification of the file;
Memory module, for storing one or more described file datas to described in the form of the first binary data respectively Memory space;
Computing module, is calculated respectively according to each described first binary data corresponding with file data difference each described Metadata is identified, and the metadata is identified for identifying the metadata information;
First processing module, for for each metadata, selecting arbitrary with idle storage space from the storage device Physical block, the metadata is stored to the physical block, the metadata includes that a metadata information and mark are described The metadata mark of one metadata information.
9. the metadata storage device of file according to claim 8, it is characterised in that the file includes the first number of files According to metadata corresponding with first file data is the first metadata, is that the storage of the first file data distribution is empty Between be the first physical block, the byte length of first file data is not more than the first byte number and deducts obtained by the second byte number Difference, first byte number refer to the word of first physical block idle storage space before first file data is stored Joint number, second byte number is the byte length of first metadata, and the first processing module includes:
Memory element, for the metadata of first file is stored to first physical block.
10. the metadata storage device of file according to claim 9, it is characterised in that the metadata storage of the file Device also includes the device for reading file, and the device of the reading file includes:
Second acquisition module, for obtaining the file name and file path of the file;
3rd acquisition module, for the corresponding relation according to the file name, file path and metadata mark for pre-setting, obtains The All Files data that obtaining the file includes distinguish corresponding metadata mark;
Second processing module, for identifying for each metadata, obtains the metadata mark from the storage device Know corresponding metadata, file data corresponding with the metadata is obtained from the storage device according to the metadata First binary data, calculates the first check value according to first binary data, when first check value with it is described During metadata identity equality, first binary data is parsed, to obtain file corresponding with first binary data Data;
File module is obtained, for obtaining the file being made up of each the described file data for parsing.
11. a kind of file restoring devices, it is characterised in that the file includes one or more file datas, the number of files Include metadata information according to corresponding metadata and identify the metadata mark of the metadata information, the metadata information To be obtained according to the memory space for file data distribution, metadata mark be according to the first binary number according to this And preset algorithm is calculated, the file data is that the storage is stored in the form of first binary data is empty Between in, the file data recovery device includes:
First acquisition module, for receiving during the request for recovering file, obtains with metadata mark from the storage device The metadata of knowledge;
First processing module, for for each metadata, reading the storage according to the metadata information in the metadata First binary data of relevant position in equipment, calculates school according to first binary data and the preset algorithm Value is tested, when the check value is with the metadata identity equality, determines that the metadata effectively, parses first binary system Data;
First comprising modules, for obtaining the file being made up of the file data for parsing the first binary data acquisition.
12. according to claim 11 file restoring device, it is characterised in that the storage device be also stored with file unit number According to the file metadata includes file metadata information and identifies the file metadata mark of the file metadata information Know, the file metadata is the corresponding metadata of the first file, and first file is stored with, and the file is corresponding to be owned Metadata, the file data recovery device also include:
Second acquisition module, for the file metadata with file metadata mark is obtained from the storage device;
Second processing module, for for each file metadata, according to the file metadata information in the file metadata The second binary data of relevant position in the storage device is read, the second school is calculated according to second binary data Value is tested, when second check value is with the file metadata identity equality, second binary data is parsed, to obtain The first file data corresponding with second binary data;
Second comprising modules, for obtaining the first file being made up of first file data;
Recovery module, for the file according to first file access pattern.
CN201310656195.0A 2013-12-06 2013-12-06 Methods, devices and systems for file metadata storage and file recovery Active CN103699585B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310656195.0A CN103699585B (en) 2013-12-06 2013-12-06 Methods, devices and systems for file metadata storage and file recovery

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201310656195.0A CN103699585B (en) 2013-12-06 2013-12-06 Methods, devices and systems for file metadata storage and file recovery

Publications (2)

Publication Number Publication Date
CN103699585A CN103699585A (en) 2014-04-02
CN103699585B true CN103699585B (en) 2017-04-19

Family

ID=50361113

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310656195.0A Active CN103699585B (en) 2013-12-06 2013-12-06 Methods, devices and systems for file metadata storage and file recovery

Country Status (1)

Country Link
CN (1) CN103699585B (en)

Families Citing this family (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103986842B (en) * 2014-05-30 2019-02-15 努比亚技术有限公司 A kind of collecting method and device of contact data
EP3148200B1 (en) * 2014-06-30 2020-06-17 Sony Corporation Information processing device and method selecting content files based on encoding parallelism type
CN104506619B (en) 2014-12-22 2018-06-05 华为技术有限公司 A kind of data backup, restoration methods and its device, server
CN107301183B (en) * 2016-04-14 2020-02-18 杭州海康威视数字技术股份有限公司 File storage method and device
CN107301177B (en) * 2016-04-14 2020-02-18 杭州海康威视数字技术股份有限公司 File storage method and device
CN107870940B (en) * 2016-09-28 2021-06-18 杭州海康威视数字技术股份有限公司 File storage method and device
CN106960011A (en) * 2017-02-28 2017-07-18 无锡紫光存储系统有限公司 Metadata of distributed type file system management system and method
CN107039077A (en) * 2017-03-20 2017-08-11 北京握奇智能科技有限公司 A kind of method and apparatus for extending the erasable chip life-span
CN108733309B (en) 2017-04-17 2021-06-11 伊姆西Ip控股有限责任公司 Storage management method, apparatus and computer readable medium
CN109426587B (en) * 2017-08-25 2020-08-28 杭州海康威视数字技术股份有限公司 Data recovery method and device
CN107861842B (en) * 2017-11-08 2021-10-15 郑州云海信息技术有限公司 Metadata damage detection method, system, equipment and storage medium
CN110879800B (en) * 2018-09-05 2023-08-18 阿里巴巴集团控股有限公司 Data writing, compressing and reading method, data processing method and device
CN110377561A (en) * 2019-07-19 2019-10-25 深圳前海微众银行股份有限公司 A kind of file management method and device
CN110688346A (en) * 2019-09-30 2020-01-14 北京金山安全软件有限公司 Element management method and device, electronic equipment and storage medium
CN110807000B (en) * 2019-10-25 2022-06-10 北京达佳互联信息技术有限公司 File repair method and device, electronic equipment and storage medium
CN113050893B (en) * 2021-03-30 2022-08-30 重庆紫光华山智安科技有限公司 High-concurrency file storage method, system, medium and electronic terminal
CN113553010B (en) * 2021-07-27 2023-09-12 成都统信软件技术有限公司 Optical disc file verification method, optical disc recording method and computing device
CN114328421B (en) * 2022-03-17 2022-06-10 联想凌拓科技有限公司 Metadata service architecture management method, computer system, electronic device and medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101067822A (en) * 2006-05-03 2007-11-07 国际商业机器公司 Hierarchical storage management of metadata
CN101167058A (en) * 2005-04-25 2008-04-23 皇家飞利浦电子股份有限公司 Apparatus, method and system for restoring files
CN102239468A (en) * 2008-12-02 2011-11-09 起元技术有限责任公司 Visualizing relationships between data elements and graphical representations of data element attributes
TW201316745A (en) * 2011-10-11 2013-04-16 Chunghwa Telecom Co Ltd Data backup system and method for mobile device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5339432B2 (en) * 2009-02-25 2013-11-13 日本電気株式会社 Storage system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101167058A (en) * 2005-04-25 2008-04-23 皇家飞利浦电子股份有限公司 Apparatus, method and system for restoring files
CN101067822A (en) * 2006-05-03 2007-11-07 国际商业机器公司 Hierarchical storage management of metadata
CN102239468A (en) * 2008-12-02 2011-11-09 起元技术有限责任公司 Visualizing relationships between data elements and graphical representations of data element attributes
TW201316745A (en) * 2011-10-11 2013-04-16 Chunghwa Telecom Co Ltd Data backup system and method for mobile device

Also Published As

Publication number Publication date
CN103699585A (en) 2014-04-02

Similar Documents

Publication Publication Date Title
CN103699585B (en) Methods, devices and systems for file metadata storage and file recovery
CN104506619B (en) A kind of data backup, restoration methods and its device, server
CN102915278A (en) Data deduplication method
CN104978151A (en) Application awareness based data reconstruction method in repeated data deletion and storage system
JP2010157204A (en) Content addressable storage system and method employing searchable block
CN102831222A (en) Differential compression method based on data de-duplication
JP2010157204A5 (en)
CN105589894B (en) Document index establishing method and device and document retrieval method and device
EP3438845A1 (en) Data updating method and device for a distributed database system
Strzelczak et al. Concurrent Deletion in a Distributed {Content-Addressable} Storage System with Global Deduplication
CN107111460A (en) Use the data de-duplication of block file
CN104360914A (en) Incremental snapshot method and device
CN104965835B (en) A kind of file read/write method and device of distributed file system
CN106445643A (en) Method and device for cloning and updating virtual machine
CN111125298A (en) Method, equipment and storage medium for reconstructing NTFS file directory tree
CN106354587A (en) Mirror image server and method for exporting mirror image files of virtual machine
CN108009049A (en) The offline restoration methods of MYISAM storage engines deletion records, storage medium
CN112800007B (en) Directory entry expansion method and system suitable for FAT32 file system
EP2856359B1 (en) Systems and methods for storing data and eliminating redundancy
CN105260423A (en) Duplicate removal method and apparatus for electronic cards
CN104778099B (en) A kind of damaged file reconstructing methods of the YAFFS2 based on old version
CN103714121A (en) Index record management method and device
CN102831240B (en) The storage means of extended metadata file and storage organization
CN102929976B (en) Backup data access method and device
CN101901172A (en) Data processing device and method

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant