CN105573861B - The restorative procedure and device of distributed file system small file - Google Patents

The restorative procedure and device of distributed file system small file Download PDF

Info

Publication number
CN105573861B
CN105573861B CN201510897346.0A CN201510897346A CN105573861B CN 105573861 B CN105573861 B CN 105573861B CN 201510897346 A CN201510897346 A CN 201510897346A CN 105573861 B CN105573861 B CN 105573861B
Authority
CN
China
Prior art keywords
logical disk
small documents
small
disk
logical
Prior art date
Application number
CN201510897346.0A
Other languages
Chinese (zh)
Other versions
CN105573861A (en
Inventor
康撼宇
郭照斌
姜国梁
季旻
苗艳超
杨鹏
Original Assignee
曙光信息产业股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 曙光信息产业股份有限公司 filed Critical 曙光信息产业股份有限公司
Priority to CN201510897346.0A priority Critical patent/CN105573861B/en
Publication of CN105573861A publication Critical patent/CN105573861A/en
Application granted granted Critical
Publication of CN105573861B publication Critical patent/CN105573861B/en

Links

Abstract

The present invention provides the restorative procedures and device of a kind of distributed file system small file, belong to field of computer technology.It include the first small documents to be repaired in the first Logical Disk the described method includes: determining the first Logical Disk to break down;Determine the first Logical Disk group comprising the first Logical Disk, storage the second small documents relevant to the first small documents in the Logical Disk that the first Logical Disk group includes;Batch reads all second small documents in the first Logical Disk group;The first small documents are repaired according to the second small documents.Present invention determine that the Logical Disk to break down, which includes the first small documents to be repaired;Batch reads all the second small documents relevant to the first small documents in the Logical Disk group comprising the Logical Disk;The first small documents are repaired according to the second small documents, so that the related small documents of Different Logic disk in Logical Disk group read, repaired by batch, avoids repairing the frequent fritter read-write operation of small documents bring one by one, improve the reparation speed of small documents, promote repairing performance.

Description

The restorative procedure and device of distributed file system small file

Technical field

The present invention relates to the restorative procedures and dress of field of computer technology more particularly to distributed file system small file It sets.

Background technique

For distributed file system, heap file can store on every piece of disk.For small documents, every piece of disk The data volume of upper storage is more considerable.When disk breaks down, will cause large amount of small documents, partial data occur not available Situation.Distributed file system can repair the small documents there are failure.

Distributed file system is limited to the tolerance of disk failure, before repairing completion, due in system The complete damage that new failure causes certain small documents is generated, is needed in the shortest possible time, to there are the small texts of failure Part is repaired.

Currently, the restorative procedure of distributed file system small file are as follows: to each small documents, be read out, repair.When When the small documents number of reparation is more, current method will cause a large amount of fritters on disk and read at random, due to distributed document The storage medium that system uses performance when carrying out fritter and reading at random is lower, and it is relatively low to will lead to repairing performance.

Summary of the invention

To solve the above problems, the embodiment of the present invention proposes a kind of restorative procedure of distributed file system small file And device.

On the one hand, the embodiment of the invention provides a kind of restorative procedures of distributed file system small file, comprising:

It determines the first Logical Disk to break down, includes the first small documents to be repaired in first Logical Disk;

It determines the first Logical Disk group comprising first Logical Disk, is deposited in the Logical Disk that the first Logical Disk group includes Store up the second small documents relevant to first small documents;

Batch reads all second small documents in the first Logical Disk group;

First small documents are repaired according to second small documents.

Optionally, second small documents are the backup small documents of first small documents and first small documents, or First small documents and the small documents formed with partial data that first small documents are same file or described first small File, first small documents backup small documents and with first small documents be same file partial data formed it is small File.

Optionally, before determining first Logical Disk to break down, further includes:

It is at least one second Logical Disk by disk cutting;

The second Logical Disk for storing third small documents is formed into the second Logical Disk group;

The third small documents stored in the second Logical Disk group are related each other.

Optionally, the third small documents stored in the second Logical Disk group are related each other, comprising: second Logical Disk All third small documents stored in group be mutually backups or the second Logical Disk group in all third small documents for storing be same A part is each other in small documents that the partial data of one file is formed or the third small documents stored in the second Logical Disk group Backup, another part are the small documents that the partial data of same file is formed.

Optionally, if composition the second Logical Disk group the second Logical Disk be it is multiple, form the second Logical Disk group Second Logical Disk is located on different nodes.

Optionally, node breaks down, determining first Logical Disk to break down, comprising:

Determine each disk on the node to break down;

Determine the third Logical Disk that each disk is syncopated as;

It regard all third Logical Disks as the first logic magnetic disc.

Optionally, disk breaks down, determining first Logical Disk to break down, comprising:

Determine the 4th Logical Disk that the disk to break down is syncopated as;

It regard all 4th Logical Disks as the first logic magnetic disc.

Optionally, after first small documents according to second small documents reparation, further includes:

The 5th Logical Disk, the disk of the 5th Logical Disk and first Logical Disk is written into the first small documents after reparation Number identical, the physical location of the 5th Logical Disk is different from the physical location of first Logical Disk;

First Logical Disk is mapped into the 5th Logical Disk.

On the other hand, the embodiment of the invention provides a kind of prosthetic devices of distributed file system small file, comprising:

First determining module includes to be repaired in first Logical Disk for determining the first Logical Disk to break down The first small documents;

Second determining module, for determining the first Logical Disk of the first Logical Disk determined comprising first determining module Group, storage the second small documents relevant to first small documents in the Logical Disk that the first Logical Disk group includes;

Read module, for reading all second small texts in the first Logical Disk group that second determining module determines in batches Part;

Repair module, the second small documents for being read according to the read module repair first small documents.

Optionally, second small documents are the backup small documents of first small documents and first small documents, or First small documents and the small documents formed with partial data that first small documents are same file or described first small File, first small documents backup small documents and with first small documents be same file partial data formed it is small File.

Optionally, described device, further includes:

Cutting module, for being at least one second Logical Disk by disk cutting;

Composite module, the second Logical Disk for that will store third small documents form the second Logical Disk group;

The third small documents stored in the second Logical Disk group are related each other.

Optionally, the third small documents stored in the second Logical Disk group are related each other, comprising: second Logical Disk All third small documents stored in group be mutually backups or the second Logical Disk group in all third small documents for storing be same A part is each other in small documents that the partial data of one file is formed or the third small documents stored in the second Logical Disk group Backup, another part are the small documents that the partial data of same file is formed.

Optionally, if composition the second Logical Disk group the second Logical Disk be it is multiple, form the second Logical Disk group Second Logical Disk is located on different nodes.

Optionally, node breaks down, first determining module, comprising:

First determination unit, for determining each disk on the node to break down;

Second determination unit, the third Logical Disk being syncopated as determining each disk that first determination unit determines;

Third determination unit, all third Logical Disks for determining second determination unit are used as the first logic Disk.

Optionally, disk breaks down, first determining module, comprising:

4th determination unit, the 4th Logical Disk being syncopated as determining the disk to break down;

5th determination unit, all 4th Logical Disks for determining the 4th determination unit are used as the first logic Disk.

Optionally, described device, further includes:

Writing module, for after repairing the first small documents be written the 5th Logical Disk, the 5th Logical Disk with it is described The reel number of first Logical Disk is identical, and the physical location of the 5th Logical Disk is different from the physical location of first Logical Disk;

Mapping block, for first Logical Disk to be mapped to the 5th Logical Disk.

It has the beneficial effect that:

It determines the first Logical Disk to break down, includes the first small documents to be repaired in the first Logical Disk;Determination includes First Logical Disk group of the first Logical Disk stores relevant to the first small documents second in the Logical Disk that the first Logical Disk group includes Small documents;Batch reads all second small documents in the first Logical Disk group;The first small documents are repaired according to the second small documents, make the The second small documents of Different Logic disk are read by batch, repair the first small documents according to the second small documents in one Logical Disk group, keep away Exempt to read one by one and repairs small documents bring and frequently enter operation, so that the reparation speed of small documents is improved, promotion repairing performance.

Detailed description of the invention

Specific embodiments of the present invention are described below with reference to accompanying drawings, in which:

Fig. 1 shows a kind of restorative procedure of the distributed file system small file provided in an embodiment of the present invention Flow diagram;

Fig. 2 shows the reparations of another distributed file system small file provided in another embodiment of the invention The flow diagram of method;

Fig. 3 shows the reparation of the first the distributed file system small file provided in another embodiment of the invention The structural schematic diagram of device;

Fig. 4 shows the reparation of second of the distributed file system small file provided in another embodiment of the invention The structural schematic diagram of device;

Fig. 5 shows the structural schematic diagram of the first determining module of one kind provided in another embodiment of the invention;

Fig. 6 shows the structural schematic diagram of the first determining module of another kind provided in another embodiment of the invention;

Fig. 7 shows the reparation of the third distributed file system small file provided in another embodiment of the invention The structural schematic diagram of device.

Specific embodiment

In order to which technical solution of the present invention and advantage is more clearly understood, below in conjunction with attached drawing to of the invention exemplary Embodiment is described in more detail, it is clear that and the described embodiments are merely a part of the embodiments of the present invention, rather than The exhaustion of all embodiments.And in the absence of conflict, the feature in the embodiment and embodiment in this explanation can be mutual It combines.

When being repaired to distributed file system small file, to each small documents, it is read out, repairs.When repairing When multiple small documents number is more, current method will cause a large amount of fritters on disk and read at random, due to distributed field system The storage medium used performance when carrying out fritter and reading at random of uniting is lower, and it is relatively low to will lead to repairing performance.The invention proposes A kind of restorative procedure of distributed file system small file, this method are applied to a kind of terminal, run and be distributed in the terminal The prosthetic device of formula file system small file, for example following Fig. 3 to Fig. 7 of the prosthetic device of the distributed file system small file The prosthetic device of distributed file system small file described in embodiment shown in any figure.It is small in the distributed file system The prosthetic device of file can determine the first Logical Disk of failure, include the first small text to be repaired in the first Logical Disk Part;Determine the first Logical Disk group comprising the first Logical Disk, storage and the first small text in the Logical Disk that the first Logical Disk group includes Relevant second small documents of part;Batch reads all second small documents in the first Logical Disk group;The is repaired according to the second small documents One small documents read the second small documents of Different Logic disk in the first Logical Disk group by batch, according to the second small documents reparation First small documents avoid reading one by one and repair small documents bring and frequently enter operation, so that the reparation speed of small documents is improved, Promote repairing performance.

In conjunction with above-mentioned implementation environment, embodiment shown in Figure 1 present embodiments provides a kind of this parallel data of two-pack The querying method in library, method flow provided in this embodiment are specific as follows:

101: determining the first Logical Disk to break down, include the first small documents to be repaired in the first Logical Disk;

102: determine include the first Logical Disk the first Logical Disk group, in the Logical Disk that the first Logical Disk group includes storage with Relevant second small documents of first small documents;

Optionally, the second small documents be the first small documents and the first small documents backup small documents or the first small documents and It is small with the backup of the small documents or the first small documents, the first small documents of the partial data formation that the first small documents are same file File and the small documents formed with the first small documents for the partial data of same file.

103: batch reads all second small documents in the first Logical Disk group;

104: repairing the first small documents according to the second small documents.

Optionally it is determined that before the first Logical Disk to break down, further includes:

It is at least one second Logical Disk by disk cutting;

The second Logical Disk for storing third small documents is formed into the second Logical Disk group;

The third small documents stored in second Logical Disk group are related each other.

Optionally, the third small documents stored in the second Logical Disk group are related each other, comprising: store in the second Logical Disk group All third small documents be mutually backups or the second Logical Disk group in all third small documents for storing be same file part A part is mutually backups in the third small documents stored in the small documents of data formation or the second Logical Disk group, and another part is The small documents that the partial data of same file is formed.

Optionally, if composition the second Logical Disk group the second Logical Disk be it is multiple, form the second of the second Logical Disk group Logical Disk is located on different nodes.

Optionally, node breaks down, and determines the first Logical Disk to break down, comprising:

Determine each disk on the node to break down;

Determine the third Logical Disk that each disk is syncopated as;

It regard all third Logical Disks as the first logic magnetic disc.

Optionally, disk breaks down, and determines the first Logical Disk to break down, comprising:

Determine the 4th Logical Disk that the disk to break down is syncopated as;

It regard all 4th Logical Disks as the first logic magnetic disc.

Optionally, after according to the second small documents the first small documents of reparation, further includes:

The 5th Logical Disk is written into the first small documents after reparation, the 5th Logical Disk is identical as the reel number of the first Logical Disk, The physical location of 5th Logical Disk is different from the physical location of the first Logical Disk;

First Logical Disk is mapped into the 5th Logical Disk.

The utility model has the advantages that

It determines the first Logical Disk to break down, includes the first small documents to be repaired in the first Logical Disk;Determination includes First Logical Disk group of the first Logical Disk stores relevant to the first small documents second in the Logical Disk that the first Logical Disk group includes Small documents;Batch reads all second small documents in the first Logical Disk group;The first small documents are repaired according to the second small documents, make the The second small documents of Different Logic disk are read by batch, repair the first small documents according to the second small documents in one Logical Disk group, keep away Exempt to read one by one and repairs small documents bring and frequently enter operation, so that the reparation speed of small documents is improved, promotion repairing performance.

In conjunction with above-mentioned implementation environment, embodiment shown in Figure 2 present embodiments provides a kind of distributed file system The restorative procedure of small file, method flow provided in this embodiment are specific as follows:

201: being at least one second Logical Disk by disk cutting;

The present embodiment is not defined to by the specific cutting method that disk cutting is at least one the second Logical Disk, including But be not limited to, according to the relationship between disk space and small documents by disk cutting be at least one second Logical Disk.

The present embodiment is not also defined the particular number of the second Logical Disk.

202: the second Logical Disk for storing third small documents is formed into the second Logical Disk group;

Wherein, the third small documents stored in the second Logical Disk group are related each other.

Distributed file system can also can when carrying out file storage by a file distribution storage at multiple small documents Small documents are backuped.Therefore, for the third small documents that are stored in the second Logical Disk group relevant physical relationship each other, this reality Apply example without limiting, including but not limited to: all third small documents stored in the second Logical Disk group are mutually backups or second All third small documents stored in Logical Disk group are the small documents or the second Logical Disk group that the partial data of same file is formed A part is mutually backups in the third small documents of middle storage, and another part is the small documents that the partial data of same file is formed.

Preferably, if composition the second Logical Disk group the second Logical Disk be it is multiple, form the second of the second Logical Disk group Logical Disk is located on different nodes.

One Logical Disk can store multiple small documents, but the multiple small documents stored are respectively positioned on same Logical Disk group. Therefore.By step 201 and step 202, Logical Disk group, Logical Disk and small documents form following relationship: a Logical Disk group packet Containing an excessively multiple Logical Disk, a logic disk storage one or more small documents.All small documents of one file are located at one A Logical Disk group, a Logical Disk belong to a Logical Disk group.

203: determining the first Logical Disk to break down;

It wherein, include the first small documents to be repaired in the first Logical Disk.

Different according to the position of failure, the first Logical Disk determines that method is also different, specifically,

● if node breaks down, it is determined that the method for the first Logical Disk of failure are as follows:

1) each disk on the node to break down is determined;

2) the third Logical Disk that each disk is syncopated as is determined;

3) all third Logical Disks are regard as the first logic magnetic disc.

● if disk breaks down, it is determined that the method for the first Logical Disk of failure are as follows:

1) the 4th Logical Disk that the disk to break down is syncopated as is determined;

2) all 4th Logical Disks are regard as the first logic magnetic disc.

It should be noted that the first logic magnetic disc determined in step 203 may be multiple.For multiple first logic magnetic Disk situation can be completed with 1) selecting first logic magnetic disc to execute following step 204 to step 208 to the first logic magnetic disc In small documents reparation, 2) repeat 1), until all first logic magnetic discs in small documents be repaired.

204: determining the first Logical Disk group comprising the first Logical Disk;

Wherein, the second small documents relevant to the first small documents are stored in the Logical Disk that the first Logical Disk group includes.First Logical Disk group includes at least one Logical Disk, if the first Logical Disk group includes multiple Logical Disks, the Logical Disk for including is located at not With on node.It include the first small documents in second small documents, i.e. the second small documents of part are identical as the first small documents.

The first Logical Disk group in this step is one in the second Logical Disk group formed in step 202.

In addition, the second small documents relevant to the first small documents are stored in the Logical Disk for including due to the first Logical Disk group, And first Logical Disk group include Logical Disk in include the first logic magnetic disc, therefore, in the second small documents include the first small documents.The Relationship between one small documents and the second small documents are as follows: the second small documents are the small text of backup of the first small documents and the first small documents Part or the first small documents and the small documents or the first small documents, the formed with partial data that the first small documents are same file The backup small documents of one small documents and the small documents formed with the first small documents for the partial data of same file.

205: batch reads all second small documents in the first Logical Disk group;

Specifically, batch reads in Logical Disk group all small documents to memory;

In order to avoid the processing of small documents one by one bring frequently enters operation, this step is by small documents all in Logical Disk group Bulk is formed, and then batch reads in Logical Disk group that all small documents are to memory, to improve the reparation speed of small documents, promotion is repaired Renaturation energy.

In addition, disk size is larger relative to memory, memory can not All Files in storage disk, it is provided in this embodiment In method, each Logical Disk that Logical Disk group includes is a part of affiliated disk, and all small documents are corresponding in Logical Disk group Data volume be less than disk size, therefore, the reasonable cutting of step 201, it is ensured that memory can store institute in Logical Disk group There are small documents.

In addition, all small documents of same file are stored in a Logical Disk group, this step batch is read in step 202 Take all small documents in Logical Disk group, it is ensured that all small documents relevant to the first small documents to be repaired are all read to interior It deposits, guarantees use when subsequent reparation.

206: repairing the first small documents according to the second small documents;

Specifically, repairing the first small documents according to the second small documents in memory.

The present embodiment is not defined the concrete mode for repairing the small documents read in memory, such as: in memory It is middle that first small documents are repaired according to the second small documents batch.

Specific restorative procedure can be carried out according to existing restorative procedure, and the present embodiment no longer describes.

207: the 5th Logical Disk is written into the first small documents after reparation;

Wherein, the 5th Logical Disk is identical as the reel number of the first Logical Disk, the physical location and the first logic of the 5th Logical Disk The physical location of disk is different.

The present embodiment is not defined the concrete mode that the 5th Logical Disk is written in the small documents after repairing, such as: it will The first small documents batch the 5th Logical Disk of write-in after reparation.

Specific wiring method can be carried out according to existing wiring method, and the present embodiment no longer describes.

208: the first Logical Disk is mapped into the 5th Logical Disk.

Specific mapping method can be carried out according to existing mapping method, and the present embodiment no longer describes.

Step 204 is recycled to step 208 and is executed, i.e., for each the first logic magnetic disc, be performed both by a step 204 to Step 208, until the small documents in all first logic magnetic discs are repaired.

In addition, before Logical Disk or Logical Disk group first, second, third, fourth, 5th etc. being only sequence in the present embodiment Number, for distinguishing the Logical Disk or Logical Disk group of different phase, no other differences.First Logical Disk can phase with the 5th Logical Disk With can also be different, the first Logical Disk group can be the same or different with the second Logical Disk group, equally, first, second, the Three, it can be the same or different between the four, the 5th Logical Disks, the present embodiment is not limited in any way.

The utility model has the advantages that

It determines the first Logical Disk to break down, includes the first small documents to be repaired in the first Logical Disk;Determination includes First Logical Disk group of the first Logical Disk stores relevant to the first small documents second in the Logical Disk that the first Logical Disk group includes Small documents;Batch reads all second small documents in the first Logical Disk group;The first small documents are repaired according to the second small documents, make the The second small documents of Different Logic disk are read by batch, repair the first small documents according to the second small documents in one Logical Disk group, keep away Exempt to read one by one and repairs small documents bring and frequently enter operation, so that the reparation speed of small documents is improved, promotion repairing performance.

Based on the same inventive concept, embodiment shown in Figure 3 present embodiments provides a kind of distributed file system The prosthetic device of small file, the principle solved the problems, such as due to these devices are repaired with a kind of distributed file system small file Compound method is similar, therefore the implementation of these devices may refer to the implementation of method, and overlaps will not be repeated.

Referring to Fig. 3, the device, comprising:

First determining module 301 includes to be repaired in the first Logical Disk for determining the first Logical Disk to break down First small documents;

Second determining module 302, for determining the first logic of the first Logical Disk determined comprising the first determining module 301 Disk group, storage the second small documents relevant to the first small documents in the Logical Disk that the first Logical Disk group includes;

Read module 303 is all second small in the first Logical Disk group that the second determining module 302 determines for reading in batches File;

Repair module 304, the second small documents for being read according to read module 303 repair the first small documents.

Wherein, the second small documents be the first small documents and the first small documents backup small documents or the first small documents and with First small documents are the small text of backup of small documents or the first small documents, the first small documents that the partial data of same file is formed Part and the small documents formed with the first small documents for the partial data of same file.

Referring to fig. 4, the device, further includes:

Cutting module 305, for being at least one second Logical Disk by disk cutting;

Composite module 306, the second Logical Disk for that will store third small documents form the second Logical Disk group;

Wherein, the third small documents stored in the second Logical Disk group are related each other.

Wherein, the third small documents stored in the second Logical Disk group are related each other, comprising: store in the second Logical Disk group All third small documents be mutually backups or the second Logical Disk group in all third small documents for storing be same file part number It is mutually backups according to a part in the third small documents stored in the small documents of formation or the second Logical Disk group, another part is same The small documents that the partial data of one file is formed.

Wherein, if composition the second Logical Disk group the second Logical Disk be it is multiple, form the second Logical Disk group second is patrolled Disk is collected to be located on different nodes.

Referring to Fig. 5, node breaks down, the first determining module 301, comprising:

First determination unit 3011, for determining each disk on the node to break down;

Second determination unit 3012, the third logic being syncopated as determining each disk that the first determination unit 3011 determines Disk;

Third determination unit 3013, all third Logical Disks for determining the second determination unit 3012 are used as first Logic magnetic disc.

Referring to Fig. 6, disk breaks down, the first determining module 301, comprising:

4th determination unit 3014, the 4th Logical Disk being syncopated as determining the disk to break down;

5th determination unit 3015, all 4th Logical Disks for determining the 4th determination unit 3014 are used as first Logic magnetic disc.

Referring to Fig. 7, the device, further includes:

The 5th Logical Disk is written for the first small documents after repairing in writing module 307, and the 5th Logical Disk is patrolled with first The reel number for collecting disk is identical, and the physical location of the 5th Logical Disk is different from the physical location of the first Logical Disk;

Mapping block 308, for the first Logical Disk to be mapped to the 5th Logical Disk.

It has the beneficial effect that:

It determines the first Logical Disk to break down, includes the first small documents to be repaired in the first Logical Disk;Determination includes First Logical Disk group of the first Logical Disk stores relevant to the first small documents second in the Logical Disk that the first Logical Disk group includes Small documents;Batch reads all second small documents in the first Logical Disk group;The first small documents are repaired according to the second small documents, make the The second small documents of Different Logic disk are read by batch, repair the first small documents according to the second small documents in one Logical Disk group, keep away Exempt to read one by one and repairs small documents bring and frequently enter operation, so that the reparation speed of small documents is improved, promotion repairing performance.

In above-described embodiment, it can be implemented using existing function component module.For example, processing module can adopt At least just have on the location-server used in existing location technology with existing data processing component and realize the function Component;It is then the component that equipment that any one has signal transfer functions all has as receiving module;Meanwhile locating The uses such as A, n parameter that reason module carries out calculate, intensity adjusts all are existing technological means, and those skilled in the art pass through Designing and developing accordingly can be realized.

For convenience of description, each section of apparatus described above is divided into various modules with function or unit describes respectively. Certainly, each module or the function of unit can be realized in same or multiple softwares or hardware in carrying out the present invention.

It should be understood by those skilled in the art that, the embodiment of the present invention can provide as method, system or computer program Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the present invention Apply the form of example.Moreover, it wherein includes the computer of computer usable program code that the present invention, which can be used in one or more, The computer program implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) produces The form of product.

The present invention be referring to according to the method for the embodiment of the present invention, the process of equipment (system) and computer program product Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates, Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or The function of being specified in multiple boxes.

These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one The step of function of being specified in a box or multiple boxes.

Although preferred embodiments of the present invention have been described, it is created once a person skilled in the art knows basic Property concept, then additional changes and modifications may be made to these embodiments.So it includes excellent that the following claims are intended to be interpreted as It selects embodiment and falls into all change and modification of the scope of the invention.

Claims (14)

1. a kind of restorative procedure of distributed file system small file characterized by comprising
It determines the first Logical Disk to break down, includes the first small documents to be repaired in first Logical Disk;
Determine include first Logical Disk the first Logical Disk group, in the Logical Disk that the first Logical Disk group includes storage with Relevant second small documents of first small documents;
Batch reads all second small documents in the first Logical Disk group;
First small documents are repaired according to second small documents;
Before determining first Logical Disk to break down, further includes:
It is at least one second Logical Disk by disk cutting;
The second Logical Disk for storing third small documents is formed into the second Logical Disk group;
The third small documents stored in the second Logical Disk group are related each other.
2. the method according to claim 1, wherein second small documents are first small documents and described The backup small documents of first small documents or first small documents and with first small documents be same file partial data The small documents of formation or first small documents, the backup small documents of first small documents and with first small documents it is The small documents that the partial data of same file is formed.
3. according to the method described in claim 2, it is characterized in that, the third small documents stored in the second Logical Disk group are mutual For correlation, comprising: all third small documents stored in the second Logical Disk group are mutually backups or the second Logical Disk group All third small documents of middle storage are the small documents that the partial data of same file is formed or deposit in the second Logical Disk group A part is mutually backups in the third small documents of storage, and another part is the small documents that the partial data of same file is formed.
4. according to the method described in claim 2, it is characterized in that, if the second Logical Disk of the second Logical Disk group of composition is more A, then the second Logical Disk for forming the second Logical Disk group is located on different nodes.
5. according to the method described in claim 4, it is characterized in that, node failure, determining first to break down Logical Disk, comprising:
Determine each disk on the node to break down;
Determine the third Logical Disk that each disk is syncopated as;
It regard all third Logical Disks as the first logic magnetic disc.
6. according to the method described in claim 4, it is characterized in that, disk failure, determining first to break down Logical Disk, comprising:
Determine the 4th Logical Disk that the disk to break down is syncopated as;
It regard all 4th Logical Disks as the first logic magnetic disc.
7. the method according to claim 1, wherein described small according to second small documents reparation described first After file, further includes:
The 5th Logical Disk, the reel number phase of the 5th Logical Disk and first Logical Disk is written into the first small documents after reparation Together, the physical location of the 5th Logical Disk is different from the physical location of first Logical Disk;
First Logical Disk is mapped into the 5th Logical Disk.
8. a kind of prosthetic device of distributed file system small file characterized by comprising
First determining module includes to be repaired the in first Logical Disk for determining the first Logical Disk to break down One small documents;
Second determining module, for determining the first Logical Disk group of the first Logical Disk determined comprising first determining module, The second small documents relevant to first small documents are stored in the Logical Disk that the first Logical Disk group includes;
Read module, for reading all second small documents in the first Logical Disk group that second determining module determines in batches;
Repair module, the second small documents for being read according to the read module repair first small documents;
Described device, further includes:
Cutting module, for being at least one second Logical Disk by disk cutting;
Composite module, the second Logical Disk for that will store third small documents form the second Logical Disk group;
The third small documents stored in the second Logical Disk group are related each other.
9. device according to claim 8, which is characterized in that second small documents are first small documents and described The backup small documents of first small documents or first small documents and with first small documents be same file partial data The small documents of formation or first small documents, the backup small documents of first small documents and with first small documents it is The small documents that the partial data of same file is formed.
10. device according to claim 9, which is characterized in that the third small documents stored in the second Logical Disk group It is related each other, comprising: all third small documents stored in the second Logical Disk group are mutually backups or second Logical Disk All third small documents stored in group are in the small documents that the partial data of same file is formed or the second Logical Disk group A part is mutually backups in the third small documents of storage, and another part is the small documents that the partial data of same file is formed.
11. device according to claim 9, which is characterized in that if the second Logical Disk of the second Logical Disk group of composition is more A, then the second Logical Disk for forming the second Logical Disk group is located on different nodes.
12. device according to claim 11, which is characterized in that node breaks down, first determining module, packet It includes:
First determination unit, for determining each disk on the node to break down;
Second determination unit, the third Logical Disk being syncopated as determining each disk that first determination unit determines;
Third determination unit, all third Logical Disks for determining second determination unit are used as the first logic magnetic Disk.
13. device according to claim 11, which is characterized in that disk breaks down, first determining module, packet It includes:
4th determination unit, the 4th Logical Disk being syncopated as determining the disk to break down;
5th determination unit, all 4th Logical Disks for determining the 4th determination unit are used as the first logic magnetic Disk.
14. device according to claim 8, which is characterized in that described device, further includes:
The 5th Logical Disk, the 5th Logical Disk and described first is written for the first small documents after repairing in writing module The reel number of Logical Disk is identical, and the physical location of the 5th Logical Disk is different from the physical location of first Logical Disk;
Mapping block, for first Logical Disk to be mapped to the 5th Logical Disk.
CN201510897346.0A 2015-12-08 2015-12-08 The restorative procedure and device of distributed file system small file CN105573861B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510897346.0A CN105573861B (en) 2015-12-08 2015-12-08 The restorative procedure and device of distributed file system small file

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510897346.0A CN105573861B (en) 2015-12-08 2015-12-08 The restorative procedure and device of distributed file system small file

Publications (2)

Publication Number Publication Date
CN105573861A CN105573861A (en) 2016-05-11
CN105573861B true CN105573861B (en) 2019-01-25

Family

ID=55884028

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510897346.0A CN105573861B (en) 2015-12-08 2015-12-08 The restorative procedure and device of distributed file system small file

Country Status (1)

Country Link
CN (1) CN105573861B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102063438A (en) * 2009-11-17 2011-05-18 阿里巴巴集团控股有限公司 Recovery method and device for damaged files
CN102567136A (en) * 2010-12-15 2012-07-11 研祥智能科技股份有限公司 Backup and recovery method of computer system, device and computer system

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7725764B2 (en) * 2006-08-04 2010-05-25 Tsx Inc. Failover system and method

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102063438A (en) * 2009-11-17 2011-05-18 阿里巴巴集团控股有限公司 Recovery method and device for damaged files
CN102567136A (en) * 2010-12-15 2012-07-11 研祥智能科技股份有限公司 Backup and recovery method of computer system, device and computer system

Also Published As

Publication number Publication date
CN105573861A (en) 2016-05-11

Similar Documents

Publication Publication Date Title
CN102629237B (en) The nonvolatile semiconductor memory searching method and the last written address
KR20080063466A (en) Flash memory management
US20150268879A1 (en) Memory management method, memory storage device and memory control circuit unit
US5473546A (en) Method for flattening hierarchical design descriptions
KR20090056966A (en) Logical super block mapping for nand flash memory
US9619430B2 (en) Active non-volatile memory post-processing
CN103038656B (en) System for concurrent test of semiconductor devices
JP3822044B2 (en) Design verification system, design verification methods and designs computer-storing verified programmable recording medium
JP2003502736A (en) Circuit simulations using dynamic partitioning and on-demand evaluation
US8386905B2 (en) Error correcting method, and memory controller and memory storage system using the same
US9146854B2 (en) Restoring storage devices based on flash memories and related circuit, system, and method
US20080134123A1 (en) Circuit-design supporting apparatus, circuit-design supporting method, computer product, and printed-circuit-board manufacturing method
US20080263500A1 (en) Circuit-design supporting apparatus, circuit-design supporting method, computer product, and printed-circuit-board manufacturing method
JP2010176180A (en) Storage system
US8510502B2 (en) Data writing method, and memory controller and memory storage apparatus using the same
US8224825B2 (en) Graph-processing techniques for a MapReduce engine
CN103488684B (en) Electric Reliability index cache data multithreading quick calculation based on
US7831944B2 (en) Circuit-design supporting apparatus, circuit-design supporting method, computer product, and printed-circuit-board manufacturing method
US8051400B2 (en) Modifying integrated circuit layout
CN103677654A (en) Method and electronic equipment for storing data
WO2015029104A1 (en) Vertically integrated system and firmware update method
KR20150118778A (en) Storage controller, storage device, storage system and method of operation of the storage controller
US7913220B2 (en) Coordinated-design supporting apparatus, coordinated-design supporting method, computer product, and printed-circuit-board manufacturing method
US8886884B2 (en) System and method for increasing read and write speeds of hybrid storage unit
US8812772B2 (en) Data merging method for non-volatile memory and controller and storage apparatus using the same

Legal Events

Date Code Title Description
C06 Publication
C10 Entry into substantive examination
GR01