CN107357869A - A kind of file operation method suitable for cloud storage - Google Patents
A kind of file operation method suitable for cloud storage Download PDFInfo
- Publication number
- CN107357869A CN107357869A CN201710532409.1A CN201710532409A CN107357869A CN 107357869 A CN107357869 A CN 107357869A CN 201710532409 A CN201710532409 A CN 201710532409A CN 107357869 A CN107357869 A CN 107357869A
- Authority
- CN
- China
- Prior art keywords
- file
- container
- coutinuous store
- space
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
- G06F16/1824—Distributed file systems implemented using Network-attached Storage [NAS] architecture
- G06F16/1827—Management specifically adapted to NAS
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention provides a kind of file operation method suitable for cloud storage, including：X class containers are configured, apply for that multiple capacity are Q initial Coutinuous store space to disk management space, each Coutinuous store space corresponds to the original allocation unit per class container；For arbitrary container Ri, by its corresponding Coutinuous store space WiIt is T to be divided into capacityiSeveral Coutinuous store data blocks, complementing part is used for retaining space；File is write based on container and read operation.Advantage is：The present invention realizes the maximum Coutinuous store of file, caused file fragmentation quantity is effectively reduced, so as to improve file read-write efficiency.When in applied to internet Dropbox system, its effect is especially apparent the present invention.
Description
Technical field
The invention belongs to file read-write operations technical field, and in particular to a kind of file operation side suitable for cloud storage
Method.
Background technology
Traditional file system, its smallest allocation particle is generally smaller, for example, being 512 bytes, 1K, 2K, 4K, 8K, 16K
With 32K etc., wherein, the most common are 4K, by taking representative value 4K as an example, i.e.,：Using 4K it is unit cutting as several by storage hard disk
Memory block.Main deficiency is existing for it：The file fragmentation quantity of some file often stored is big, thus causes file
Reading efficiency is low.
For convenience of the deficiency of traditional file systemses is understood, it is exemplified below：(1) assume for hard disk D, be initially empty；First
When beginning to hard disk deposit data, basic principle is：Ensure file Coutinuous store, so as to improve file reading efficiency.Thus, it is supposed that
The file that 100 file sizes are 100K is incorporated into hard disk D, so, the 1st file continuously occupies the 1st storage
Block is to the 25th memory block；2nd file continuously occupies the 26th memory block to the 50th memory block；3rd file is continuous
The 51st memory block is occupied to the 100th memory block, the rest may be inferred, and hard disk D preceding 2500 Coutinuous store blocks are stored into number
According to.(2) in subsequent process, file deletion action occurs, it is assumed that the 99th the 1st file, the 3rd file, the 5th file ... text
Part is deleted, now, the empty memory block for multiple dispersed placements that hard disk D occurs；(3) a certain moment, if necessary into hard disk D
When writing the file that size is 1000K, 1000K file cutting can only be 10 file units by file system, each file
Unit is stored into each empty memory block.It can be seen that 1000K file is made up of 10 file fragmentations.
That is, for traditional file system, often there is substantial amounts of file fragmentation, on the one hand, in write-in data
When, it is necessary to record the position of each file fragmentation, take substantial amounts of resource；On the other hand, when reading file, it is necessary to scatter-read
Take each file fragmentation, hence it is evident that there is the low deficiency of file reading efficiency.
Therefore, how effectively to solve above-mentioned deficiency possessed by traditional file systemses, improve file read-write efficiency, be current
Urgent thing.
The content of the invention
The defects of existing for prior art, the present invention provide a kind of file operation method suitable for cloud storage, can had
Effect solves the above problems.
The technical solution adopted by the present invention is as follows：
The present invention provides a kind of file operation method suitable for cloud storage, comprises the following steps：
Step 1, X class containers are configured, are designated as respectively：Container R1, container R2... container RX；X is natural number；If T1<T2<…<
TX；Then：Container R1The capacity of each file stored meets claimed below：0<File size≤T1；Container R2What is stored is every
The capacity of individual file meets claimed below：T1<File size≤T2；Container R3The capacity of each file stored meets following
It is required that：T2<File size≤T3, the rest may be inferred, container RXThe capacity of each file stored meets claimed below：TX-1<Text
Part capacity≤TX；
Step 2, apply for that multiple capacity are Q initial Coutinuous store space to disk management space, each is continuously deposited
Store up the corresponding original allocation unit per class container in space；Wherein, by container R1, container R2... container RXIt is corresponding respectively initially to connect
Continuous memory space is designated as：Coutinuous store space W1, Coutinuous store space W2... Coutinuous store space WX；Per class Container Management most
Small mikey is Q；
Step 3, Container Management unit is created, the Container Management unit is used to record by the way of the first allocation table often
The corresponding relation in individual container and corresponding Coutinuous store space；
For arbitrary container Ri, i ∈ (1,2 ... X), by its corresponding Coutinuous store space WiIt is T to be divided into capacityi's
Several Coutinuous store data blocks, complementing part are used for retaining space；
Step 4, space management unit in container is created, the space management unit is used for by the way of the second allocation table
The distribution condition of container internal storage space is recorded, specifically, second assignment table record is per the available company inside class container
The position of Coutinuous store data block can be used by renewing inside storage data number of blocks and container；
Step 5, when receiving file write request, file size value is obtained first, it is assumed that file size value is T0, sentence
Disconnected file size value T0Whether more than TXIf be not above, step 6- steps 8 are performed, and terminate flow；If it does, then
Step 9 is performed, and terminates flow；
Step 6, in X class containers, obtain and file size value T0Corresponding container Rj；Wherein, j ∈ (1,2 ... X)；
Step 7, space management unit in the container is searched, judges container RjAlso whether presence can use Coutinuous store data
Block, if it does, according to container RjThe second allocation table, obtain container RjAvailable Coutinuous store data block position；By container
RjSome available Coutinuous store data block to distribute to file size value be T0File, be T by file size value0File write
Enter to container RjSome available Coutinuous store data block in；Due to container RjThe capacity of available Coutinuous store data block be Tj,
TjMore than or equal to T0, therefore, file size value is T0File can be continuously stored in container RjAvailable Coutinuous store data block
In, realize the Coutinuous store of file；
If it does not, newly apply for Coutinuous store space of the capacity for Q from Container Management unit to disk management space, more
New first allocation table；Then, step 7 is re-executed；
Step 8, second allocation table is updated；
Step 9, T is calculated0Divided by TX, it is assumed that quotient a, remainder b；
With TXIt is T by file size value for segmentation unit0File from back to front or cutting from front to back, therefore, there are
It is T to a capacityXFile unit, and file size be b remaining file part；
It is T for a capacityXFile unit, each file unit is respectively written into container RXSome is available continuous
In data storage block, and update the second allocation table；
It is T to calculate a capacityXFile unit index value, the index value of each file unit is used to record this document
The storage location of unit；Capability value is c after a index value alignment；C+b is less than TX, by the method for step 6- steps 8, by capacity
Be worth for c+b index value and remaining file part storage to its closest to and more than or equal to c+b container in；
Step 10, in addition, pre-established file directory；The file directory be used for storage file title, file size and
The corresponding relation of file storage location；Therefore, after execution of step 8 or step 9, the file directory is updated；Wherein, for
Step 8, the position of corresponding Coutinuous store data block in the container that file storage location is stored by file；For step 9, text
Part storage location is：Corresponding Coutinuous store number in the container that capability value is stored by c+b index value and remaining file part
According to the position of block.
Preferably, in step 1, T1For 1Kb；T2For 2Kb；T3For 4Kb；T4For 8Kb；T5For 12Kb；T6For 16Kb ... TXFor
1M；Wherein, between 4K~1M, every the corresponding container of 4K.
Preferably, in step 2, Q values are 1G.
Preferably, in addition to file reading process, file reading process include：
Step 11, when receiving file read request, file name is carried in the file read request, searches institute
File directory is stated, obtains the file size and file storage location of this document；
Step 12, it is assumed that file size value is T0, judge file size value T0Whether more than TXIf be not above, root
According to the file storage location recorded in the file directory, the container and in this embodiment of storage this document can be directly obtained
Specific storage location；Therefore, the corresponding storage location of corresponding container is navigated to, you can read this document；
If it does, parse the file storage location recorded in the file directory, on the one hand, obtain in top of file
Hold；On the other hand, the index value for each file unit that file is included is obtained, further according to index value, it is each to navigate to storage
The container of file unit and specific storage location in this embodiment, so as to read each file unit；By what is read
Each file unit and top of file content is spliced in order, that is, reads complete file.
A kind of file operation method suitable for cloud storage provided by the invention has advantages below：
The present invention realizes the maximum Coutinuous store of file, effectively reduces caused file fragmentation quantity, so as to
Improve file read-write efficiency.When in applied to internet Dropbox system, its effect is especially apparent the present invention.
Brief description of the drawings
Fig. 1 is that file provided by the invention writes flow chart；
Fig. 2 is that file provided by the invention reads flow chart.
Embodiment
In order that technical problem solved by the invention, technical scheme and beneficial effect are more clearly understood, below in conjunction with
Drawings and Examples, the present invention will be described in further detail.It should be appreciated that specific embodiment described herein only to
The present invention is explained, is not intended to limit the present invention.
For convenience of understanding the present invention, the central scope of the present invention is introduced first：
Assuming that configuring 258 class containers altogether, it is respectively：1K containers, 2K containers, 4K containers, 8K containers, 12K containers ... 1M hold
Device；Initial Coutinuous store space per class container allocation is 1G.
By taking 8K containers as an example, the file size of its each file for being used to store is：More than 4K and it is less than or equal to 8K；Assuming that
8K container original allocations 1G Coutinuous store space, 1G Coutinuous store space are divided into the Coutinuous store that multiple sizes are 8K
Data block, complementing part are used for retaining space；It is the Coutinuous store data block using 8K as memory cell that is, 8K containers
, in order, it can be referred to as：1st memory cell, the 2nd memory cell, the 3rd memory cell etc., each storage are single
Member takes 8K.Therefore, for 5K files, when being written into some memory cell, there can be 3K free space, the free time is empty
Between will not be reallocated to alternative document, can be idle condition always.For example, it is assumed that the 1st memory cell is written into 5K files, then
3K free spaces be present in the 1st memory cell；When needing to write 6K another file again, 6K file directly writes to
2 memory cell, the 3K free spaces without taking the 1st memory cell；That is, each memory cell allows to exist
A small amount of free space, still, by 8K containers allow the maximum capacity of the file of write-in to be 8K, so, as long as allowing to write
Enter the file to 8K containers, this document must be Coutinuous store in 8K containers, so as to ensure that its read-write efficiency.
It should be noted that in the present invention, because the minimum value for allowing to write file to each container is defined, institute
With, it will be understood that the quantity for the free space that each container occurs is considerably less, for example, for 8K containers, it allows write-in
The minimum value of file still above 4K, so, the free space that each memory cell occurs necessarily is less than 4K.Therefore, with realization
The effect that Coutinuous store is brought is compared, and the unfavorable factor of the free space sacrificed can be neglected.
When setting (vessel) class in the following manner：1K containers, 2K containers, 4K containers, 8K containers, 12K containers ... 1M hold
Device, it will be understood that the file if necessary to write-in is less than or equal to 1M, can directly write in container corresponding with file size,
So as to realize that the file of write-in necessarily takes Coutinuous store space.
And the file write if desired is more than 1M, for example, being 2054K, then need to carry out at cutting 2054K files
Reason, it is syncopated as 2 1M file unit, remainder 6K；Then, 2 1M file unit is respectively stored into 1M containers,
Realize Coutinuous store of each 1M file unit in 1M containers；Then, for realize subsequently the reading to file, it is necessary to obtain
The index value of 2 1M file unit, index value are the storage address of each file unit, 8 bytes, are 1K after alignment, so
Afterwards, 1K indexes add remaining 6K residues file, are 7K file sections, store in 8K containers, ensure that 7K file sections hold in 8K
Coutinuous store in device.
In the present invention, for the file more than 1M, the design feature of above-mentioned file write-in flow is：
(1) to reduce the file unit quantity obtained after file cutting as far as possible, cutting is carried out by segmentation unit of 1M；Its
In, 1M refers to：In all kinds of containers, the value of maximum Coutinuous store data block, that is, Coutinuous store data block corresponding to 1M containers
Value；So as at utmost ensure file Coutinuous store；
(2) after file cutting, 1M containers are arrived in each file unit storage, ensure that the Coutinuous store of each file unit；
(3) after file cutting, the index of each file unit and file remainder are arrived suitable as an entirety, storage
In the container answered, further reduce the quantity of scattered file, realize maximum file Coutinuous store.
Therefore, the present invention is designed and improved to file system from multiple angles, so as to farthest realize file
Coutinuous store, improve file read-write efficiency.
In addition, the file read/write method that the present invention is realized based on the design of container, has further the advantage that：
Traditional file systemses, smallest allocation particle is usually 4K,, can wave when storage is less than 4K file by taking 4K as an example
Take certain storage resource, for example, when being stored in 2K file, 4-2=2K storage resource can be wasted.And in the present invention, it is right
The type of container can flexible design, therefore, in all kinds of containers, the capacity of Coutinuous store data block can flexible design, when design
When the minimum capacity of Coutinuous store data block is 1K or 2K, i.e.,：When being designed with 1K containers or 2K containers, the storage resource of waste
At most not over 1K or 2K, this improves the utilization rate of storage resource, reduce the amount of the storage resource of waste；In addition,
When handling a small amount of small documents, the memory module based on container of the invention designed, make storage file more compact, reading efficiency meeting
It is substantially better than traditional file systemses.
Based on above-mentioned design, a kind of file operation method suitable for cloud storage provided by the invention, with reference to figure 1, including
Following steps：
Step 1, X class containers are configured, are designated as respectively：Container R1, container R2... container RX；X is natural number；If T1<T2<…<
TX；Then：Container R1The capacity of each file stored meets claimed below：0<File size≤T1；Container R2What is stored is every
The capacity of individual file meets claimed below：T1<File size≤T2；Container R3The capacity of each file stored meets following
It is required that：T2<File size≤T3, the rest may be inferred, container RXThe capacity of each file stored meets claimed below：TX-1<Text
Part capacity≤TX；
In practical application, the quantity of the Container Type of configuration and the file maximum capacity that each container is stored, equal root
Flexibly set according to actual demand.As actually conventional mode, mode arranged below can be used：258 class containers, T are configured altogether1
For 1Kb；T2For 2Kb；T3For 4Kb；T4For 8Kb；T5For 12Kb；T6For 16Kb ... TXFor 1M；Wherein, between 4K~1M, every
The corresponding container of 4K.
Step 2, apply for that multiple capacity are Q initial Coutinuous store space to disk management space, each is continuously deposited
Store up the corresponding original allocation unit per class container in space；Wherein, by container R1, container R2... container RXIt is corresponding respectively initially to connect
Continuous memory space is designated as：Coutinuous store space W1, Coutinuous store space W2... Coutinuous store space WX；Per class Container Management most
Small mikey is Q；
Q concrete numerical value is set also according to actual demand, it is generally the case that Q values are 1G.
Step 3, Container Management unit is created, the Container Management unit is used to record by the way of the first allocation table often
The corresponding relation in individual container and corresponding Coutinuous store space；
For arbitrary container Ri, i ∈ (1,2 ... X), by its corresponding Coutinuous store space WiIt is T to be divided into capacityi's
Several Coutinuous store data blocks, complementing part are used for retaining space；
Step 4, space management unit in container is created, the space management unit is used for by the way of the second allocation table
The distribution condition of container internal storage space is recorded, specifically, second assignment table record is per the available company inside class container
The position of Coutinuous store data block can be used by renewing inside storage data number of blocks and container；
Step 5, when receiving file write request, file size value is obtained first, it is assumed that file size value is T0, sentence
Disconnected file size value T0Whether more than TXIf be not above, step 6- steps 8 are performed, and terminate flow；If it does, then
Step 9 is performed, and terminates flow；
Step 6, in X class containers, obtain and file size value T0Corresponding container Rj；Wherein, j ∈ (1,2 ... X)；
Step 7, space management unit in the container is searched, judges container RjAlso whether presence can use Coutinuous store data
Block, if it does, according to container RjThe second allocation table, obtain container RjAvailable Coutinuous store data block position；By container
RjSome available Coutinuous store data block to distribute to file size value be T0File, be T by file size value0File write
Enter to container RjSome available Coutinuous store data block in；Due to container RjThe capacity of available Coutinuous store data block be Tj,
TjMore than or equal to T0, therefore, file size value is T0File can be continuously stored in container RjAvailable Coutinuous store data block
In, realize the Coutinuous store of file；
If it does not, newly apply for Coutinuous store space of the capacity for Q from Container Management unit to disk management space, more
New first allocation table；Then, step 7 is re-executed；
Step 8, second allocation table is updated；
Step 9, T is calculated0Divided by TX, it is assumed that quotient a, remainder b；
With TXIt is T by file size value for segmentation unit0File from back to front or cutting from front to back, therefore, there are
It is T to a capacityXFile unit, and file size be b remaining file part；
It is T for a capacityXFile unit, each file unit is respectively written into container RXSome is available continuous
In data storage block, and update the second allocation table；
It is T to calculate a capacityXFile unit index value, the index value of each file unit is used to record this document
The storage location of unit；Capability value is c after a index value alignment；C+b is less than TX, by the method for step 6- steps 8, by capacity
Be worth for c+b index value and remaining file part storage to its closest to and more than or equal to c+b container in；
Step 10, in addition, pre-established file directory；The file directory be used for storage file title, file size and
The corresponding relation of file storage location；Therefore, after execution of step 8 or step 9, the file directory is updated；Wherein, for
Step 8, the position of corresponding Coutinuous store data block in the container that file storage location is stored by file；For step 9, text
Part storage location is：Corresponding Coutinuous store number in the container that capability value is stored by c+b index value and remaining file part
According to the position of block.
Above-mentioned steps can be summarized simply as follows：
Assuming that configure 1K altogether, 2K, 4K, 8K, 12K, 16K, 20K ... 996K, 1M totally 258 containers.
File if necessary to write-in is then directly distributed most suitable container to file, directly referred to using node less than 1M
To the dispensing position of corresponding container.
File if necessary to write-in, then by file cutting, obtains integer 1M file units and less than 1M more than 1M
File remainder, by (1M indexes+file remainder+integer 1M file units) less than 1M composing document body.Its
In, 1M indexes align by 1M, the index ID of one 1M block of every 32 expressions.If (the file remainder of 1M indexes+less than 1M
Point) it is less than 1M, corresponding container is stored directly in, node is pointing directly at its dispensing position；If (the file of 1M indexes+less than 1M
Remainder) it is more than or equal to 1M, 1M indexes increase a directory entry, then complete 1M regions are moved into, and remainder is directly deposited
Storage is in corresponding container.
For convenience of understanding file write step provided by the invention, the implementation of three file write-ins is set forth below
Example：
Present embodiment assumes that configuring 258 class containers altogether, it is respectively：1K containers, 2K containers, 4K containers, 8K containers, 12K hold
Device ... 1M containers；Initial Coutinuous store space per class container allocation is 1G.
Example 1：Need the file that one size of write-in is 799K.
Because file size is less than 1M, directly applies for space in most suitable 800K containers, 800K containers, be by 800K
Cutting is carried out for unit, can be that this file distributes 800K containers if it is idle to find 200 units (800K sizes)
In the 200th unit, more new node, write data.
Example 2：Need the file that one size of write-in is 20981K.
Because file size is more than 1M, cutting is carried out to file, by+20 1M of 501K files remainder (20480K) texts
Part unit forms, and 20 1M index size is 20*4=80 bytes, is 1K after alignment, plus the 501K of file remainder,
For 502K (being less than 1M), so, for whole file by (1K indexes, 501K file remainders, 20 1M compositions), preceding two parts are straight
It is connected in 504K container and applies for space, it is assumed that is the 300th unit, then the position of log file is 300.Below 20
Individual 1M file units, by 1M container allocations space, you can realize that data write.
Example 3：Need to write the file that size is 2044K.
Because file is more than 1M, so by 1M indexes (index of 1 M totally 4 bytes, spatial alignment take 1K)+1020K texts
+ 1 1M file unit of part remainder, because index+file remainder is also 1M alignment space, so, more
New be 2 1M containers, and top of file is with 1K byte representation 1M indexes (actual with 8 bytes), first 1M appearance described in node
Device position.
It is real with reference to figure 2, in addition to file reading process, file reading process on the basic principle of above-mentioned file write-in
Border is the inverse process of file writing process, is comprised the following steps：
Step 11, when receiving file read request, file name is carried in the file read request, searches institute
File directory is stated, obtains the file size and file storage location of this document；
Step 12, it is assumed that file size value is T0, judge file size value T0Whether more than TXIf be not above, root
According to the file storage location recorded in the file directory, the container and in this embodiment of storage this document can be directly obtained
Specific storage location；Therefore, the corresponding storage location of corresponding container is navigated to, you can read this document；
If it does, parse the file storage location recorded in the file directory, on the one hand, obtain in top of file
Hold；On the other hand, the index value for each file unit that file is included is obtained, further according to index value, it is each to navigate to storage
The container of file unit and specific storage location in this embodiment, so as to read each file unit；By what is read
Each file unit and top of file content is spliced in order, that is, reads complete file.
The embodiment of file reading is set forth below：
Need to read the file that a size is 2059K：
During write-in：
Because file is more than 1M, it is made up of+2 1M file units (2048K) of 11K files remainder, 2 1M file lists
Index (the real storage location for being used to record 2 1M file units) size of member is 2*4=8 bytes, and it is 1K to round up after aliging,
It is 12K (being less than 1M) plus the 11K of file remainder, therefore, index and file remainder are stored directly in 12K appearance
In device, two 1M file units are stored in two Coutinuous store data blocks of 1M containers, therefore, totally three continuous segments.
After these three continuous segments are fully completed write-in, by filename, file size, the time, 12K containers Data Position
Write Deng attribute information in file directory, file directory can use following form：
| File name | Size | Time | Position | Other |
| B.TXT | 2069K | On January 1st, 2017 | 12K container X positions | ··· |
Therefore, when carrying out file reading, by file directory, file size, time, the data bit of 12K containers are read
The information such as put.The 12K contents in 12K containers are read again, obtain 2 1M of 8 bytes position, and 11K file remainder
Point.Again by 2 1M position, file content is read in 1M containers.
In summary, a kind of file operation method suitable for cloud storage provided by the invention has advantages below：
The present invention realizes the maximum Coutinuous store of file, effectively reduces caused file fragmentation quantity, so as to
Improve file read-write efficiency.When in applied to internet Dropbox system, its effect is especially apparent the present invention.
Described above is only the preferred embodiment of the present invention, it is noted that for the ordinary skill people of the art
For member, under the premise without departing from the principles of the invention, some improvements and modifications can also be made, these improvements and modifications also should
Depending on protection scope of the present invention.
Claims (4)
1. a kind of file operation method suitable for cloud storage, it is characterised in that comprise the following steps：
Step 1, X class containers are configured, are designated as respectively：Container R1, container R2... container RX；X is natural number；If T1<T2<…<TX；
Then：Container R1The capacity of each file stored meets claimed below：0<File size≤T1；Container R2What is stored is each
The capacity of file meets claimed below：T1<File size≤T2；Container R3The capacity of each file stored meets following want
Ask：T2<File size≤T3, the rest may be inferred, container RXThe capacity of each file stored meets claimed below：TX-1<File
Capacity≤TX；
Step 2, apply for that multiple capacity are Q initial Coutinuous store space to disk management space, each Coutinuous store is empty
Between the corresponding original allocation unit per class container；Wherein, by container R1, container R2... container RXIt is corresponding respectively initially continuously to deposit
Storage space is designated as：Coutinuous store space W1, Coutinuous store space W2... Coutinuous store space WX；Minimum per class Container Management is empty
Between unit be Q；
Step 3, Container Management unit is created, the Container Management unit is used to record each appearance by the way of the first allocation table
The corresponding relation in device and corresponding Coutinuous store space；
For arbitrary container Ri, i ∈ (1,2 ... X), by its corresponding Coutinuous store space WiIt is T to be divided into capacityiIt is some
Individual Coutinuous store data block, complementing part are used for retaining space；
Step 4, space management unit in container is created, the space management unit is used to record by the way of the second allocation table
The distribution condition of container internal storage space, specifically, second assignment table record per class container inside available continuously deposit
The position of Coutinuous store data block can be used by storing up inside data number of blocks and container；
Step 5, when receiving file write request, file size value is obtained first, it is assumed that file size value is T0, judge text
Part capability value T0Whether more than TXIf be not above, step 6- steps 8 are performed, and terminate flow；If it does, then perform
Step 9, and flow is terminated；
Step 6, in X class containers, obtain and file size value T0Corresponding container Rj；Wherein, j ∈ (1,2 ... X)；
Step 7, space management unit in the container is searched, judges container RjAlso whether presence can use Coutinuous store data block, such as
Fruit is present, according to container RjThe second allocation table, obtain container RjAvailable Coutinuous store data block position；By container Rj's
It is T that some available Coutinuous store data block, which distributes to file size value,0File, be T by file size value0File be written to
Container RjSome available Coutinuous store data block in；Due to container RjThe capacity of available Coutinuous store data block be Tj, TjGreatly
In equal to T0, therefore, file size value is T0File can be continuously stored in container RjAvailable Coutinuous store data block in, it is real
The Coutinuous store of existing file；
If it does not, newly applying for Coutinuous store space of the capacity for Q from Container Management unit to disk management space, institute is updated
State the first allocation table；Then, step 7 is re-executed；
Step 8, second allocation table is updated；
Step 9, T is calculated0Divided by TX, it is assumed that quotient a, remainder b；
With TXIt is T by file size value for segmentation unit0File from back to front or cutting from front to back, therefore, be obtained a
Capacity is TXFile unit, and file size be b remaining file part；
It is T for a capacityXFile unit, each file unit is respectively written into container RXSome available Coutinuous store
In data block, and update the second allocation table；
It is T to calculate a capacityXFile unit index value, the index value of each file unit is used to record this document unit
Storage location；Capability value is c after a index value alignment；C+b is less than TX, it is c+b by capability value by the method for step 6- steps 8
Index value and remaining file part storage to its closest to and more than or equal to c+b container in；
Step 10, in addition, pre-established file directory；The file directory is used for storage file title, file size and file
The corresponding relation of storage location；Therefore, after execution of step 8 or step 9, the file directory is updated；Wherein, for step
8, the position of corresponding Coutinuous store data block in the container that file storage location is stored by file；For step 9, file is deposited
Storage space is set to：Corresponding Coutinuous store data block in the container that capability value is stored by c+b index value and remaining file part
Position.
2. the file operation method according to claim 1 suitable for cloud storage, it is characterised in that in step 1, T1For
1Kb；T2For 2Kb；T3For 4Kb；T4For 8Kb；T5For 12Kb；T6For 16Kb ... TXFor 1M；Wherein, between 4K~1M, every 4K
A corresponding container.
3. the file operation method according to claim 1 suitable for cloud storage, it is characterised in that in step 2, Q values are
1G。
4. the file operation method according to claim 1 suitable for cloud storage, it is characterised in that also read including file
Process, file reading process include：
Step 11, when receiving file read request, file name is carried in the file read request, searches the text
Part catalogue, obtain the file size and file storage location of this document；
Step 12, it is assumed that file size value is T0, judge file size value T0Whether more than TXIf be not above, according to institute
The file storage location recorded in file directory is stated, the container and tool in this embodiment of storage this document can be directly obtained
Body storage location；Therefore, the corresponding storage location of corresponding container is navigated to, you can read this document；
If it does, parse the file storage location recorded in the file directory, on the one hand, obtain top of file content；
On the other hand, the index value for each file unit that file is included is obtained, further according to index value, navigates to each file of storage
The container of unit and specific storage location in this embodiment, so as to read each file unit；It is each by what is read
File unit and top of file content are spliced in order, that is, read complete file.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710532409.1A CN107357869A (en) | 2017-07-03 | 2017-07-03 | A kind of file operation method suitable for cloud storage |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710532409.1A CN107357869A (en) | 2017-07-03 | 2017-07-03 | A kind of file operation method suitable for cloud storage |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107357869A true CN107357869A (en) | 2017-11-17 |
Family
ID=60292013
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710532409.1A Pending CN107357869A (en) | 2017-07-03 | 2017-07-03 | A kind of file operation method suitable for cloud storage |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107357869A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108388468A (en) * | 2018-01-26 | 2018-08-10 | 杨立群 | A kind of Flase File System dispatching method and device |
| CN114063907A (en) * | 2021-10-20 | 2022-02-18 | 郑州云海信息技术有限公司 | Storage space allocation method, system, storage medium and equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5129088A (en) * | 1987-11-30 | 1992-07-07 | International Business Machines Corporation | Data processing method to create virtual disks from non-contiguous groups of logically contiguous addressable blocks of direct access storage device |
| CN101446976A (en) * | 2008-12-26 | 2009-06-03 | 中兴通讯股份有限公司 | File storage method in distributed file system |
| CN102096722A (en) * | 2011-03-21 | 2011-06-15 | 成都市华为赛门铁克科技有限公司 | File storage method and device |
| CN102934094A (en) * | 2010-06-16 | 2013-02-13 | 微软公司 | Hierarchical allocation for file system storage device |
| CN103577470A (en) * | 2012-08-03 | 2014-02-12 | 上海交通大学 | File system and method for improving performances of web server |
| CN104050251A (en) * | 2014-06-11 | 2014-09-17 | 深圳市茁壮网络股份有限公司 | File management method and management system |
-
2017
- 2017-07-03 CN CN201710532409.1A patent/CN107357869A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5129088A (en) * | 1987-11-30 | 1992-07-07 | International Business Machines Corporation | Data processing method to create virtual disks from non-contiguous groups of logically contiguous addressable blocks of direct access storage device |
| CN101446976A (en) * | 2008-12-26 | 2009-06-03 | 中兴通讯股份有限公司 | File storage method in distributed file system |
| CN102934094A (en) * | 2010-06-16 | 2013-02-13 | 微软公司 | Hierarchical allocation for file system storage device |
| CN102096722A (en) * | 2011-03-21 | 2011-06-15 | 成都市华为赛门铁克科技有限公司 | File storage method and device |
| CN103577470A (en) * | 2012-08-03 | 2014-02-12 | 上海交通大学 | File system and method for improving performances of web server |
| CN104050251A (en) * | 2014-06-11 | 2014-09-17 | 深圳市茁壮网络股份有限公司 | File management method and management system |
Non-Patent Citations (2)
| Title |
|---|
| 丁凯: "高性能小文件应用系统Sfs设计与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑（月刊）2013年》 * |
| 陈卫卫、吴海佳、胥光辉: "分布式存储中文件分割的最优化模型", 《解放军理工大学学报（自然科学版）》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108388468A (en) * | 2018-01-26 | 2018-08-10 | 杨立群 | A kind of Flase File System dispatching method and device |
| CN108388468B (en) * | 2018-01-26 | 2022-02-22 | 杨立群 | Fast file system scheduling method and device |
| CN114063907A (en) * | 2021-10-20 | 2022-02-18 | 郑州云海信息技术有限公司 | Storage space allocation method, system, storage medium and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101241476B (en) | Dummy storage system and method | |
| CN102662992B (en) | Method and device for storing and accessing massive small files | |
| CN102255962B (en) | Distributive storage method, device and system | |
| CN101996217B (en) | Method for storing data and memory device thereof | |
| CN101997918B (en) | Method for allocating mass storage resources according to needs in heterogeneous SAN (Storage Area Network) environment | |
| CN100583832C (en) | Data management method and system | |
| CN102629247B (en) | Method, device and system for data processing | |
| CN103473239B (en) | A kind of data of non relational database update method and device | |
| CN108519862A (en) | Storage method, device, system and the storage medium of block catenary system | |
| CN101504670A (en) | Data operation method, system, client terminal and data server | |
| CN103761053B (en) | A kind of data processing method and device | |
| CN109766312A (en) | A kind of block chain storage method, system, device and computer readable storage medium | |
| CN103092927B (en) | File rapid read-write method under a kind of distributed environment | |
| CN103605726A (en) | Method and system for accessing small files, control node and storage node | |
| WO2012126317A1 (en) | File storage method and device thereof | |
| CN105378687A (en) | File accessing method and related device | |
| CN107016100A (en) | A kind of metadata management method based on Nonvolatile memory file system | |
| CN103838853A (en) | Mixed file system based on different storage media | |
| CN100424699C (en) | Attribute extensible object file system | |
| CN102915340A (en) | Expanded B+ tree-based object file system | |
| CN103514249A (en) | Method and system for automatic data reduction and storage device | |
| CN101968795A (en) | Cache method for file system with changeable data block length | |
| CN105677904B (en) | Small documents storage method and device based on distributed file system | |
| CN101571869A (en) | File memory and read method of smart card and device thereof | |
| CN108628541A (en) | A kind of method, apparatus and storage system of file storage |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20171117 |