CN106469120A - Scrap cleaning method, device and equipment - Google Patents
Scrap cleaning method, device and equipment Download PDFInfo
- Publication number
- CN106469120A CN106469120A CN201610776743.7A CN201610776743A CN106469120A CN 106469120 A CN106469120 A CN 106469120A CN 201610776743 A CN201610776743 A CN 201610776743A CN 106469120 A CN106469120 A CN 106469120A
- Authority
- CN
- China
- Prior art keywords
- data block
- chunk
- metadata
- value
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
- G06F12/023—Free address space management
- G06F12/0253—Garbage collection, i.e. reclamation of unreferenced memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/06—Addressing a physical block of locations, e.g. base addressing, module addressing, memory dedication
- G06F12/0646—Configuration or reconfiguration
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A kind of scrap cleaning method, device and equipment.Method includes：If the valid data occupancy of a chunk is less than pre-determined threshold, the index of metadata of data block and the data block of being stored with wherein this chunk, then acquisition index of metadata includes FID and the positional information of this data block；After determining that according to positional information data block is valid data block, it is data block distribution target chunk, data block is write target chunk；According to the metadata information of the affiliated file of FID searching data block, positional information in metadata information is updated.By this index of metadata can the promptly affiliated file of locking data block metadata information it is achieved that the purpose that just positional information of this data block in metadata information can be modified without ergodic process, improve arrangement efficiency.
Description
Technical field
The present invention relates to technical field of data storage, particularly to a kind of scrap cleaning method, device and equipment.
Background technology
It is known that disk is in use for some time, with the write of repeatedly file and deletion, continuously empty in disk
Free space is less and less, so that identical file can not be stored entirely in continuous free space, but is dispersed into multiple
Fragment, is saved in the different free spaces of disk.As shown in figure 1, after the file for 32KB and 16KB for the size is deleted, being formed
The free time of 48KB, but the idle storage space of this 48KB is simultaneously discontinuous.If the file that size is 48KB will newly be write,
Then need to be split into 2 pieces, be respectively written in two spaces.So, subsequently when reading the file of this 48KB, just need
Go to read to two different spaces respectively.For reading to single space, increased magnetic head moves back and forth number of times,
Readwrite performance is caused drastically to decline.At present in order to avoid above-mentioned the occurrence of, generally individually can periodically carry out fragment to disk whole
Reason.
During prior art conducting disk defragmentation, file system can travel through the B+ being formed by the metadata of local file
Tree construction, after B+ tree construction traverses a file, each data block place chunk of being comprised according to this document
The valid data occupancy of (burst), to judge whether to disk fragmentses arrangement, carries out disk fragmentses arrangement, then if necessary
By the data block migration of storage in this chunk to target chunk, and the positional information of this data block in respective meta-data is carried out
Update.Wherein, B+ tree construction is as shown in Fig. 2 file system can distribute a FID (File for each file of storage
Identifier, file identification), afterwards the metadata of this document is inserted by B+ tree construction according to this FID.Leve17 in fig. 2
It is used for preserving the metadata of file, the positional information of all data blocks that this metadata includes FID and file is comprised, wherein
The positional information of data block points to the data block in Leve18.And chunk is file system when carrying out space distribution by disk
Divide out according to fixed size, such as one chunk of 4MB size.Under normal circumstances, two continuous chunk in logic,
Its position on physical disk is also adjacent.Wherein, after file system traverses a file in B+ tree construction, obtaining should
The valid data occupancy of each data block place chunk that file is comprised；If a data block place chunk's has
Effect data occupancy is less than pre-determined threshold (such as 20%), then need to carry out disk fragmentses arrangement, and concrete arrangement process can be as Fig. 3 institute
Show.The valid data occupancy of data block 1 place chunk is less than pre-determined threshold in figure 3, then data block 1 is write target
Chunk, is then updated to the positional information of data block 1；For data block 2, the valid data occupancy of its place chunk
Higher than pre-determined threshold, then it is not required to be arranged.So assume only have an idle chunk shown in Fig. 4 before being arranged,
After being arranged by the way disk fragmentses are focused on a chunk, just there are three idle chunk.
At least there is problems with prior art：
Above-mentioned scrap cleaning method needs to travel through the B+ tree construction of whole metadata, when disk occupancy is high, this whole
The efficiency of reason mode is very low.It is assumed that file size average out to 32KB such as a 4T disk, then this disk is writeable
Enter 120,000 small documents；If only 1,000 small documents need to be migrated, that is, store at least one of this 1,000 small documents
The valid data occupancy of chunk be less than pre-determined threshold, then migrate this 1,000 files when, if due to directly travel through above-mentioned extremely
A few chunk is simultaneously migrated it is impossible to be obtained the metadata information of this 1,000 small documents, and then metadata information is carried out
Update, therefore also need to take the mode of the B+ tree construction traveling through whole metadata, read the metadata of whole 120,000 small documents,
Judged with small documents one by one and processed, cause above-mentioned defragmentation mode relatively complicated and complicated, the plenty of time can be consumed.
Content of the invention
Embodiments provide a kind of scrap cleaning method, device and equipment.Described technical scheme is as follows：
A kind of first aspect, there is provided scrap cleaning method, methods described includes：
Obtain valid data occupancy, the data block that is stored with described chunk and the described data block of chunk in memorizer
Index of metadata；
If the valid data occupancy of described chunk is less than pre-determined threshold, obtain the index of metadata of described data block,
Wherein, in described index of metadata, the FID including the affiliated file of described data block and the original position information of described data block；
After the original position information according to described data block determines that described data block is valid data block, by described data block
As data block to be migrated, it is described data block distribution target chunk, by described data block migration to described target chunk；
According to described FID, search the metadata information of the affiliated file of described data block, will comprise in described metadata information
Described data block migration before described original position information updating be described data block migration after destination locations information.
In the first possible implementation of first aspect, after by described data block migration to target chunk,
The index of metadata of described data block also can be write in target chunk, so that when the significant figure of target chunk in subsequent process
When being less than pre-determined threshold according to occupancy, conveniently according to the index of metadata being written in target chunk again in data block migration
Afterwards above-mentioned metadata information is updated, detailed process is as follows：
Described destination locations information and described FID are write described target as the index of metadata of described data block
In chunk.Further, based on TLV (Type Length Value, type lengths values) form, in described target chunk
The described destination locations information of middle write and described FID, obtain the index of metadata of described data block, wherein, described purpose position
Confidence ceases the purpose storage address after the length value including described data block and described data block migration.
In conjunction with a first aspect, in the possible implementation of the second of first aspect, burst in described acquisition memorizer
Before the valid data occupancy of chunk, methods described also includes：
One bitmap file is set for described chunk, each of institute's bitmap file value bit is used for identifying described
The occupied situation in corresponding space in chunk；
Wherein, when the value of a value bit in institute's bitmap file is first constant, characterize described value bit instruction
Space taken by valid data；When the value of a value bit in institute's bitmap file is second constant, characterize described
The space of value bit instruction is not taken by valid data.
In conjunction with the possible implementation of the second of first aspect, in the third possible implementation of first aspect
In, the described valid data occupancy obtaining burst chunk in memorizer, including：
In statistics institute bitmap file, value is the value bit number of described first constant；
Obtain value bit total number in institute's bitmap file；
Calculate the ratio between the value bit number that described value is described first constant and described value bit total number, obtain
Valid data occupancy to described chunk.
In conjunction with the possible implementation of the second of first aspect, in the 4th kind of possible implementation of first aspect
In, described original position information include described data block migration before former storage address and described data block length value, described
Method also includes：
According to described former storage address and described length value, the bitmap file of described chunk determines described data block
Take up room at least one value bit corresponding；
If the value of each of at least one value bit described value bit is described first constant it is determined that described number
It is valid data block according to block.
In conjunction with second to the 4th kind of possible implementation, in the 5th kind of possible implementation of first aspect,
Methods described also includes：
If the valid data occupancy of described chunk is less than pre-determined threshold, by each of described chunk significant figure
All write corresponding target chunk of distribution according to block after, respectively each of the bitmap file of described chunk value bit is entered as
Described second constant.
In conjunction with a first aspect, in the 6th kind of possible implementation of first aspect, described according to described file identification,
Search the metadata information of the affiliated file of described data block, including：
Search the metadata node of the described FID that is stored with metadata tree structure, store in described metadata node
The metadata information of the affiliated file of described data block, at least includes the FID of described file and described literary composition in described metadata information
The positional information of each data block that part comprises；
Wherein, first number of locally stored all files is have recorded in described metadata tree structure with metadata node form
It is believed that breath.
A kind of second aspect, there is provided defragmentation device, described device includes：
First acquisition module, for obtaining the valid data occupancy of chunk in memorizer, is stored with described chunk
Data block and the index of metadata of described data block；
Second acquisition module, if the valid data occupancy for described chunk is less than pre-determined threshold, obtains described number
According to the index of metadata of block, wherein, in described index of metadata, the FID including the affiliated file of described data block and described data
The original position information of block；
In the original position information according to described data block, distribute module, for determining that described data block is valid data block
Afterwards, using described data block as data block to be migrated, it is described data block distribution target chunk, by described data block migration
To described target chunk；
Searching modul, for according to described FID, searching the metadata information of the affiliated file of described data block；
Update module, for the described original position information before the described data block migration that will comprise in described metadata information
It is updated to the destination locations information after described data block migration.
In the first possible implementation of second aspect, described device also includes：
Writing module, for based on TLV form, writing described destination locations information and institute in described target chunk
State FID, obtain the index of metadata of described data block, wherein, described destination locations information include described data block length value,
And the purpose storage address after described data block migration.
In conjunction with second aspect, in the possible implementation of the second of second aspect, described device also includes：
Setup module, for arranging a bitmap file, each of institute's bitmap file value bit for described chunk
For identifying the occupied situation in corresponding space in described chunk；
Wherein, when the value of a value bit in institute's bitmap file is first constant, characterize described value bit instruction
Space taken by valid data；When the value of a value bit in institute's bitmap file is second constant, characterize described
The space of value bit instruction is not taken by valid data.
In conjunction with the possible implementation of the second of second aspect, in the third possible implementation of second aspect
In, described first acquisition module, in the bitmap file for counting, value is the value bit number of described first constant；Obtain
Value bit total number in institute's bitmap file；Calculate the value bit number that described value is described first constant and described value bit
Ratio between total number, obtains the valid data occupancy of described chunk.
In conjunction with the possible implementation of the second of second aspect, in the 4th kind of possible implementation of second aspect
In, described original position information include described data block migration before former storage address and described data block length value, described
Device also includes：
Determining module, for according to described former storage address and described length value, true in the bitmap file of described chunk
Fixed described data block takes up room at least one value bit corresponding；If each of at least one value bit described value bit
Value is described first constant it is determined that described data block is valid data block.
In conjunction with second to the 4th kind of possible implementation, in the 5th kind of possible implementation of second aspect,
Described device also includes：
Assignment module, if the valid data occupancy for described chunk is less than pre-determined threshold, by described chunk
Each of after valid data block all writes corresponding target chunk of distribution, respectively will be each in the bitmap file of described chunk
Individual value bit is entered as described second constant.
In conjunction with second aspect, in the 6th kind of possible implementation of second aspect, described searching modul, in unit
Search the metadata node of the described FID that is stored with data tree structure, in described metadata node, store described data block institute
Belong to the metadata information of file, at least include the FID of described file in described metadata information and that described file comprises is each
The positional information of individual data block；
Wherein, first number of locally stored all files is have recorded in described metadata tree structure with metadata node form
It is believed that breath.
A kind of third aspect, there is provided De-fragmentation apparatus, described equipment includes：Internal memory and processor, described internal memory is deposited
Contain program code, described processor is used for calling described program code, execution is following to be operated：
Obtain valid data occupancy, the data block that is stored with described chunk and the described data block of chunk in memorizer
Index of metadata；
If the valid data occupancy of described chunk is less than pre-determined threshold, obtain the index of metadata of described data block,
Wherein, in described index of metadata, the FID including the affiliated file of described data block and the original position information of described data block；
After the original position information according to described data block determines that described data block is valid data block, by described data block
As data block to be migrated, it is described data block distribution target chunk, by described data block migration to described target chunk；
According to described FID, search the metadata information of the affiliated file of described data block, will comprise in described metadata information
Described data block migration before described original position information updating be described data block migration after destination locations information.
The beneficial effect that technical scheme provided in an embodiment of the present invention is brought is：
When to chunk write data, simultaneously can writing data blocks index of metadata, such conducting disk fragment is whole
During reason, if the valid data occupancy of a chunk is less than pre-determined threshold, in this chunk, the data block of storage is carried out
During arrangement, the index of metadata based on write can be quickly by belonging to data block migration to target chunk and rapid locking data block
The corresponding metadata information of file, so according to the index of metadata being written in chunk as keyword, can be directly in storage
Magnanimity metadata information in quick search to this data block metadata information, and then by this data in this metadata information
The positional information of block is updated it is achieved that just can carry out to the positional information of data block in metadata information without ergodic process
The purpose of modification, above-mentioned disk fragmentses Collator Mode is easy to operate, greatlys save the time, improves arrangement efficiency.
Brief description
For the technical scheme being illustrated more clearly that in the embodiment of the present invention, will make to required in embodiment description below
Accompanying drawing be briefly described it should be apparent that, drawings in the following description are only some embodiments of the present invention, for
For those of ordinary skill in the art, on the premise of not paying creative work, other can also be obtained according to these accompanying drawings
Accompanying drawing.
Fig. 1 is a kind of schematic diagram of disk space that background of invention provides；
Fig. 2 is a kind of schematic diagram of metadata tree structure that background of invention provides；
Fig. 3 is a kind of schematic diagram of metadata tree structure that background of invention provides；
Fig. 4 is a kind of schematic diagram of disk space that background of invention provides；
Fig. 5 is a kind of storage schematic diagram of index of metadata provided in an embodiment of the present invention；
Fig. 6 is a kind of composition schematic diagram of index of metadata provided in an embodiment of the present invention；
Fig. 7 is a kind of structural representation of De-fragmentation apparatus provided in an embodiment of the present invention；
Fig. 8 is a kind of flow chart of scrap cleaning method provided in an embodiment of the present invention；
Fig. 9 is a kind of structural representation of defragmentation device provided in an embodiment of the present invention.
Specific embodiment
Below in conjunction with accompanying drawing, embodiment of the present invention is described further in detail.
Before the embodiment of the present invention is carried out with explanation explained in detail, first to possible in embodiment of the present invention subsequent process
Some nouns being related to explain.
Chunk (burst), in present example, file system can be by disk according to fixing big when carrying out space distribution
Little (such as 4MB size) dividing, that is, the space of a 4MB size is referred to as a chunk.That is, disk logically divides
Become chunk, chunk can be with data storage block, and the position logical address in chunk describes.Logical address and physical address pair
Should, therefore, the adjacent data block of logical address, after write disk, physical address is also adjacent.
Index of metadata, also referred to as reverse indexing, the embodiment of the present invention when writing data to chunk, simultaneously
Index of metadata also can be write.Wherein, FID and this data block of the affiliated file of data block are at least included in index of metadata
Positional information.Wherein, this positional information may include the storage address of this data block and the length value of this data block.Need
Illustrate, storage address herein refers to the initial memory address of this data block.Certainly, above-mentioned positional information is except taking number
Storage address according to block combines outside this form of length value of data block, also can take initial memory address combination termination storage
The mode of location, the embodiment of the present invention is not specifically limited to this.The embodiment of the present invention only combines length value with storage address
Mode is illustrated.
When a file is larger, under normal circumstances when storing this document, this document cutting can be divided for some
Do not store in multiple different chunk.The some that cutting obtains can respectively as a data block be stored in multiple not
In same chunk.Wherein, each data block that file comprises can be in the same size, also can be inconsistent, the embodiment of the present invention pair
This is not specifically limited.
Additionally, in embodiments of the present invention, each data block all corresponds to an original position information and destination locations information.
Wherein, the length value of the former storage address before this data block migration and this data block is referred to as the original position information of this data block.
The above-mentioned former storage address referring to refers to this data block and is stored in storage address during original chunk before migration.In the same manner, should
The length value of the purpose storage address after data block migration and this data block is referred to as the destination locations information of this data block.Purpose
Storage address refers to this data block and is stored in storage address during target chunk after migration.The embodiment of the present invention is only deposited with former
Storage address and the mode of purpose storage address, original position information and destination locations information, to data block before and after migrating
Storage location makes a distinction.
For each data block in a chunk, when the index of metadata to each data block stores, can adopt
Take off and state several ways realization：Stored corresponding index of metadata before each data block, or, after each data block
Store corresponding index of metadata, or, the index of metadata unification of each data block is stored in the head of this chunk, or
Person, the index of metadata unification of each data block is stored in the afterbody of this chunk.For example, Fig. 5 shows metadata rope
Draw in one of chunk storage mode.As shown in figure 5, when writing index of metadata in chunk, can be in each data
The earlier stored position of block is respectively written into corresponding index of metadata, namely writes at the start memory location of each data block
Enter the index of metadata of each data block.Such as, the index of metadata of data block 1 just writes on rising in this chunk in Figure 5
At beginning position.Wherein, FID is nonnegative integer.When opening file or new files, kernel all can return a file identification.Read
Written document is also required to specify file to be read and write using file identification.
In embodiments of the present invention, index of metadata can be concentrated the head writing on a chunk or afterbody to be because
The index of metadata of one data block generally only tens byte-sized, well below the minimum sector-size of hard disk, general feelings
Under condition, minimum sector is 512 byte-sized or 4096 byte-sized.So in order to avoid a sector is only written a first number
According to index, in turn result in space waste, the index of metadata of multiple data blocks is merged by the embodiment of the present invention, for example according to
Shown in Fig. 6, the index of metadata of 2 data blocks is merged.Wherein, the storage of index of metadata adopts TLV (Type
Length Value, type lengths values) form.
Due to hard disk in use for some time, with repeatedly file write and delete, free space less and less and
Usually discontinuous, easily form fragment, therefore now need to carry out defragmentation.Wherein, defragmentation process usually can be related to
To the migration of data block, for being for the chunk that valid data occupancy is less than pre-determined threshold, can will deposit in this chunk
The data block of storage is migrated.In embodiments of the present invention, due to all having been written into the index of metadata of each data block
In chunk, so during conducting disk defragmentation, if the valid data occupancy of a chunk is less than pre-determined threshold,
When migrating the data block that this chunk stores, directly the index of metadata according to each data block just can the unit shown in Fig. 2
Corresponding metadata node, and this number by storage in this metadata node is quickly found in data tree structure (B+ tree construction)
It is revised as the purpose storage address after migrating according to the former storage address of block.Wherein, store this data block in this metadata node
The positional information of all data blocks that the FID of affiliated file and this document comprise.So conducting disk defragmentation when, need not
Travel through whole metadata tree structure, improve defragmentation efficiency.
Fig. 7 is a kind of structural representation of De-fragmentation apparatus provided in an embodiment of the present invention.Referring to Fig. 7, this equipment bag
Include internal memory 701 and processor 702, internal memory 701 has program stored therein code, processor 702 is used for caller code, execution is following
Operation：
Obtain valid data occupancy, the data block that is stored with described chunk and the described data block of chunk in memorizer
Index of metadata；
If the valid data occupancy of described chunk is less than pre-determined threshold, obtain the index of metadata of described data block,
Wherein, in described index of metadata, the FID including the affiliated file of described data block and the original position information of described data block；
After the original position information according to described data block determines that described data block is valid data block, by described data block
As data block to be migrated, it is described data block distribution target chunk, by described data block migration to described target chunk；
According to described FID, search the metadata information of the affiliated file of described data block, will comprise in described metadata information
Described data block migration before described original position information updating be described data block migration after destination locations information.
In another embodiment, processor 702 is used for caller code, and execution is following to be operated：
Based on type lengths values TLV form, described target chunk writes described destination locations information and described
FID, obtains the index of metadata of described data block, wherein, described destination locations information include described data block length value, with
And the purpose storage address after described data block migration.
In another embodiment, processor 702 is used for caller code, and execution is following to be operated：
One bitmap file is set for described chunk, each of institute's bitmap file value bit is used for identifying described
The occupied situation in corresponding space in chunk；
Wherein, when the value of a value bit in institute's bitmap file is first constant, characterize described value bit instruction
Space taken by valid data；When the value of a value bit in institute's bitmap file is second constant, characterize described
The space of value bit instruction is not taken by valid data.
In another embodiment, processor 702 is used for caller code, and execution is following to be operated：
In statistics institute bitmap file, value is the value bit number of described first constant；
Obtain value bit total number in institute's bitmap file；
Calculate the ratio between the value bit number that described value is described first constant and described value bit total number, obtain
Valid data occupancy to described chunk.
In another embodiment, processor 702 is used for caller code, and execution is following to be operated：
According to described former storage address and described length value, the bitmap file of described chunk determines described data block
Take up room at least one value bit corresponding；
If the value of each of at least one value bit described value bit is described first constant it is determined that described number
It is valid data block according to block.
In another embodiment, processor 702 is used for caller code, and execution is following to be operated：
If the valid data occupancy of described chunk is less than pre-determined threshold, by each of described chunk significant figure
All write corresponding target chunk of distribution according to block after, respectively each of the bitmap file of described chunk value bit is entered as
Described second constant.
In another embodiment, processor 702 is used for caller code, and execution is following to be operated：
Search the metadata node of the described FID that is stored with metadata tree structure, store in described metadata node
The metadata information of the affiliated file of described data block, at least includes the FID of described file and described literary composition in described metadata information
The positional information of each data block that part comprises；
Wherein, first number of locally stored all files is have recorded in described metadata tree structure with metadata node form
It is believed that breath.
Equipment provided in an embodiment of the present invention, to chunk write data when, simultaneously can writing data blocks metadata rope
Draw, during such conducting disk defragmentation, if the valid data occupancy of a chunk is less than pre-determined threshold, to this
In chunk, the data block of storage is when being arranged, the index of metadata based on write can quickly by data block migration to target
Chunk and the corresponding metadata information of the affiliated file of rapid locking data block, so according to the metadata rope being written in chunk
Draw as keyword, can directly storage magnanimity metadata information in quick search to this data block metadata information,
And then the positional information of this data block in this metadata information is updated it is achieved that just can be to metadata without ergodic process
The purpose that in information, the positional information of data block is modified, above-mentioned disk fragmentses Collator Mode is easy to operate, greatlys save
Time, improve arrangement efficiency.
Fig. 8 is a kind of flow chart of scrap cleaning method provided in an embodiment of the present invention.Referring to Fig. 8, the embodiment of the present invention
The method flow providing includes：
801st, periodically obtain the valid data occupancy of each of memorizer chunk.
In embodiments of the present invention, the space management of hard disk typically adopts bitmap (bitmap file) to identify in chunk
Which space occupied.The valid data occupancy of one chunk, refers to and is currently used in storage valid data in this chunk
Space account for the ratio in whole spaces in this chunk.Wherein, valid data are for invalid data.Invalid data
Refer to those to have deleted or migrated, but also in the memory space taking chunk, do not carry out the data of space release.With
As a example the space size of every 4KB uses a bit, then the chunk of a 4MB size needs 128 bytes to be used for space management.Time
Go through 128 bytes bitmap of each chunk, just can calculate the valid data occupancy of each chunk, detailed process is as follows.
File system can be respectively provided with a bitmap file, each of this bitmap file value bit for each chunk
For identifying the occupied situation in corresponding space in chunk.That is, the space hold situation of front 4KB is with this space character string
First numerical value bit-identify, the space hold situation and then descending 4KB is equally with second numerical value in this space character string
Bit-identify, the like, the 128*8 numerical value bit-identify in this space character string for the space hold situation of last 4KB.Its
In, when the value of a value bit is first constant (usual value is 1), characterize the chunk space of this value bit instruction
Taken by valid data；When the value of a value bit is second constant (usual value is 0), characterize the instruction of this value bit
Chunk space is not taken by valid data.To sum up, when periodicity obtains the valid data occupancy of each chunk, generally
Following manner is taken to realize：
The first step, for each chunk, file system can in preset duration statistics once this bitmap file value
Value bit number for 1.
Wherein, the big I of preset duration is 10s, 20s or other numerical value etc., and the embodiment of the present invention is big to preset duration
Little specifically do not limited.
Second step, obtain value bit total number in this bitmap file.
In embodiments of the present invention a bit is used with the space size of every 4KB, the size of each chunk for 4MB is
Example, then in a bitmap file, value bit total number is 128*8.
3rd step, calculating value are the ratio between 1 value bit number and value bit total number, obtain this chunk's
Valid data occupancy.
With value be 1 value bit number as 64*8 as a example, then the valid data occupancy of this chunk be 64/128=
50%.
802nd, judge whether the valid data occupancy of each chunk is less than pre-determined threshold；If a chunk's is effective
Data occupancy is less than pre-determined threshold, then execute following step 803；If the valid data occupancy of a chunk is higher than default
Thresholding, then so far terminate to the handling process of this chunk.
Wherein, the big I of pre-determined threshold is 10% or 20% etc., and the embodiment of the present invention is not entered to the size of pre-determined threshold
Row is concrete to be limited.After obtaining the valid data occupancy of each chunk, it is compared with above-mentioned pre-determined threshold, just
Whole chunk that valid data occupancy is less than pre-determined threshold can be obtained.For a chunk, if its valid data
Occupancy is less than pre-determined threshold, then prove that the free space in this chunk will be far longer than occupied space.If so existing many
Individual valid data occupancy is respectively less than the chunk of pre-determined threshold, then can form much discontinuous free spaces on hard disk.For
This needs to carry out defragmentation, and disk fragmentses are brought together, so that the free space in hard disk is as continuous as possible,
Detailed process is referring to following step 803.
If the valid data occupancy of 803 1 chunk is less than pre-determined threshold, obtain each of this chunk number
Index of metadata according to block and each data block.
Wherein, the file identification of the affiliated file of this data block, this data block are at least included in this index of metadata at this
Former storage address when being stored in chunk and the length value of this data block.In embodiments of the present invention, by this data block
Former storage address and the length value of this data block be referred to as the original position information of this data block.Wherein, a file generally wraps
Include multiple data blocks.
In embodiments of the present invention, if index of metadata is centrally stored in the head of chunk, from chunk head solution
After analysing the index of metadata of each data block, first according to referring in the index of metadata that obtains and above-mentioned steps 801
Bitmap file is judging whether each data block is valid data block.Judge that whether a data block is the ancestor of valid data block
Purport is to judge whether the memory space for storing this data block is currently occupied；If the memory space storing this data block is worked as
Front occupied, then show that this data block is valid data block.Wherein, the occupancy situation of memory space, can pass through bitmap file
Determining, detailed process is as follows for the value of middle corresponding value bit：
For each of this chunk data block, the former storage address according to this data block and length value, at this
Determine in the bitmap file of chunk that this data block takes up room at least one value bit corresponding；If this at least one value bit
Each of value bit value be 1 it is determined that this data block be valid data block.If existing in this at least one value bit
Value is 0 value bit it is determined that this data block is the data block having discharged.
For example, the storage location of 8KB in the former storage address of data block as chunk, the length value of data block is
As a example 24KB, then the storage location taking up room as 8KB to 32KB in this chunk for this data block, this correspondence that takes up room
In bitmap file, the 3rd value bit is to the 8th value bit.If the 3rd value bit is each to the 8th this 5 value bit of value bit
Individual value is all 1, then prove that this data block is valid data block；Otherwise it was demonstrated that this data block is the data block having discharged.
It should be noted that after introducing index of metadata, not needing during the data of storage to change in deleting data block
The index of metadata of this data block it is only necessary in modification bitmap file corresponding value bit value, so delete processing is together not
Processing mode before introducing index of metadata is identical.
804th, for one of this chunk valid data block, distribute target chunk for this valid data block, and should
Data block writes this target chunk, writes the index of metadata of this data block in this target chunk simultaneously.
In embodiments of the present invention, for data block distribution target chunk when can choose from idle chunk it is also possible to
It is more than the chunk of pre-determined threshold from the currently active data occupancy and chooses, the embodiment of the present invention is not specifically limited to this.
When carrying out the migration of valid data block, multiple valid data blocks can be moved in a chunk, namely each valid data
Between block, target chunk can be identical, also can migrate in different chunk.Such as, multiple valid data blocks are all moved to one
In individual free time chunk.After finishing the migration of multiple valid data blocks, need to count the current significant figure of this chunk
According to occupancy；If current valid data occupancy is less than pre-determined threshold, also need to will be stored in the data in this chunk again
Block is migrated.
For the situation moving to idle chunk, after a data block is migrated, can not carry out significant figure immediately
According to the statistics of occupancy, if but also do not have other data blocks to write after waiting for a period of time, then carry out valid data occupancy
The statistics of rate.For example, if ceaselessly there being data block to write in an originally idle chunk, often within a period of time
All carrying out timing after write-once data block, if there not being data block to write within the time-count cycle setting again, carrying out significant figure
Statistics according to occupancy.
Each time after new writing data blocks, the presently written position of target chunk is intended to offset backward, until this target
Till chunk writes completely.Additionally, when this data block is written to newly assigned target chunk, simultaneously also can be by this data block
Index of metadata is written in target chunk.
It should be noted that the purpose after now the address of this data block should be this data block migration in index of metadata is deposited
Storage address.It should be noted that the chunk address before the former storage address referring in the embodiment of the present invention and this data block migration
Correlation, before such as migrating, chunk address is the storage location of 4MB, and side-play amount in this chunk for this data block is 1MB, then should
The former storage address of data block is the storage location of 5MB.Chunk ground equally and after this data block migration for the purpose storage address
Location is related, and after such as migrating, chunk address is the storage location of 8MB, and side-play amount in this chunk for this data block is similarly
1MB, then the purpose storage address of this data block is the storage location of 9MB.
That is, file system can obtain the length value of this data block, purpose storage address and file identification；Based on TLV
Form, this length value, purpose storage address and file identification are written in target chunk, complete first number of this data block
According to ablation process in target chunk for the index.
805th, the file identification according to the affiliated file of this data block, searches the metadata information of the affiliated file of this data block,
When the original position information updating of this data block comprising in this metadata information is stored in this target chunk for this data block
Destination locations information.
In embodiments of the present invention, the metadata information of file can be preserved using the metadata tree structure shown in Fig. 2.Unit
Have recorded the metadata information of locally stored all files with metadata node form in data tree structure.Wherein, locally deposit
One file of storage potentially includes multiple data blocks, and each data block is potentially stored in different chunk.One file
The metadata information of this document can be stored in metadata node, wherein in metadata information, at least include the file identification of this document
The positional information of each data block comprising with this document.Due to after data block is migrated, metadata tree structure
Middle storage or this data block migration before positional information it is therefore desirable to be updated to the positional information of this data block, be
This needs the first file identification according to the affiliated file of this data block, the metadata letter of locking storage this document in metadata tree structure
Breath metadata node, afterwards by this data block the original position information updating before migration be this data block in target chunk
Destination locations information during storage.Because the embodiment of the present invention is the length value side of the storage address data block taking data block
Formula carrys out Identifier Location Information, and before and after migration, the length value of data block will not change, and is therefore substantially to storage
Location is updated, and also the former storage address before data block migration will be updated to the purpose storage address after migrating.
For a chunk, above-mentioned steps 802 can be repeated to step 805, until complete in this chunk
Till portion's valid data block all writes new chunk.It should be noted that by each of chunk valid data block
All after corresponding target chunk of write distribution, respectively each of the bitmap file of this chunk value bit is entered as 0, shows
This chunk is current to be an idle chunk.
Method provided in an embodiment of the present invention, to chunk write data when, simultaneously can writing data blocks metadata rope
Draw, during such conducting disk defragmentation, if the valid data occupancy of a chunk is less than pre-determined threshold, to this
In chunk, the data block of storage is when being arranged, the index of metadata based on write can quickly by data block migration to target
Chunk and the corresponding metadata information of the affiliated file of rapid locking data block, so according to the metadata rope being written in chunk
Draw as keyword, can directly storage magnanimity metadata information in quick search to this data block metadata information,
And then the positional information of this data block in this metadata information is updated it is achieved that whole metadata tree structure need not be traveled through
Just the purpose that the positional information of data block in metadata information can be modified, above-mentioned disk fragmentses Collator Mode operation side
Just, greatly save the time, improve arrangement efficiency.
Referring to Fig. 9, embodiments provide a kind of defragmentation device, this device includes：
First acquisition module 901, for obtaining the valid data occupancy of chunk in memorizer, stores in described chunk
There are data block and the index of metadata of described data block；
Second acquisition module 902, if the valid data occupancy for described chunk is less than pre-determined threshold, obtains institute
State the index of metadata of data block, wherein, in described index of metadata, FID including the affiliated file of described data block and described
The original position information of data block；
In the original position information according to described data block, distribute module 903, for determining that described data block is valid data
After block, using described data block as data block to be migrated, it is described data block distribution target chunk, described data block is moved
Move to described target chunk；
Searching modul 904, for according to described FID, searching the metadata information of the affiliated file of described data block；
Update module 905, for the described original position before the described data block migration that will comprise in described metadata information
Information updating be described data block migration after destination locations information.
In another embodiment, this device also includes：
Writing module 906, for based on type lengths values TLV form, writing described purpose position in described target chunk
Confidence breath and described FID, obtain the index of metadata of described data block, wherein, described destination locations information includes described number
According to the purpose storage address after the length value of block and described data block migration.
In another embodiment, this device also includes：
Setup module 907, for arranging a bitmap file, each of institute's bitmap file number for described chunk
Value position is used for identifying the occupied situation in corresponding space in described chunk；
Wherein, when the value of a value bit in institute's bitmap file is first constant, characterize described value bit instruction
Space taken by valid data；When the value of a value bit in institute's bitmap file is second constant, characterize described
The space of value bit instruction is not taken by valid data.
In another embodiment, described first acquisition module 901, in the bitmap file for counting, value is described
The value bit number of first constant；Obtain value bit total number in institute's bitmap file；Calculate described value normal for described first
Ratio between the value bit number of number and described value bit total number, obtains the valid data occupancy of described chunk.
In another embodiment, the former storage address before described original position information includes described data block migration and institute
State the length value of data block, this device also includes：
Determining module 908, for according to described former storage address and described length value, in the bitmap file of described chunk
The described data block of middle determination takes up room at least one value bit corresponding；If each of at least one value bit described numerical value
The value of position is described first constant it is determined that described data block is valid data block.
In another embodiment, this device also includes：
Assignment module 909, if the valid data occupancy for described chunk is less than pre-determined threshold, will be described
After each of chunk valid data block all writes corresponding target chunk of distribution, respectively by the bitmap file of described chunk
Each of value bit be entered as described second constant.
In another embodiment, described searching modul 904, is stored with for lookup in metadata tree structure described
The metadata node of FID, stores the metadata information of the affiliated file of described data block, described unit number in described metadata node
It is believed that at least including the FID of described file in breath and the positional information of each data block that described file comprises；
Wherein, first number of locally stored all files is have recorded in described metadata tree structure with metadata node form
It is believed that breath.
Device provided in an embodiment of the present invention, to chunk write data when, simultaneously can writing data blocks metadata rope
Draw, during such conducting disk defragmentation, if the valid data occupancy of a chunk is less than pre-determined threshold, to this
In chunk, the data block of storage is when being arranged, the index of metadata based on write can quickly by data block migration to target
Chunk and the corresponding metadata information of the affiliated file of rapid locking data block, so according to the metadata rope being written in chunk
Draw as keyword, can directly storage magnanimity metadata information in quick search to this data block metadata information,
And then the positional information of this data block in this metadata information is updated it is achieved that whole metadata tree structure need not be traveled through
Just the purpose that the positional information of data block in metadata information can be modified, above-mentioned disk fragmentses Collator Mode operation side
Just, greatly save the time, improve arrangement efficiency.
It should be noted that：The defragmentation device that above-described embodiment provides when carrying out defragmentation, only with above-mentioned each
The division of functional module is illustrated, and in practical application, can distribute above-mentioned functions by different work(as desired
Module can complete, the internal structure of device will be divided into different functional modules, with complete described above all or portion
Divide function.In addition, the defragmentation device that above-described embodiment provides belongs to same design with scrap cleaning method embodiment, its tool
Body is realized process and is referred to embodiment of the method, repeats no more here.
One of ordinary skill in the art will appreciate that all or part of step realizing above-described embodiment can pass through hardware
To complete it is also possible to the hardware being instructed correlation by program is completed, described program can be stored in a kind of computer-readable
In storage medium, storage medium mentioned above can be read only memory, disk or CD etc..
The foregoing is only presently preferred embodiments of the present invention, not in order to limit the present invention, all spirit in the present invention and
Within principle, any modification, equivalent substitution and improvement made etc., should be included within the scope of the present invention.
Claims (15)
1. a kind of scrap cleaning method is it is characterised in that methods described includes：
Obtain valid data occupancy, the data block that is stored with described chunk and the described data block of burst chunk in memorizer
Index of metadata；
If the valid data occupancy of described chunk is less than pre-determined threshold, obtain the index of metadata of described data block, its
In, in described index of metadata, the file identification FID including the affiliated file of described data block and the original position confidence of described data block
Breath；
The original position information according to described data block determine described data block be valid data block after, using described data block as
Data block to be migrated, is described data block distribution target chunk, by described data block migration to described target chunk；
According to described FID, search the metadata information of the affiliated file of described data block, by the institute comprising in described metadata information
State the destination locations information after the described original position information updating before data block migration is described data block migration.
2. method according to claim 1 it is characterised in that described for described data block distribution target chunk after, institute
Method of stating also includes：
Based on type lengths values TLV form, described target chunk writes described destination locations information and described FID, obtains
To the index of metadata of described data block, wherein, described destination locations information includes the length value of described data block and described
Purpose storage address after data block migration.
3. method according to claim 1 it is characterised in that in described acquisition memorizer burst chunk valid data
Before occupancy, methods described also includes：
One bitmap file is set for described chunk, each of institute's bitmap file value bit is used for identifying described chunk
The occupied situation in middle corresponding space；
Wherein, when the value of a value bit in institute's bitmap file is first constant, characterize the sky of described value bit instruction
Between taken by valid data；When the value of a value bit in institute's bitmap file is second constant, characterize described numerical value
The space of position instruction is not taken by valid data.
4. method according to claim 3 it is characterised in that in described acquisition memorizer burst chunk valid data
Occupancy, including：
In statistics institute bitmap file, value is the value bit number of described first constant；
Obtain value bit total number in institute's bitmap file；
Calculate the ratio between the value bit number that described value is described first constant and described value bit total number, obtain institute
State the valid data occupancy of chunk.
5. method according to claim 3 is it is characterised in that before described original position information includes described data block migration
Former storage address and described data block length value, methods described also includes：
According to described former storage address and described length value, determine that described data block takies in the bitmap file of described chunk
Space at least one value bit corresponding；
If the value of each of at least one value bit described value bit is described first constant it is determined that described data block
For valid data block.
6. the method according to any claim in claim 3 to 5 is it is characterised in that methods described also includes：
If the valid data occupancy of described chunk is less than pre-determined threshold, by each of described chunk valid data block
All after corresponding target chunk of write distribution, respectively each of the bitmap file of described chunk value bit is entered as described
Second constant.
7. method according to claim 1 it is characterised in that described according to described FID, search literary composition belonging to described data block
The metadata information of part, including：
Search the metadata node of the described FID that is stored with metadata tree structure, store described in described metadata node
The metadata information of the affiliated file of data block, at least includes the FID of described file and described file bag in described metadata information
The positional information of each data block containing；
Wherein, the metadata letter of locally stored all files is have recorded in described metadata tree structure with metadata node form
Breath.
8. a kind of defragmentation device is it is characterised in that described device includes：
First acquisition module, for obtaining the valid data occupancy of burst chunk in memorizer, is stored with described chunk
Data block and the index of metadata of described data block；
Second acquisition module, if the valid data occupancy for described chunk is less than pre-determined threshold, obtains described data block
Index of metadata, wherein, in described index of metadata, file identification FID including the affiliated file of described data block and described
The original position information of data block；
Distribute module, for, after the original position information according to described data block determines that described data block is valid data block, inciting somebody to action
Described data block, as data block to be migrated, is described data block distribution target chunk, and described data block migration is extremely described
Target chunk；
Searching modul, for according to described FID, searching the metadata information of the affiliated file of described data block；
Update module, for the described original position information updating before the described data block migration that will comprise in described metadata information
For the destination locations information after described data block migration.
9. device according to claim 8 is it is characterised in that described device also includes：
Writing module, for based on type lengths values TLV form, writing described destination locations information in described target chunk
And described FID, obtain the index of metadata of described data block, wherein, described destination locations information includes described data block
Purpose storage address after length value and described data block migration.
10. device according to claim 8 is it is characterised in that described device also includes：
Setup module, for arranging a bitmap file for described chunk, each of institute's bitmap file value bit is used for
Identify the occupied situation in corresponding space in described chunk；
Wherein, when the value of a value bit in institute's bitmap file is first constant, characterize the sky of described value bit instruction
Between taken by valid data；When the value of a value bit in institute's bitmap file is second constant, characterize described numerical value
The space of position instruction is not taken by valid data.
11. devices according to claim 10 it is characterised in that described first acquisition module, for counting described bitmap
In file, value is the value bit number of described first constant；Obtain value bit total number in institute's bitmap file；Calculate described
Value is the ratio between the value bit number of described first constant and described value bit total number, obtains having of described chunk
Effect data occupancy.
12. devices according to claim 10 are it is characterised in that described original position information includes described data block migration
Front former storage address and the length value of described data block, described device also includes：
Determining module, for according to described former storage address and described length value, determining institute in the bitmap file of described chunk
State data block to take up room at least one value bit corresponding；If the value of each of at least one value bit described value bit
It is described first constant it is determined that described data block is valid data block.
13. devices according to any claim in claim 10 to 12 are it is characterised in that described device also includes：
Assignment module, if the valid data occupancy for described chunk is less than pre-determined threshold, will be every in described chunk
After one valid data block all writes corresponding target chunk of distribution, respectively by each of bitmap file of described chunk number
Value position is entered as described second constant.
14. devices according to claim 8 it is characterised in that described searching modul, for looking in metadata tree structure
Look for the metadata node of the described FID that is stored with, in described metadata node, store the metadata of the affiliated file of described data block
Information, at least includes the FID of described file and the position of each data block that described file comprises in described metadata information
Information；
Wherein, the metadata letter of locally stored all files is have recorded in described metadata tree structure with metadata node form
Breath.
A kind of 15. De-fragmentation apparatus are it is characterised in that described equipment includes：Internal memory and processor, described memory storage has journey
Sequence code, described processor is used for calling described program code, and execution is following to be operated：
Obtain valid data occupancy, the data block that is stored with described chunk and the described data block of burst chunk in memorizer
Index of metadata；
If the valid data occupancy of described chunk is less than pre-determined threshold, obtain the index of metadata of described data block, its
In, in described index of metadata, the file identification FID including the affiliated file of described data block and the original position confidence of described data block
Breath；
The original position information according to described data block determine described data block be valid data block after, using described data block as
Data block to be migrated, is described data block distribution target chunk, by described data block migration to described target chunk；
According to described FID, search the metadata information of the affiliated file of described data block, by the institute comprising in described metadata information
State the destination locations information after the described original position information updating before data block migration is described data block migration.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610776743.7A CN106469120A (en) | 2016-08-30 | 2016-08-30 | Scrap cleaning method, device and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610776743.7A CN106469120A (en) | 2016-08-30 | 2016-08-30 | Scrap cleaning method, device and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106469120A true CN106469120A (en) | 2017-03-01 |
Family
ID=58230403
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610776743.7A Pending CN106469120A (en) | 2016-08-30 | 2016-08-30 | Scrap cleaning method, device and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106469120A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107368260A (en) * | 2017-06-30 | 2017-11-21 | 北京奇虎科技有限公司 | Memory space method for sorting, apparatus and system based on distributed system |
| CN107688542A (en) * | 2017-08-17 | 2018-02-13 | 歌尔股份有限公司 | Stored data processing method and device |
| CN109388347A (en) * | 2018-10-15 | 2019-02-26 | 郑州云海信息技术有限公司 | A kind of data migration method and device |
| CN110221781A (en) * | 2019-06-04 | 2019-09-10 | Oppo广东移动通信有限公司 | A kind of manufacturing method of disk fragments, device, storage medium and intelligent terminal |
| CN111221468A (en) * | 2018-11-26 | 2020-06-02 | 杭州海康威视系统技术有限公司 | Storage block data deleting method and device, electronic equipment and cloud storage system |
| CN111290970A (en) * | 2018-12-06 | 2020-06-16 | 爱思开海力士有限公司 | Memory system and operating method thereof |
| CN111400302A (en) * | 2019-11-28 | 2020-07-10 | 杭州海康威视系统技术有限公司 | Method, device and system for modifying continuously stored data |
| CN112667525A (en) * | 2020-12-23 | 2021-04-16 | 北京浪潮数据技术有限公司 | Used space measuring method and component of persistent memory |
| CN114925020A (en) * | 2022-07-20 | 2022-08-19 | 中电云数智科技有限公司 | Snapshot version data migration method based on data increment writing mode |
| CN115168298A (en) * | 2022-06-09 | 2022-10-11 | 荣耀终端有限公司 | File system fragmentation evaluation method and electronic equipment |
| CN117270789A (en) * | 2023-10-25 | 2023-12-22 | 苏州元脑智能科技有限公司 | Data control method and system based on distributed storage and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103488685A (en) * | 2013-09-02 | 2014-01-01 | 上海网达软件股份有限公司 | Fragmented-file storage method based on distributed storage system |
| CN104239234A (en) * | 2014-10-13 | 2014-12-24 | 合一网络技术(北京)有限公司 | High-efficiency local cache management and reading method |
| CN104346357A (en) * | 2013-07-29 | 2015-02-11 | 中国科学院声学研究所 | File accessing method and system for embedded terminal |
| CN105589812A (en) * | 2015-12-16 | 2016-05-18 | 成都华为技术有限公司 | Disk defragmentation method, disk defragmentation device and host |
-
2016
- 2016-08-30 CN CN201610776743.7A patent/CN106469120A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104346357A (en) * | 2013-07-29 | 2015-02-11 | 中国科学院声学研究所 | File accessing method and system for embedded terminal |
| CN103488685A (en) * | 2013-09-02 | 2014-01-01 | 上海网达软件股份有限公司 | Fragmented-file storage method based on distributed storage system |
| CN104239234A (en) * | 2014-10-13 | 2014-12-24 | 合一网络技术(北京)有限公司 | High-efficiency local cache management and reading method |
| CN105589812A (en) * | 2015-12-16 | 2016-05-18 | 成都华为技术有限公司 | Disk defragmentation method, disk defragmentation device and host |
Non-Patent Citations (1)
| Title |
|---|
| 高岩 等: "一种按需分配增量卷的自动扩容方法研究", 《全国信息存储技术大会》 * |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107368260A (en) * | 2017-06-30 | 2017-11-21 | 北京奇虎科技有限公司 | Memory space method for sorting, apparatus and system based on distributed system |
| CN107688542A (en) * | 2017-08-17 | 2018-02-13 | 歌尔股份有限公司 | Stored data processing method and device |
| CN109388347A (en) * | 2018-10-15 | 2019-02-26 | 郑州云海信息技术有限公司 | A kind of data migration method and device |
| CN111221468A (en) * | 2018-11-26 | 2020-06-02 | 杭州海康威视系统技术有限公司 | Storage block data deleting method and device, electronic equipment and cloud storage system |
| CN111221468B (en) * | 2018-11-26 | 2023-10-13 | 杭州海康威视系统技术有限公司 | Storage block data deleting method and device, electronic equipment and cloud storage system |
| CN111290970B (en) * | 2018-12-06 | 2023-04-07 | 爱思开海力士有限公司 | Memory system and operating method thereof |
| CN111290970A (en) * | 2018-12-06 | 2020-06-16 | 爱思开海力士有限公司 | Memory system and operating method thereof |
| CN110221781A (en) * | 2019-06-04 | 2019-09-10 | Oppo广东移动通信有限公司 | A kind of manufacturing method of disk fragments, device, storage medium and intelligent terminal |
| CN111400302A (en) * | 2019-11-28 | 2020-07-10 | 杭州海康威视系统技术有限公司 | Method, device and system for modifying continuously stored data |
| CN111400302B (en) * | 2019-11-28 | 2023-09-19 | 杭州海康威视系统技术有限公司 | Modification method, device and system for continuous storage data |
| CN112667525A (en) * | 2020-12-23 | 2021-04-16 | 北京浪潮数据技术有限公司 | Used space measuring method and component of persistent memory |
| CN115168298A (en) * | 2022-06-09 | 2022-10-11 | 荣耀终端有限公司 | File system fragmentation evaluation method and electronic equipment |
| CN114925020A (en) * | 2022-07-20 | 2022-08-19 | 中电云数智科技有限公司 | Snapshot version data migration method based on data increment writing mode |
| CN117270789A (en) * | 2023-10-25 | 2023-12-22 | 苏州元脑智能科技有限公司 | Data control method and system based on distributed storage and electronic equipment |
| CN117270789B (en) * | 2023-10-25 | 2024-02-06 | 苏州元脑智能科技有限公司 | Data control method and system based on distributed storage and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106469120A (en) | Scrap cleaning method, device and equipment | |
| US11100071B2 (en) | Key-value store tree data block spill with compaction | |
| US11693830B2 (en) | Metadata management method, system and medium | |
| CN107038206B (en) | LSM tree establishing method, LSM tree data reading method and server | |
| KR102031588B1 (en) | Method and system for implementing index when saving file | |
| CN112395212B (en) | Method and system for reducing garbage recovery and write amplification of key value separation storage system | |
| CN110188108B (en) | Data storage method, device, system, computer equipment and storage medium | |
| CN104933169B (en) | Based on the preferential file system fragmentation method for sorting of hot spot file | |
| CN109445702B (en) | block-level data deduplication storage system | |
| CN111399777A (en) | Differentiated key value data storage method based on data value classification | |
| EP3316150B1 (en) | Method and apparatus for file compaction in key-value storage system | |
| CN107209714A (en) | The control method of distributed memory system and distributed memory system | |
| CN103714013B (en) | A kind of collocation method and device of the memory space of file system | |
| CN107665219B (en) | Log management method and device | |
| CN104238962A (en) | Method and device for writing data into cache | |
| CN111522507B (en) | Low-delay file system address space management method, system and medium | |
| CN105159616A (en) | Disk space management method and device | |
| KR20170065374A (en) | Method for Hash collision detection that is based on the sorting unit of the bucket | |
| KR100907477B1 (en) | Apparatus and method for managing index of data stored in flash memory | |
| CN106294189B (en) | Memory defragmentation method and device | |
| CN103530067A (en) | Data operation method and device | |
| CN106528703A (en) | Deduplication mode switching method and apparatus | |
| CN105068941A (en) | Cache page replacing method and cache page replacing device | |
| CN103425802A (en) | Method for quickly retrieving magnetic disk file | |
| CN105630406B (en) | Using MRAM as the storage system of editor's buffer area and editor's caching method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170301 |
|
| RJ01 | Rejection of invention patent application after publication |