CN104035729A - Block device thin-provisioning method for log mapping - Google Patents
Block device thin-provisioning method for log mapping Download PDFInfo
- Publication number
- CN104035729A CN104035729A CN201410218798.7A CN201410218798A CN104035729A CN 104035729 A CN104035729 A CN 104035729A CN 201410218798 A CN201410218798 A CN 201410218798A CN 104035729 A CN104035729 A CN 104035729A
- Authority
- CN
- China
- Prior art keywords
- mapping
- resource
- record
- write
- map information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000013507 mapping Methods 0.000 title claims abstract description 370
- 238000000034 method Methods 0.000 title claims abstract description 121
- 238000003860 storage Methods 0.000 claims abstract description 151
- 238000007726 management method Methods 0.000 claims abstract description 53
- 238000013468 resource allocation Methods 0.000 claims abstract description 20
- 238000009826 distribution Methods 0.000 claims abstract description 18
- 230000008569 process Effects 0.000 claims description 60
- 230000002441 reversible effect Effects 0.000 claims description 38
- 238000011084 recovery Methods 0.000 claims description 14
- 230000009471 action Effects 0.000 claims description 11
- 238000004064 recycling Methods 0.000 claims description 9
- 239000000976 ink Substances 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 230000007423 decrease Effects 0.000 claims description 6
- 238000012217 deletion Methods 0.000 claims description 6
- 230000037430 deletion Effects 0.000 claims description 6
- 238000012856 packing Methods 0.000 claims description 4
- 239000013589 supplement Substances 0.000 claims description 4
- 238000007599 discharging Methods 0.000 claims description 3
- 230000002045 lasting effect Effects 0.000 claims description 3
- 239000002245 particle Substances 0.000 claims description 3
- 238000011176 pooling Methods 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 15
- 238000012795 verification Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 5
- 239000008187 granular material Substances 0.000 description 3
- 241001269238 Data Species 0.000 description 2
- 238000002679 ablation Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003760 hair shine Effects 0.000 description 2
- 101000648827 Homo sapiens TPR and ankyrin repeat-containing protein 1 Proteins 0.000 description 1
- 102100028173 TPR and ankyrin repeat-containing protein 1 Human genes 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000005498 polishing Methods 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
- 230000002459 sustained effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a block device thin-provisioning method for log mapping. The method comprises the step of hard disk data distribution, wherein storage resources are formatted, and resource pooling management is carried out on hard disk data distribution so as to support thin-provisioning resource allocation and storage and management of log mapping information, the step of thin-provisioning resource management, wherein on the basis of a hard disk with pooled resources, the data are sequentially written into a next free resource of the hard disk in a writing on-demand allocation mode of the storage resources, and mapping information corresponding to write operation is generated; the step of mapping information logging, wherein logging storage is carried out on the mapping information of the write operation according to an additional allocation mode, and on the basis of an asynchronous updating mode, the mapping information is amended so as to achieve data writing of the multi-user and multi-virtual-volume concurrence sequence.
Description
Technical field
The present invention relates to field of data storage, particularly one is simplified collocation method.
Background technology
Along with modern society's information data increases, capacity, performance and the maintainability of large data challenge storage system.Along with the data volume of storage maintains sustained and rapid growth, in non-service affecting situation, storage system need to increase new hard disk and carry out capacity dilatation.But in traditional storage system, when certain application needs when a part of storage space, system divides goes out a part and enough large space allocates in advance to this application.Now, this application does not temporarily have enough data to fill the storage space of distributing to it.Because this part storage space is reserved out, therefore system cannot offer vacant space other application use.In addition, the Data Growth that this application program produces likely exceeds the volume space of aforementioned distribution in the future.
Simplify configuration (Thin-Provisioning) storage system and can realize dynamic assignment and the growth of storage resources.Simplify configuring technical also referred to as " automatically simplifying configuration ".Physical store resource only has in the time that data write operation occurs and is just assigned with, and the method for this allocation space is also referred to as " distribution according to need " (On-Demand Allocation) or " while writing, distributing ".Simplify configuration all storage spaces are pooled in a unified storage pool (Storage Pool), and use onesize " section " (Segment) by space separating, " section " in distribution according to need storage pool.Simplify configure storage system and offer user's virtual volume (Volume).Can being set by storage administrator by apparent capacity of virtual volume, its virtual address space is divided with onesize " section ".Write operation requests according to application to " volume ", system is set up map information as required, while after this this " section " being carried out to all read-write operations, is redirected write operation to corresponding hard disk address according to map information.Simplify configuration and provide the virtual storage capacity larger than actual physical storage capacity by Storage Virtualization.In the time that remaining memory capacity is less in system, by adding new hard disk, simplifies configure storage system and can again expand storage space.Always the above, simplify configuring technical and have very high storage space utilization factor, and can dynamically realize storage space dilatation.
In prior art, there is following problem:
1) performance of storage is mainly limited by write performance, and 90% read operation will be hit main memory (RAM), and on hard disk, the majority of I/O request is write operation, and the write performance of therefore optimizing storage is very crucial.
But multiple virtual volumes order is concurrent will cause simplifying the hydraulic performance decline of configuration store while writing same hard disk.The storage resources of a hard disk may be distributed to multiple virtual volumes, write fashionable (the video monitoring environment of for example multi-cam at these virtual volumes simultaneously, the non-linear editing environment of film post-production), multiple concurrent order ablation process show as concurrent discrete writing on hard disk, this write mode postpones higher, thereby cause hard disk bandwidth very low, hard disk writes the bandwidth order that may decline to a great extent and writes 10% of bandwidth.
2) in addition, existing log approach (data block journal file, Journaling File System) is applicable to " file system layer ", and simplifies " the block device layer " of configuration store in file system layer lower floor.Therefore, existing log approach inapplicable.
Summary of the invention
The present invention proposes in conjunction with daily record and simplify the technology of resource allocation management, by all next idling-resources of allocation scheme write sequence when writing of data and map information.Advantage of the present invention is to improve hard disk bandwidth availability ratio, reduces concurrent sequential write expense, realizes resource demand assigned and simplifies configuration store.The object of the invention is to overcome the existing configure storage system of simplifying and write in multireel order the defect that hard disk Time Bandwidth declines, in conjunction with daily record technology with simplify resource allocation management, when having proposed data and map information and all writing by storage resources, allocation scheme writes the method for next idling-resource, thereby provides the concurrent order of the many virtual volumes of a kind of multi-user of being applicable to the configuration store method of simplifying that write, high performance dummy block will mechanical floor.
For reaching above-mentioned purpose, the block device that the invention provides a kind of daily record mapping is simplified collocation method, it is characterized in that, described method comprises:
Hard disc data layout step: by storage resources format, hard disc data layout is carried out to resource pool management, to support to simplify the resource distribution of configuration and the store and management of daily record map information;
Simplify resource allocation management process: based on the hard disk of described resource pool, adopt the mode of distributing according to need when data are write according to storage resources sequentially to write next idling-resource of described hard disk, and generate corresponding write operation map information;
Map information daily record step: described write operation map information is carried out to daily record storage according to appending allocation scheme, and carry out the retouching operation of described map information based on asynchronous refresh mode, write with the data that realize the concurrent order of the many virtual volumes of multi-user.
The block device of above-mentioned daily record mapping is simplified collocation method, it is characterized in that, described method also comprises:
System bootstrapping and recovering step: based on two-stage checkpoint method and dynamic log district management method, that realizes described storage resources writes the time-division with addition of speed, and the fast quick-recovery of system bootstrapping and the fast quick-recovery of collapse.
The block device of above-mentioned daily record mapping is simplified collocation method, it is characterized in that, described unit comprises:
Grain: for the least unit of Storage Resource Management (SRM);
Section: the fixed length resource being made up of the multiple continuous described grain of fixed qty, comprises: data and oppositely mapping table.Described reverse mapping table is for described section of reverse map information for the described grain of data of lasting storage, and described reverse map information is for obtaining virtual address from hard disk address lookup;
Bunch: by the described section of multiple continuous fixed length resources that form of fixed qty, storage pool, to the unit of dynamic log district Resources allocation, comprising: bunch numbering, idle grain quantity, can distribute a quantity, idle grain bitmap and timestamp;
Hard disk metadata information: the data message reading for described system bootstrapping and recovering step, comprising: physical hard disk information, storage pool information, virtual volume information, map retrieval information and idling-resource information;
Metadata volume: for forward map information and the retouching operation thereof of forward mapping table are write to daily record, described metadata volume is corresponding one by one with virtual volume.
The block device of above-mentioned daily record mapping is simplified collocation method, it is characterized in that, described reverse mapping table comprises: virtual grain number, check information, virtual volume mark, the hard disk grain that discharges number and timestamp, whether described check information completes with the reverse mapping table of described grain for the described grain that checks storage data, the rise time of described timestamp for recording a described reverse map information, described reverse map information and data are taked asynchronous write mode.
The block device of above-mentioned daily record mapping is simplified collocation method, it is characterized in that, described forward mapping table is stored in main memory, for storing the forward map information that obtains hard disk address from virtual address inquiry, described forward mapping table comprises district's mapping, described district is mapped as the mapping between continuous virtual grain and continuous hard disk grain, and the mapping of described district comprises: physical store resource apparatus mark, the initial grain of physics numbering, virtual volume mark, virtual initial grain numbering and quantity continuously.
The block device of above-mentioned daily record mapping is simplified collocation method, it is characterized in that, described in simplify resource allocation management process and comprise:
Write operation step: distribute and Journaled allocation strategy while writing based on resource, data are appended in order and write hard disk, and generate map information;
Read operation step: by searching the map information of described forward mapping table, carry out data and read.
The block device of above-mentioned daily record mapping is simplified collocation method, it is characterized in that, described map record daily record step comprises:
Map information is write step: the new map information that described write operation step is generated writes the described reverse mapping table of hard disk, upgrades the described forward mapping table being stored in main memory after success, and by the map information asynchronous write daily record of described forward mapping table;
Map information postpones write step: described forward map information grouping is packaged as the forward map record of described grain size, and the timestamp of described forward map record is all described forward map information timestamp minimum value in packing moment group;
Map information modify steps: the described map record that is shone upon and remapped by releasing, discharges the storage resources of virtual volume or carry out mapping relations conversion between described virtual volume;
Map information recycling step: forward map record writes daily record finish after, reclaim the storage resources of old map record, described map record comprises: described forward map record, remove map record and the record that remaps.
The block device of above-mentioned daily record mapping is simplified collocation method, it is characterized in that, described distribute while writing based on resource and Journaled allocation strategy comprises:
Described write operation is divided into align to write with non-alignment and writes, described alignment is written as and writes the next free block in daily record end, and described non-alignment is written as and writes the daily record position that the last time writes, and that writes as described alignment supplements.
The block device of above-mentioned daily record mapping is simplified collocation method, it is characterized in that, described system bootstrapping and recovering step comprise:
Two-stage checkpoint step: by the forward map information of one-level checkpoint stored virtual volume, and store the retrieving information of described forward map information by secondary checkpoint;
Dynamic log district management process: the region of Resources allocation when dynamic mobile and increase and decrease write, to realize the dynamic assignment of resource, and realize the retrieval in dynamic log district;
System recovering step: by reading forward map information, dynamic log district information and mapping amendment record, completion system recovers.
The block device of above-mentioned daily record mapping is simplified collocation method, it is characterized in that, described write operation step also comprises:
Receive write operation step: the data that write are divided into multiple write operation data blocks according to described grain, the data that are described grain length for described data block length, take described alignment WriteMode, be less than the data of described grain length for described data block length, take described non-alignment WriteMode;
Search forward mapping table step: the map information of searching described data block in the described forward mapping table in main memory, if search described map information success, judge whether to take described alignment WriteMode, if search described map information failure, distribute new storage resources according to the described WriteMode that appends;
Step is write in alignment: for the described data block that need to take described alignment WriteMode, described data block is write to daily record, be first free block after writing position last time, and virtual volume address information is recorded, forward mapping table information and timestamp described in renewal main memory in the asynchronous end at daily record present segment;
Non-alignment is write step: for the described data block that need to take described non-alignment WriteMode, described data block is write to described grain at the physical location of the last time of daily record, and map information in described forward mapping table remains unchanged;
Recycling step: reclaim the storage resources losing efficacy, the success of virtual volume write operation.
The block device of above-mentioned daily record mapping is simplified collocation method, it is characterized in that, described map information modify steps also comprises:
Remove mapping step: for discharging the storage resources of described virtual volume;
Step remaps: for carry out the conversion of mapping relations between described virtual volume.
The block device of above-mentioned daily record mapping is simplified collocation method, it is characterized in that, described releasing mapping step also comprises:
Receive and remove request step: virtual volume receives the mapping request of removing, wait to remove the mapping range of mapping according to described releasing mapping acquisition request, and be the buffer memory of writing of described grain integral multiple according to described mapping range length allocation length;
Search releasing step: in main memory, search described forward mapping table for the virtual address of each mapping (enum) data piece to be removed in described mapping range, search the corresponding list item that hits described forward mapping table, and write buffer memory described in releasing map information is recorded in, and write under buffer memory and write with a brush dipped in Chinese ink into daily record described in inciting somebody to action;
Delete mapping step: the virtual address to mapping (enum) data piece to be removed described in described mapping range is carried out deletion action, delete the corresponding list item in forward mapping table described in main memory, the timestamp that upgrades correspondence mappings interval in described forward mapping table is current time, and described in inciting somebody to action, mapping (enum) data piece to be removed is labeled as idling-resource.
The block device of above-mentioned daily record mapping is simplified collocation method, it is characterized in that, described in the step that remaps also comprise:
The reception request step that remaps: virtual volume receives remapping request, the mapping range remapping according to the described acquisition request that remaps, and be the buffer memory of writing of described integral multiple according to described mapping range length allocation length;
Search mapping step: in main memory, search source virtual volume to be mapped according to each virtual address to be remapped in described mapping range, in the virtual volume of described source, search the corresponding list item that hits described forward mapping table, and carry out target virtual volume to be mapped according to same virtual address and search, in described target virtual volume, search the list item that hits described forward mapping table, the operation note of remapping is write in buffer memory described, and under write with a brush dipped in Chinese ink into daily record;
Step of updating: search successful list item in the forward mapping table of source virtual volume described in deletion main memory, the timestamp that upgrades mapping range in the virtual volume of described source is current time, and upgrade the list item of searching successfully of described target virtual volume, the timestamp that upgrades mapping range in described target virtual volume is current time, and the data block of removing mapping in described target virtual volume is labeled as to idling-resource.
The block device of above-mentioned daily record mapping is simplified collocation method, it is characterized in that, described two-stage checkpoint step also comprises:
Scanning step: the information of the mapping table of forward described in main memory is distributed by group, and described group is all mappings in described bunch, and described group is scanned;
Step toward division: if described group takes up room and be greater than the specified size of described grain, be split into multiple groups by described group, described group after division is scanned;
Combining step: if described group takes up room and be less than the specified size of described grain, described group is merged, described group after being combined scans;
Recording step: if described group take up room in the specified size of described grain distributes the buffer area of a described particle size, the mapping range in described group and current time stamp are recorded to described buffer area;
Discharge recycling step: if described forward mapping table belongs to described metadata volume, the map retrieval district of described hard disk head will be brushed under described buffer area, and the storage resources of the map record of recovery inefficacy, if described forward mapping table belongs to described virtual volume, will under described buffer area, brush described metadata volume, and releasing reverse mapping resource.
The block device of above-mentioned daily record mapping is simplified collocation method, it is characterized in that, described dynamic log district management process also comprises:
Resource allocation step while writing: for writing time-division timing, from dynamic log area Resources allocation;
Dynamic log management process: while recovery for system, read correlation map information, carry out timestamp comparison, by the mapping information recovering of up-to-date timestamp in main memory.
The block device of above-mentioned daily record mapping is simplified collocation method, it is characterized in that, described system recovering step also comprises:
Read retrieving information step: read the forward mapping table on hard disk, load described forward mapping table to main memory, read the dynamic log district retrieving information on described hard disk, carry out location, dynamic log district;
Scanning step: dynamic log district described in sequential scanning, read mapping amendment record, and according to described mapping amendment record inquiry forward mapping table, if the timestamp of described mapping amendment record is less than the timestamp of described forward mapping table, abandon described mapping amendment record, if the timestamp of described mapping amendment record is greater than the timestamp of described forward mapping table, described mapping amendment record is updated to described forward mapping table, and reclaims the storage resources of described mapping amendment record.
The block device of above-mentioned daily record mapping is simplified collocation method, it is characterized in that, described in while writing resource allocation step also comprise:
Query resource step: inquiry, when the storage resources of prevariety, if described bunch does not have idling-resource, is inquired about next described bunch;
Resources allocation step: if in described query resource step, inquiry is described when available free resource in prevariety, from described when memory allocated resource prevariety;
Check dynamic resource step: check total idling-resource quantity in current dynamic log district, if described idling-resource is less than certain threshold value, carry out described dynamic log management process.
The block device of above-mentioned daily record mapping is simplified collocation method, it is characterized in that, described dynamic log management process also comprises:
Search idling-resource step: available free described bunch of sequential search in memory resource pool, if searched successfully, by the updating location information of described bunch to the dynamic log retrieval in main memory, the buffer area of monthly dynamics log searching section length;
Upgrade bunch step: in described buffer area, insert timestamp and described dynamic log district bunch relevant information, will under buffer area, brush the dynamic log retrieval district of all hard disks in storage pool, and upgrade the dynamic log district information in main memory;
Notifying process: described in search in idling-resource step, if search described bunch of idle failure, output notice user profile.
Compared with prior art, beneficial effect of the present invention is:
1) combine daily record resource and append and distribute and simplify resource allocation distribution according to need, by the data that write from the concurrent order of multiple virtual volumes and new map information all when writing allocation scheme write hard disk, general allocation strategy is write for appending, expense is little.Therefore in the time of the concurrent sequential write of multi-user's multithread, hard disk bandwidth availability ratio of the present invention high (>85%);
2) daily record of map information writes, and has realized the dynamic scatter storage of map information, has simplified the storage administration complicacy of map information;
3) mapping amendment record first sequentially writes the daily record of self, and then asynchronous refresh, to the daily record of forward mapping, has been realized mapping amendment rapidly;
4) system bootstrapping speed is fast, and the present invention loads map information by forward map retrieval, then only needs to scan dynamic log district and loads up-to-date map information.
Brief description of the drawings
Fig. 1 is the inventive method schematic flow sheet;
Fig. 2 A～Fig. 2 E is the inventive method detailed process schematic diagram;
Fig. 3 is physical hard disk schematic layout pattern of the present invention;
Fig. 4 is the present invention's bunch schematic layout pattern;
Fig. 5 is the schematic layout pattern of section of the present invention;
Fig. 6 is embodiment of the present invention write operation steps flow chart schematic diagram;
Fig. 7 is that the embodiment of the present invention is removed mapping step schematic flow sheet;
Fig. 8 is the embodiment of the present invention steps flow chart schematic diagram that remaps;
Fig. 9 is that the embodiment of the present invention is upgraded map information steps flow chart schematic diagram;
Figure 10 is that embodiment of the present invention mapping is appended and write and two-stage checkpoint steps flow chart schematic diagram;
Figure 11 is embodiment of the present invention two-stage checkpoint steps flow chart schematic diagram;
Figure 12 is embodiment of the present invention resource allocation flow schematic diagram while writing;
Figure 13 is embodiment of the present invention dynamic log district management process schematic flow sheet;
Figure 14 is that embodiment of the present invention system is recovered schematic flow sheet.
Wherein, Reference numeral:
S1～S4, S21～S22, S31～S34, S41～S43, S211～S215, S411～S415, S3311～S3313, S3321～S3323, S4211～S4213, S4221～S4223: the administration step of various embodiments of the present invention
Embodiment
Provide the specific embodiment of the present invention below, in conjunction with diagram, the present invention has been made to detailed description.
" append and write " that the present invention proposes can maximize the bandwidth of writing of hard disk (comprising solid-state hard disk SSD), but the managing mapping information complexity of storage resources.Append and write the daily record and the log-structured file system (Log-structured File System, LFS) that derive from database use, can keep continuous order to write.LFS use an order, can only append the structure of writing (Append), a series of changes to file are recorded in a large journal entry, and by this journal entry write-once hard disk.Therefore, LFS writes large consecutive data block of hard disk at every turn, thereby improves the utilization factor of hard disk bandwidth.After system crash, just recovery file system fast of the afterbody that only need check journal file.Because daily record writes, system need to safeguard that the map information of storage resources realizes the distribution according to need of resource and idling-resource management.Meanwhile, memory allocated when map information is write, this makes the Comparison of Management complexity of storage resources.
Key problem in technology point of the present invention is:
1) when resource is write, distribute and towards the Journaled allocation strategy of optimizing write performance, can maintain the lower expense that writes.
A) write operation being divided into " alignment is write " and " non-alignment is write " processes respectively.
B) alignment is write: write the next free block in daily record end.
C) non-alignment is write: write the daily record position that last time of this address writes, as supplementing of " alignment being write ".
2) the hard disc data layout (also referred to as " format ") of resource poolization management, can support to simplify the store and management of the resource distribution of configuration and management, daily record map information.
A) grain: the least unit of Storage Resource Management (SRM), its size is specified in the time of resource allocation pond.
B) section: the fixed length resource being made up of multiple continuous grains, comprises data and oppositely mapping.Key feature:
I) reverse mapping table (oppositely map record).The oppositely reverse mapping of all grains in the lasting memory paragraph of mapping table.Oppositely mapping can (the LBA address of hard disc data piece) inquiry obtain virtual address from hard disk address.The reverse map information that writes hard disk comprises: virtual grain number, verification, virtual volume mark, the hard disk grain that discharges number, timestamp.Wherein, hard disk address, hard disk grain Ke You place obtains by hard disk layout calculation, also can be stored in reverse map information.Timestamp is for recording a rise time of oppositely shining upon.
Ii) check information.For checking whether the grain of storage upper layer data and its reverse map information all complete.In the time that system crash recovers, if the verification on hard disk is consistent with the verification of recalculating, thinks that so the grain of storage upper layer data and its reverse map information are all effective, otherwise think that these two is all invalid.Verification obtains by the partial content calculating in corresponding grain, makes probability that verification makes mistakes be less than or equal to the probability of the irrecoverable read error of hard disk.
Iii) asynchronous write of reverse map information and data.Write " section tail " oppositely the IO of map information and the IO that writes idle " grain " in " section " need to meet following order agreement: until after data write and reverse map information writes and all complete, virtual volume ability writes successfully to upper strata return data.
C) bunch: the fixed length resources that are made up of multiple continuous sections, storage pool is to the unit of dynamic log district Resources allocation.Key message comprises: bunch numbering, idle grain quantity, can distribute a quantity, idle grain bitmap, timestamp.
D) hard disk metadata information: read in system bootstrapping process.Key message comprises: physical hard disk information, storage pool information, virtual volume information, map retrieval information, idling-resource information.
3) daily recordization of map information storage.In daily record, store forward mapping table, that reduces forward map information (obtaining hard disk address from virtual address, to distinguish reverse mapping) writes expense and management complexity.
A) write operation generates new map record, first map record is write to the reverse mapping table on hard disk, upgrades the forward mapping table in main memory after success, then returns to write operation success.Asynchronously forward map record is write to daily record by checkpoint thread on backstage.
B) basic comprising of forward mapping table is district mapping (extent mapping), the mapping of reflection continuous integral number " grain ", i.e. mapping between continuous virtual and continuous hard disk grain.Here " continuously " refers to that the numbering of grain is continuous, continuous virtual address numbering and continuous hard disk address number.District's mapping comprises 5 core datas: device flag, physics initial " grain " numbering of physical store resource, virtual volume mark, virtual initial " grain " are numbered, the quantity of continuous " grain ".
C) forward map information delay (Lazy) writes daily record (alignment is write).
I) forward map information is packaged as a granule size " map record ".By map information grouping in main memory, shine upon the space requirement taking and meet certain ratio in all districts in a group, is no more than one " grain ", also not considerably less, adds that packet header can be by one " grain " lower brush.Packet header information comprises: Magic, bag mark, affiliated virtual volume mark, verification.
Ii) value of timestamp in forward " map record ": get the packing moment, all still minimum value of the timestamp of uncompleted amendment forward map operation in this group, represent that the map information that is less than this timestamp in this group stores daily record lastingly into.
Iii), after forward " map record " writes daily record and finishes, reclaim the storage resources of old " map record ".Map record comprises: forward map record, remove map record, record remaps.
4) two-stage checkpoint method, can accelerate the speed of system bootstrapping of the present invention and the speed that collapse recovers.
A) the forward map information of one-level checkpoint stored virtual volume.
B) retrieving information (also can be described as indirect index piece or secondary index) of secondary checkpoint storage forward map information.
5) dynamic log district and management method thereof, can accelerate the speed of resource distribution of the present invention, system bootstrapping and the speed that collapse recovers.
A) region of Resources allocation, dynamic mobile and increase and decrease while writing.Along with resource is distributed, extract from dynamic log area do not have idling-resource bunch.Its idling-resource reduces to after certain proportion, from memory resource pool by strategy be added with idling-resource bunch.
B) retrieval in dynamic log district.In the time adding free cluster to dynamic log district, the composition information in dynamic log district stores hard disk meta-data region into lastingly by a checkpoint thread.
6) mapping retouching operation (as, remove mapping unmap and the remap that remaps) daily recordization storage, can realize Modify rapidly resource mapping and fast quick-recovery.
A) from mapping retouching operation structure " map record ", " map record " write to daily record (alignment is write), after success, upgrade the forward mapping table in main memory, finally return to mapping successfully modified.
B) " map record " of removing mapping comprises following key message: action type (represent um-map), virtual volume mark, the virtual grain number that is disengaged mapping, storage resources place hard disk mark, hard disk grain number, timestamp.Timestamp is used for recording the rise time of one " map record ".
C) " map record " that remap comprises following key message: the virtual grain of the virtual grain of action type (represent re-map), source virtual volume mark, source virtual volume number, target virtual volume mark, target virtual volume number, storage resources place hard disk mark, hard disk grain number, timestamp.Timestamp is used for recording the rise time of one " map record ".
7) timestamp of map record relatively when mapping information recovering.These map records that may participate in comparison comprise: forward mapping, oppositely mapping, releasing are shone upon, remapped.When system is recovered, determine the validity of the map record in dynamic log district by comparing timestamp, the comparison other of its timestamp is the timestamp of forward mapping table.
Related notion of the present invention
Before the concrete enforcement of the present invention is elaborated, first the related notion in the present invention is described.
1) LBA (Logical Block Addressing) (Logical block addressing, LBA), herein also referred to as hard disk address, for representing the position of data block on computer memory device, is generally used for second level storage, for example hard disk.LBA adopts simple linear addressing scheme, and data block is used integer index, and first piece is expressed as LBA0, and second piece is expressed as LBA1, by that analogy.
2) sector (Sector), the base unit of reading writing harddisk or " block device ", fixed size 512B (512 bytes), start address is pressed 512B alignment.
3) grain (Granule), allocation unit and map unit when the writing of fixed size.
4) virtual grain, the expression of the virtual address space of grain on virtual volume.
5) hard disk grain, the expression of the hard disk address space of grain on hard disk.
6) granularity (Granularity), the size of grain.Be generally 64KB, span, from 4KB to 512KB, press 4KB for 4KB size integral multiple and start address and is alignd.
7) section (Segment), comprises one " data segment " and one " section tail ", for upgrading in groups in batches the unit of map information, the time is close to some data that write and upgrades together map information." section tail " is for the write sequence of data optimum performance.
8) data segment (Data Segment), the resource allocation unit of fixed size, comprise 256 " grains " (general size for 16MB), in section, an address space the large piece such as is divided into by " granularity ", and the start address of " section " is alignd by " granularity ".
9) mapping (Map), refers to the mapping of initial virtual address initial hard disk address (LBA) of " grain " to daily record of from virtual volume " grain ", shines upon also referred to as forward.For obtaining hard disk address from virtual address inquiry.
10) mapping range, represent a physics and map information of continuous storage resources all in logic, comprise 5 core datas: hard disk mark, initial " hard disk grain " numbering, virtual volume mark, initial " virtual grain " numbering, the quantity of " grain " continuously.
11) oppositely mapping (Reverse Map), refers to the mapping of initial hard disk address (LBA) initial virtual address of " grain " to virtual volume of " grain " from daily record.For obtaining virtual address from hard disk address lookup.
12) section tail (Segment Tail), also referred to as metadata grain (Meta Granule), the reverse mapping of each " grain " in record " data segment ", the hard disk address (LBA) of each " grain " is to the mapping of virtual address, the size of section tail is the positive integer times of " granularity ", and general size is one " granularity ".
13) remove mapping (Un-map), also referred to as trim or discard." grain " on virtual volume carried out to demapping, will make corresponding storage resources (both " grain " in the daily record) free time.
14) remap (Re-map), the atomic operation sequence being made up of Un-map and Map, realizes " zero-copy " (Zero Copy) migration of the upper storage resources of different virtual volume.Remap and will make a storage resources be mapped to another one virtual volume B from a virtual volume A.
15) storage pool (Pool), manages unappropriated storage resources on all hard disks concentratedly.Newly add the storage resources of hard disk first to add in storage pool, then distributed according to need by storage pool.Idle storage resources is by storage pool unified management and distribution.
16) bunch (Cluster), the unit using to daily record from storage pool memory allocated resource, comprises 256 sections.When the storage resources having distributed in daily record is soon write when full, the present invention from storage pool, distribute one new " bunch " or idling-resource more " bunch " use to daily record.Idling-resource in bunch is by " grain " record.
17) dynamic log district (Dynamic Logging Area), the daily record region that while writing, resource is distributed, this region has bunch forming of idle storage resources by multiple.Along with resource is distributed, extract from dynamic log area do not have idling-resource bunch.Its idling-resource reduces to after certain proportion, from memory resource pool by strategy be added with idling-resource bunch.
18) virtual volume (Volume), by the block device of system manager's designated capabilities, can be from storage pool dynamic on-demand memory allocated resource, the storage resources of distribution may, from different physical hard disks, be available to user's memory device.
19) metadata volume (Meta Volume), for writing daily record by " forward mapping " and amendment thereof as data.In the present invention, metadata volume is corresponding one by one with virtual volume, and each virtual volume has a metadata volume, and each metadata volume has a virtual volume.
The block device that the invention provides a kind of daily record mapping is simplified collocation method, and Fig. 1 is the inventive method schematic flow sheet, and as shown in Figure 1, the method comprises:
Hard disc data layout step S1: by storage resources format, hard disc data layout is carried out to resource pool management, to support to simplify the resource distribution of configuration and the store and management of daily record map information;
Simplify resource allocation management process S2: based on the hard disk of resource pool, the mode of distributing according to need when data are write according to storage resources order writes next idling-resource of hard disk, and generates corresponding write operation map information;
Map information daily record step S3: write operation map information is carried out to daily record storage according to appending allocation scheme, and carry out the retouching operation of map information based on asynchronous refresh mode, write with the data that realize the concurrent order of the many virtual volumes of multi-user.
System bootstrapping and recovering step S4: based on two-stage checkpoint method and dynamic log district management method, that realizes storage resources writes the time-division with addition of speed, and the acceleration of system bootstrapping and collapse recovery.
Wherein, oppositely mapping table comprises: virtual grain number, check information, virtual volume mark, the hard disk grain that discharges number and timestamp, whether described check information completes with the reverse mapping table of described grain for the described grain that checks storage data, the rise time of described timestamp for recording a described reverse map information, described reverse map information and data are taked asynchronous write mode.
Wherein, forward mapping table is stored in main memory, for storing the forward map information that obtains hard disk address from virtual address inquiry, described forward mapping table comprises district's mapping, described district is mapped as the mapping between continuous virtual grain and continuous hard disk grain, and the mapping of described district comprises: physical store resource apparatus mark, the initial grain of physics numbering, virtual volume mark, virtual initial grain numbering and quantity continuously.
Wherein, Fig. 2 A～Fig. 2 E is the inventive method detailed process schematic diagram, as shown in Figure 2 A, simplifies resource allocation management process S2 and comprises:
Write operation step S21: distribute and Journaled allocation strategy while writing based on resource, data are appended in order and write virtual volume, and generate map information;
Read operation step S22: by searching the map information of forward mapping table, carry out data and read.
Wherein, while writing based on resource, distribution and Journaled allocation strategy comprise:
Described write operation is divided into align to write with non-alignment and writes, described alignment is written as and writes the next free block in daily record end, and described non-alignment is written as and writes the daily record position that the last time writes, and that writes as described alignment supplements.
Wherein, as shown in Figure 2 A, write operation step S21 also comprises:
Receive write operation step S211: the data that write are divided into multiple write operation data blocks according to grain, and the data that are grain length for data block length, take the WriteMode that aligns, and are less than the data of grain length for data block length, take non-alignment WriteMode;
Search forward mapping table step S212: the map information of searching data block in the forward mapping table in main memory, if search map information success, judge whether to take the WriteMode that aligns, if search map information failure, distribute new storage resources according to appending WriteMode;
Step S213 is write in alignment: for needs take the to align data block of WriteMode, data block is write to daily record, and the asynchronous end at daily record present segment records virtual volume address information, upgrade forward mapping table information and timestamp in main memory;
Non-alignment is write step S214: for the described data block that need to take non-alignment WriteMode, data block is write to grain at the physical location of last time of daily record, and map information in forward mapping table remains unchanged;
Recycling step S215: reclaim the storage resources losing efficacy, the success of virtual volume write operation.
Wherein, as shown in Figure 2 A, map record daily record step S3 comprises:
Map information is write step S31: the new map information that write operation step is generated writes the reverse mapping table of hard disk, upgrades the forward mapping table being stored in main memory after success, and by the map information asynchronous write daily record of forward mapping table;
Map information postpones write step S32: the grouping of forward map information is packaged as the forward map record of grain size, and the timestamp of forward map record is all forward map information timestamp minimum value in packing moment group;
Map information modify steps S33: the map record that is shone upon and remapped by releasing, discharges the storage resources of virtual volume or carry out mapping relations conversion between virtual volume;
Map information recycling step S34: forward map record writes daily record finish after, reclaim the storage resources of old map record, map record comprises: forward map record, remove map record and the record that remaps.
Wherein, map information modify steps S33 also comprises:
Remove mapping step S331: for discharging the storage resources of virtual volume;
The step that remaps S332: for carry out the conversion of mapping relations between virtual volume.
Wherein, as shown in Figure 2 C, remove mapping step S331 and also comprise:
Receive and remove request step S3311: virtual volume receives the mapping request of removing, wait to remove according to removing mapping acquisition request the mapping range shining upon, and according to the storage space of the required map record of mapping range length computation, be the buffer memory of writing of an integral multiple according to the storage size allocated length of map record;
Search and remove step S3312: the virtual address for each mapping (enum) data piece to be removed in mapping range is searched forward mapping table in main memory, search the corresponding list item that hits forward mapping table, and releasing map information is recorded in and writes buffer memory, and will write under buffer memory and write with a brush dipped in Chinese ink into daily record;
Delete mapping step S3313: the virtual address to mapping (enum) data piece to be removed in mapping range is carried out deletion action, delete the corresponding list item in forward mapping table in main memory, the timestamp that upgrades correspondence mappings interval in forward mapping table is current time, and mapping (enum) data piece to be removed is labeled as to idling-resource.
Wherein, the step S332 that remaps also comprises:
The reception request step S3321 that remaps: virtual volume receives the request of remapping, the mapping range remapping according to the acquisition request that remaps, and according to the storage space of the required map record of mapping range length computation, be the buffer memory of writing of an integral multiple according to the storage size allocated length of map record;
Search mapping step S3322: in main memory, search source virtual volume to be mapped according to each virtual address to be remapped in mapping range, in the virtual volume of source, search the corresponding list item that hits forward mapping table, and carry out target virtual volume to be mapped according to same virtual address and search, in target virtual volume, search the list item that hits forward mapping table, by the operation note of remapping writing in buffer memory, and under write with a brush dipped in Chinese ink into daily record;
Step of updating S3323: search successful list item in the forward mapping table of source virtual volume in deletion main memory, in the virtual volume of renewal source, the timestamp of mapping range is current time, and more fresh target virtual volume search successfully list item, more in fresh target virtual volume, the timestamp of mapping range is current time, and the data block of removing mapping in target virtual volume is labeled as to idling-resource.
Wherein, as shown in Figure 2 A, system bootstrapping and recovering step S4 comprise:
Two-stage checkpoint step S41: by the forward map information of one-level checkpoint stored virtual volume, and by the retrieving information of secondary checkpoint storage forward map information;
Dynamic log district management process S42: the region of Resources allocation when dynamic mobile and increase and decrease write, to realize the dynamic assignment of resource, and realize the retrieval in dynamic log district;
System recovering step S43: by reading forward map information, dynamic log district information and mapping amendment record, completion system recovers.
Wherein, as shown in Figure 2 D, two-stage checkpoint step S41:
Scanning step S411: the information of forward mapping table in main memory is distributed by group, organize into bunch in all mappings, group is scanned;
Step toward division S412: be greater than the specified size of grain if group takes up room, group be split into multiple groups, the group after division is scanned;
Combining step S413: be less than the specified size of grain if group takes up room, group merged, the group after being combined scans;
Recording step S414: if taking up room in the specified size of grain of group distributes the buffer area of a particle size, the mapping range in group and current time stamp are recorded to buffer area;
Discharge recycling step S415: if forward mapping table belongs to metadata volume, the map retrieval district of hard disk head will be brushed under buffer area, and the storage resources of the map record of recovery inefficacy, if forward mapping table belongs to virtual volume, brush metadata volume by under buffer area, and releasing reverse mapping resource.
Wherein, dynamic log district management process S42 also comprises:
Resource allocation step S421 while writing: for writing time-division timing, from dynamic log area Resources allocation;
Dynamic log management process S422: read correlation map information for system, carry out timestamp comparison, by the mapping information recovering of up-to-date timestamp in main memory.
Wherein, system recovering step S43 also comprises:
Read retrieving information step S431: read the forward mapping table on hard disk, load forward mapping table to main memory, read the dynamic log district retrieving information on hard disk, carry out location, dynamic log district;
Scanning step S432: sequential scanning dynamic log district, read mapping amendment record, and according to mapping amendment record inquiry forward mapping table, if the timestamp of mapping amendment record is less than the timestamp of forward mapping table, abandon mapping amendment record, if the timestamp of mapping amendment record is greater than the timestamp of forward mapping table, mapping amendment record is updated to forward mapping table, and reclaims the storage resources of mapping amendment record.
Wherein, as shown in Figure 2 E, while writing, resource allocation step S421 also comprises:
Query resource step S4211: inquiry is when the storage resources of prevariety, if bunch do not have idling-resource, next bunch of inquiry;
Resources allocation step S4212: if in query resource step, inquiry is when available free resource in prevariety, from working as memory allocated resource prevariety;
Check dynamic resource step S4213: check total idling-resource quantity in current dynamic log district, if idling-resource is less than certain threshold value, carry out dynamic log management process.
Wherein, dynamic log management process S422 also comprises:
Search idling-resource step S4221: in memory resource pool sequential search available free bunch, if searched successfully, by bunch updating location information to the dynamic log retrieval in main memory, the buffer area of monthly dynamics log searching section length;
Upgrade bunch step S4222: in buffer area, insert timestamp and dynamic log district bunch relevant information, will under buffer area, brush the dynamic log retrieval district of all hard disks in storage pool, and upgrade the dynamic log district information in main memory;
Notifying process S4223: search in idling-resource step, if search idle bunch failure, output notice user profile.
Below in conjunction with the step that illustrates the specific embodiment of the invention, describe block device in detail and simplify layoutprocedure.
1) data layout on hard disk
Hard disk layout (Layout) manages for adding storage resources to memory resource pool, comprises a head and follow-up linearly aligned multiple bunches, and Fig. 2 is physical hard disk schematic layout pattern of the present invention, as shown in Figure 2.Bunch size fix, general size is 4GB+16MB, wherein 4GB is the space of storing the data on upper strata, the space that 16MB is reverse mapping table.If hard disk last has the space of one bunch of less than, the space at so last end is as a bunch of distribution, the idling-resource amount that the anti-reflection of space, end equals to the space of mapping table bunch.Bunch idling-resource represented by bitmap.
Bootstrapping when the hard disk metadata information of the head storage system that is mainly used in starts, the rapid loading configure storage system of simplifying of the present invention.The renewal frequency of the metadata information of these heads is very low.The general size of head is 32MB, bunch data layout as shown in Figure 3, Fig. 3 is physical hard disk schematic layout pattern of the present invention, comprises these metadata informations: physical hard disk information, storage pool information, virtual volume information, map retrieval information, idling-resource information.Physical hard disk information comprises: the length of the start address of the mark of physical hard disk mark, physical hard disk storage resources quantity, affiliated storage pool, the idling-resource information of this hard disk, the idling-resource information of this hard disk.Storage pool information comprises: storage pool mark, granularity, storage pool title, physical hard disk quantity.Virtual volume information comprises: virtual volume mark, virtual volume title, affiliated storage pool mark, virtual volume capacity, virtual volume type, metadata volume mark.Map retrieval information: mapping indirectly.The position of the available free resource on this hard disk of idling-resource information index.
| Physical hard disk information | Storage pool information | Virtual volume information | Map retrieval information | Idling-resource information |
Table 1 hard disk head layout
Bunch be the unit of Resources allocation from storage pool, Fig. 4 is the present invention's bunch schematic layout pattern, and cluster data layout as shown in Figure 4, comprises consistency check information and middle tactic 256 sections of head and the tail.Bunch general size be 4GB+16MB, large I is calculated from " granularity ".For example, when granularity is 64KB, 256* (16MB+64KB)=4GB+16MB.
Section is the unit of batch updating mapping, Fig. 5 is the schematic layout pattern of section of the present invention, and segment data layout as shown in Figure 5, comprises 257 " grains " of order, front 256 " grains ", for storing the data from upper strata, last 1 " grain " is for the storage reverse mapping of 256 " grains " above.Granularity is 64KB, and section size is generally 16MB+64KB.
2) write operation
In the time writing virtual volume, by daily record mode memory allocated resource, Fig. 6 is embodiment of the present invention write operation steps flow chart schematic diagram, writes flow process as shown in Figure 5:
The data volume writing is divided by " grain ", and write operation can be divided into by " grain " and aligns and be less than or equal to multiple write operations of " granularity ".These write operations can be divided into following two kinds of situations: the first is " alignment is write ", and block size is just that the start address of " granularity " and piece can be divided exactly by " granularity "; The second is " non-alignment is write ", i.e. piece discontented " granularity " and not across two " grains " that address is continuous.In the time that upper strata write operation is not " granularity " integral multiple by " granularity " alignment and size, " non-alignment is write " supplements as a kind of.If exist the upper strata buffer memory can polishing data, make the present invention only use " alignment is write ".
Step S451: first, find the map information of the forward table in main memory of this piece;
Step S452: if the map information of haveing been friends in the past turns S453; Otherwise turn S454;
Step S453: if " alignment is write " turns S454; Otherwise be " non-alignment is write ", turn S458;
Step S454: from dynamic log area memory allocated resource, according to appending WriteMode memory allocated resource;
Step S455: if be allocated successfully, turn S456.Otherwise there is no idle storage resources, wait new hard disk to be added, turn S454;
Step S456: write by being somebody's turn to do " grain " first free block moving after the hard disk sequence of positions of last piece of daily record, record virtual volume address information at the section tail of the current section of daily record asynchronously;
Step S457: upgrade forward table map information and timestamp in main memory;
Step S458: reclaim the storage resources of " old " grain, turn S460;
Step S459: data are write to " grain " at the physical location of the last time of daily record, and maintain the map information of original " grain " in main memory forward table;
Step S460: virtual volume returns to write operation success
3) amendment map operation
In the present invention, revising map operation comprises: remove mapping, remap.Remove map operation and will discharge the storage resources of a virtual volume.The operation of remapping is transformed into the mapping relations of a virtual volume on another one virtual volume.To the operation of these amendment mappings, the present invention first stores daily record into by " map record ", then upgrades the mapping relations buffer memory of main memory, and discharges old storage resources." map record " of removing mapping comprises following key message: action type (represent um-map), virtual volume mark, the virtual grain number that is disengaged mapping, storage resources place hard disk mark, hard disk grain number, timestamp.Timestamp is used for recording the rise time of one " map record ".
31) Fig. 7 is that the embodiment of the present invention is removed mapping step schematic flow sheet, removes mapping flow process as shown in Figure 7:
Step S461: virtual volume A receives and removes mapping request, obtains the mapping range that needs to remove mapping.
Step S462: write buffer memory by required " map record " the required memory allocation of mapping range size.This writes cache size for " granularity " integral multiple, and address is with " granularity " alignment.
Step S463: the virtual address of each piece in mapping range is carried out to following steps:
Step S4631: search forward mapping table.If the not buffer memory in main memory of content of forward mapping table corresponding to virtual address, reads forward mapping table from hard disk so, be loaded in main memory.Then, in main memory, search forward mapping table by the virtual address of mapping block.
Step S4632: if hit forward mapping table searching, find corresponding list item, perform step so S4633, otherwise exit this circulation.
Step S4633: by removing the mark, virtual device number of map operation, virtual grain number, hard reel number, hard disk grain number, current time stamp etc., be recorded to the above-mentioned buffer memory of writing.
Step S464: above-mentioned writing under buffer memory write with a brush dipped in Chinese ink into daily record, and wiring method is according to the flow process of writing shown in above-mentioned Fig. 6.
Step S465: after above-mentioned write operation completes, the virtual address of each piece in mapping range is carried out to following steps:
Step S4651: delete the corresponding list item finding in main memory forward mapping table.
Step S4652: the timestamp that upgrades corresponding " mapping range " in main memory forward mapping table is current time.
Step S4653: in idling-resource bitmap, the piece that this piece of mark is corresponding is idling-resource.
Step S466: virtual volume A returns to remove and shines upon successfully.
32) Fig. 8 is the embodiment of the present invention steps flow chart schematic diagram that remaps, and remaps flow process as shown in Figure 8:
" map record " that remap comprises following key message: the virtual grain of the virtual grain of action type (represent re-map), source virtual volume mark, source virtual volume number, target virtual volume mark, target virtual volume number, storage resources place hard disk mark, hard disk grain number, timestamp.Timestamp is used for recording the rise time of one " map record ".
Step S471: virtual volume A receives the request of remapping, the mapping range that acquisition need to be remapped.
Step S472: distribute and write buffer memory by mapping range size.This writes cache size for " granularity " integral multiple, and address is with " granularity " alignment.
Step S473: the virtual address of each piece in mapping range is carried out to following steps:
Step S4731: the forward mapping table of searching " source book ".If the not buffer memory in main memory of content of forward mapping table corresponding to virtual address, reads the forward mapping table of source book so from hard disk, be loaded in main memory.Then, in main memory, search forward mapping table by the virtual address of mapping block.
Step S4732: if hit searching in source book forward mapping table process, find corresponding list item, perform step so S4733, otherwise exit this circulation.
Step S4733: the forward mapping table of searching " target volume " according to same virtual address.If the not buffer memory in main memory of content of forward mapping table corresponding to virtual address, reads the forward mapping table of target volume so from hard disk, be loaded in main memory.Then, in main memory, search forward mapping table by the virtual address of mapping block.
Step S4734: if hit searching in target volume forward mapping table process, find corresponding list item, perform step so S4735, otherwise exit this circulation.
Step S4735: by the virtual grain in the virtual reel number in mark, source, source of the operation of remapping number, the virtual reel number of target, the virtual grain of target number, hard reel number, hard disk grain number, current time stamp etc., be recorded to the above-mentioned buffer memory of writing.
Step S474: above-mentioned writing under buffer memory write with a brush dipped in Chinese ink into daily record, and wiring method is according to the flow process of writing of above-mentioned Fig. 6.
Step S475: after above-mentioned bag write operation completes, the virtual address of each piece in mapping range is carried out to following steps:
A) delete the corresponding list item finding in the source book forward mapping table in main memory.
B) in the source book forward mapping table in renewal main memory, the timestamp of corresponding " mapping range " is current time.
C) upgrade the corresponding list item of finding in the target volume forward mapping table in main memory, point to the hard disk piece of source book.
D) in the target volume forward mapping table in renewal main memory, the timestamp of corresponding " mapping range " is current time.
E), in idling-resource bitmap, old of the upper correspondence of target-marking volume is idling-resource.
Step S476: virtual volume A returns and remaps successfully.
4) read operation
A), while reading from virtual volume, first search the forward table cache in main memory.
If b) search missly, read forward table from daily record so, be then loaded into main memory.Returning to step S481 searches again.
If c) find map information, be redirected read operation to actual hard disk position according to map information so;
D) otherwise, do not find map information, return to null value.
5) write map information and two-stage checkpoint
Shown in earlier figures 6, step S457 in embodiment of the present invention write operation flow process, upgrades the map information of the forward table in main memory in the call back function that write operation completes, and is about to a new mapping range and adds in forward table or merge in forward table.Upgrade the flow process of forward table map information in main memory as shown in Figure 9, Fig. 9 is that the embodiment of the present invention is upgraded map information steps flow chart schematic diagram, and the flow process of upgrading map information is as follows:
Step S491: after granularity has been write, by hard disk down trigger call back function.Typical case as, the bi_end_io () of bio in Linux.
Step S492: create new " mapping range " memory object according to " the hard disk address " of writing position and " virtual address ".
Step S493: whether comprise corresponding mapping according to forward table in " virtual address " inquiry main memory.
Step S494: if main memory forward table comprises mapping, attempt so the mapping of new mapping range object encoding to merge in main memory forward table, then go to step S496.Otherwise, execution step S495.
Step S495: if do not find correspondence mappings in main memory forward table, so new mapping range object is added in main memory forward table.
Step S496: return to forward table and upgraded.
6) mapping is appended and is write and two-stage checkpoint
The map information that writes " metadata volume " appends and writes hard disk, and map information is processed through two-stage checkpoint, and Figure 10 is that embodiment of the present invention mapping is appended and write and two-stage checkpoint steps flow chart schematic diagram, as shown in figure 10.Step 1,2,3,4 represents data writing volume, adopts the flow process of writing shown in Fig. 6.Step 5,6,7,8,9 represents first order check point, and the map information of " book " writes " metadata volume ", also adopts the flow process of writing shown in Fig. 6.This map information writes the mode of hard disk and has avoided the synchronous read operation of old map information and reduced storage delay.
Metadata volume is a kind of virtual volume, and storing process is the same with common virtual volume, adopts the flow process of writing shown in Fig. 6, the same flow process of reading.
Step 10 and 11 represents second level check point, and the map information of metadata volume is saved in the fixing meta-data region in the address of all physical equipments in storage pool by checkpoint, second level flow process, needs to reclaim the storage resources of inefficacy map record after having write.
In check point flow process, in main memory, the map information of forward table carries out lower brush by group (Group).One " group " represent one " bunch " in all mappings, " mapping range " wherein pressed " Kd-Trees " tissue.When checkpoint flow process be scheduled when operation is scanned in main memory " group " all in forward table, and attempt by each " group " take up room be controlled at " granularity " 1/4 to 3/4, larger " group " is split into two " groups ", and maybe may merge two less " groups " is one " group ".If the size of " group " meets afore mentioned rules, checkpoint flow process will be distributed the buffer area of " granularity " so, the mapping range being somebody's turn to do in " group " is recorded to this buffer area, and in this buffer area, record current timestamp, then will under this buffer area, brush " metadata volume ".
Figure 11 is embodiment of the present invention two-stage checkpoint steps flow chart schematic diagram, the two-stage checkpoint flow process of map information as shown in figure 11:
Step S502: all " groups " in forward table in scanning main memory.Scan " group " of current sensing.If current " group " is empty, exit so.
Step S503: if one " group " take up room be greater than " granularity " 3/4, this " group " is split into two " groups " so.Scan pointer points to first " group " after division.Go to step S502." group " if of two adjacency take up room be all less than " granularity " 1/4, this two " group " merges into one " group " so.Scan pointer points to " group " after merging.Go to step S502;
Step S504: if the size of " group " " granularity " 1/4～3/4 between, distribute so the buffer area of " granularity ".
Step S505: the mapping range being somebody's turn to do in " group " is recorded to this buffer area, and records current timestamp in this buffer area.Timestamp is got the minimum value of all " mapping range " timestamps in this " group ".
Step S506: if the mapping of this forward table record " book " goes to step S507 so.Otherwise the mapping of this forward table record " metadata volume ", goes to step S509 so.
Step S507: will brush " metadata volume " under this buffer area, adopt the flow process of writing in Fig. 6.
Step S508: the reverse mapping object that discharges buffer memory in main memory.Scan pointer points to the next one " group " in main memory.Go to step S502.
Step S509: will brush the map retrieval district that is positioned at hard disk head under this buffer area.
Step S510: reclaim grain corresponding to old map information on hard disk, these grains are not retrieved.Go to step 502.
The retrieving information of mapping stores a fixed address region of hard disk into.In the time that system crash recovers, read these retrieving informations from these fixed address, recover accordingly the map information in main memory, finally scan dynamic log district, recover the map information in dynamic log district.
7) dynamic log district management method
In the process in scanning dynamic log district, system reads all map informations and a section tail effective map information, relatively their timestamp, by the mapping information recovering of up-to-date (timestamp value is large) in main memory.Effectively map information checks by verification, and the consistent map information of verification is effective.
71) dynamic log district size dynamic change in ablation process.Writing time-division timing, will be from dynamic log area Resources allocation, Figure 12 is embodiment of the present invention resource allocation flow schematic diagram while writing, as shown in figure 12, the flow process that while writing, resource is distributed is as follows:
Step S521: inquiry is from working as prevariety memory allocated resource.
Step S522: if there is no idling-resource when prevariety, point to so next available free storage resources in dynamic log district bunch, go to step S521.Otherwise execution step S523.
Step S523: from distribute " the hard disk grain " of sufficient amount when prevariety.
Step S524: the total idling-resource quantity that checks current dynamic log district.If idling-resource is less than certain threshold value, wake so dynamic log district management process up.
Step S525: the pointer that returns to " the hard disk grain " that point to above-mentioned distribution.
72) as shown in figure 13, Figure 13 is embodiment of the present invention dynamic log district management process schematic flow sheet to dynamic log district management process, and flow process is as follows:
Step S531: according to the data layout of above-mentioned hard disk search the next one have idling-resource bunch.
Step S532: if find have idling-resource bunch, perform step so S533, otherwise execution step S537.
Step S533: the updating location information when prevariety is retrieved to the dynamic log in main memory.
Step S534: the buffer area that distributes the dynamic log retrieval district size of a hard disk head.
Step S535: by current time stamp and all current dynamic logs district bunch relevant information be recorded to above-mentioned buffer area.Bunch relevant information comprise: quantity in dynamic log district bunch, bunch mark, the mark of bunch place hard disk etc.
Step S536: will brush the dynamic log retrieval district of all hard disks in memory resource pool under above-mentioned buffer area.Go to step S538.
Step S537: notify user there is no idle storage resources.
Step S538: exit dynamic log district management process.
Dynamic log retrieval district is positioned at the fixed address region on hard disk.In the time that system is recovered, read these retrieving informations from these fixed address, the dynamic log district while finding system crash accordingly, scanning dynamic log district, recovers the map information in dynamic log district.
8) system recovery method
After system crash, Figure 14 be embodiment of the present invention system recover schematic flow sheet, recovery flow process of the present invention as shown in figure 14:
Step S541: read the forward map retrieval information of hard disk meta-data region, read accordingly the forward mapping table on hard disk, load forward mapping table to main memory;
Step S542: read the dynamic log district retrieving information of hard disk meta-data region, locate accordingly dynamic log district;
Step S543: sequential scanning dynamic log district, read mapping amendment record (comprising: oppositely mapping, releasing are shone upon, remapped) wherein, these mapping amendment records are carried out to following processing;
Step S5431: the virtual grain number of pressing mapping amendment record is inquired about the forward mapping table in main memory;
Step S5432: if the timestamp of mapping amendment record is less than the timestamp of forward mapping table, abandon so this mapping amendment record; Otherwise, mapping amendment record is updated to the forward mapping table in main memory;
Step S5433: the storage resources that reclaims mapping amendment record.
Step S544: system has been recovered.
In sum, allocation strategy and the storage system of simplifying two kinds of technology of configuration demand Resource management when a kind of combination log store resource provided by the invention is write, the data block of having distributed when having realized data and map information and writing by strategy, the map record of usage logization storage is optimized write performance and the amendment performance of map information, the speed loading while carrying out optimization system bootstrapping with checkpoint.The present invention has realized quick daily recordization ground Storage Mapping record, can in the time of the concurrent sequential write of the many virtual volumes of multi-user, make full use of hard disk bandwidth, realizes distribution according to need storage resources and maximizes utilization ratio of storage resources.
Certainly; the present invention also can have other various embodiments; in the situation that not deviating from spirit of the present invention and essence thereof; those of ordinary skill in the art are when making according to the present invention various corresponding changes and distortion, but these corresponding changes and distortion all should belong to the protection domain of the appended claim of the present invention.
Claims (18)
1. the block device of daily record mapping is simplified a collocation method, it is characterized in that, described method comprises:
Hard disc data layout step: by storage resources format, hard disc data layout is carried out to resource pool management, to support to simplify the resource distribution of configuration and the store and management of daily record map information;
Simplify resource allocation management process: based on the hard disk of described resource pool, the mode of distributing according to need when data are write according to storage resources order writes next idling-resource of described hard disk, and generates the corresponding map information of write operation;
Map information daily record step: described write operation map information is carried out to daily record storage according to appending allocation scheme, and carry out the retouching operation of described map information based on asynchronous refresh mode, write with the data that realize the concurrent order of the many virtual volumes of multi-user.
2. the block device of daily record mapping is simplified collocation method according to claim 1, it is characterized in that, described method also comprises:
System bootstrapping and recovering step: based on two-stage checkpoint method and dynamic log district management method, that realizes described storage resources writes the time-division with addition of speed, and the fast quick-recovery of system bootstrapping and the fast quick-recovery of system crash.
3. the block device of daily record mapping is simplified collocation method according to claim 1, it is characterized in that, described unit comprises:
Grain: for the least unit of Storage Resource Management (SRM);
Section: the fixed length resource being made up of the multiple continuous described grain of fixed qty, comprises: data and oppositely mapping table.Described reverse mapping table is for described section of reverse map information for the described grain of data of lasting storage, and described reverse map information is for obtaining virtual address from hard disk address lookup;
Bunch: by the described section of multiple continuous fixed length resources that form of fixed qty, storage pool, to the unit of dynamic log district Resources allocation, comprising: bunch numbering, idle grain quantity, can distribute a quantity, idle grain bitmap and timestamp;
Hard disk metadata information: the data message reading for described system bootstrapping and recovering step, comprising: physical hard disk information, storage pool information, virtual volume information, map retrieval information and idling-resource information;
Metadata volume: for forward map information and the retouching operation thereof of forward mapping table are write to daily record, described metadata volume is corresponding one by one with virtual volume.
4. the block device of daily record mapping is simplified collocation method according to claim 3, it is characterized in that, described reverse mapping table comprises: virtual grain number, check information, virtual volume mark, the hard disk grain that discharges number and timestamp, whether described check information completes with the reverse mapping table of described grain for the described grain that checks storage data, the rise time of described timestamp for recording a described reverse map information, described reverse map information and data are taked asynchronous write mode.
5. the block device of daily record mapping is simplified collocation method according to claim 3, it is characterized in that, described forward mapping table is stored in main memory, for storing the forward map information that obtains hard disk address from virtual address inquiry, described forward mapping table comprises district's mapping, described district is mapped as the mapping between continuous virtual grain and continuous hard disk grain, and the mapping of described district comprises: physical store resource apparatus mark, the initial grain of physics numbering, virtual volume mark, virtual initial grain numbering and quantity continuously.
6. the block device of daily record mapping is simplified collocation method according to claim 1, it is characterized in that, described in simplify resource allocation management process and comprise:
Write operation step: distribute and Journaled allocation strategy while writing based on resource, data are appended in order and write hard disk, and generate map information;
Read operation step: by searching the map information of described forward mapping table, carry out data and read.
7. the block device of daily record mapping is simplified collocation method according to claim 1, it is characterized in that, described map record daily record step comprises:
Map information is write step: the new map information that described write operation step is generated writes the described reverse mapping table of hard disk, upgrades the described forward mapping table being stored in main memory after success, and by the map information asynchronous write daily record of described forward mapping table;
Map information postpones write step: described forward map information grouping is packaged as the forward map record of described grain size, and the timestamp of described forward map record is all described forward map information timestamp minimum value in packing moment group;
Map information modify steps: the described map record that is shone upon and remapped by releasing, discharges the storage resources of virtual volume or carry out mapping relations conversion between described virtual volume;
Map information recycling step: forward map record writes daily record finish after, reclaim the storage resources of old map record, described map record comprises: described forward map record, remove map record and the record that remaps.
8. the block device of daily record mapping is simplified collocation method according to claim 6, it is characterized in that, described distribute while writing based on resource and Journaled allocation strategy comprises:
Described write operation is divided into align to write with non-alignment and writes, described alignment is written as and writes the next free block in daily record end, and described non-alignment is written as and writes the daily record position that the last time writes, and that writes as described alignment supplements.
9. the block device of daily record mapping is simplified collocation method according to claim 2, it is characterized in that, described system bootstrapping and recovering step comprise:
Two-stage checkpoint step: by the forward map information of one-level checkpoint stored virtual volume, and store the retrieving information of described forward map information by secondary checkpoint;
Dynamic log district management process: the region of Resources allocation when dynamic mobile and increase and decrease write, to realize the dynamic assignment of resource, and realize the retrieval in dynamic log district;
System recovering step: by reading forward map information, dynamic log district information and mapping amendment record, completion system recovers.
10. the block device of daily record mapping is simplified collocation method according to claim 6, it is characterized in that, described write operation step also comprises:
Receive write operation step: the data that write are divided into multiple write operation data blocks according to described grain, the data that are described grain length for described data block length, take described alignment WriteMode, be less than the data of described grain length for described data block length, take described non-alignment WriteMode;
Search forward mapping table step: the map information of searching described data block in the described forward mapping table in main memory, if search described map information success, judge whether to take described alignment WriteMode, if search described map information failure, distribute new storage resources according to the described WriteMode that appends;
Step is write in alignment: for the described data block that need to take described alignment WriteMode, described data block is write to first free block after writing position last time in daily record, and virtual volume address information is recorded, forward mapping table information and timestamp described in renewal main memory in the asynchronous end at daily record present segment;
Non-alignment is write step: for the described data block that need to take described non-alignment WriteMode, described data block is write to described grain at the physical location of the last time of daily record, and map information in described forward mapping table remains unchanged;
Recycling step: reclaim the storage resources losing efficacy, the success of virtual volume write operation.
11. block devices that daily record is shone upon are according to claim 7 simplified collocation method, it is characterized in that, described map information modify steps also comprises:
Remove mapping step: for discharging the storage resources of described virtual volume;
Step remaps: for carry out the conversion of mapping relations between described virtual volume.
12. simplify collocation method according to the block device of daily record mapping described in claim 11, it is characterized in that, described releasing mapping step also comprises:
Receive and remove request step: virtual volume receives the mapping request of removing, wait to remove the mapping range of mapping according to described releasing mapping acquisition request, and according to the storage space of the required map record of described mapping range length computation, be the buffer memory of writing of described grain integral multiple according to the memory allocation length of described map record;
Search releasing step: in main memory, search described forward mapping table for the virtual address of each mapping (enum) data piece to be removed in described mapping range, search the corresponding list item that hits described forward mapping table, and write buffer memory described in releasing map information is recorded in, and write under buffer memory and write with a brush dipped in Chinese ink into daily record described in inciting somebody to action;
Delete mapping step: the virtual address to mapping (enum) data piece to be removed described in described mapping range is carried out deletion action, delete the corresponding list item in forward mapping table described in main memory, the timestamp that upgrades correspondence mappings interval in described forward mapping table is current time, and described in inciting somebody to action, mapping (enum) data piece to be removed is labeled as idling-resource.
13. simplify collocation method according to the block devices of daily record mapping described in claim 11, it is characterized in that, described in the step that remaps also comprise:
The reception request step that remaps: virtual volume receives the request of remapping, the mapping range remapping according to the described acquisition request that remaps, and according to the storage space of the required map record of described mapping range length computation, be the buffer memory of writing of described grain integral multiple according to the memory allocation length of described map record;
Search mapping step: in main memory, search source virtual volume to be mapped according to each virtual address to be remapped in described mapping range, in the virtual volume of described source, search the corresponding list item that hits described forward mapping table, and carry out target virtual volume to be mapped according to same virtual address and search, in described target virtual volume, search the list item that hits described forward mapping table, the operation note of remapping is write in buffer memory described, and under write with a brush dipped in Chinese ink into daily record;
Step of updating: search successful list item in the forward mapping table of source virtual volume described in deletion main memory, the timestamp that upgrades mapping range in the virtual volume of described source is current time, and upgrade the list item of searching successfully of described target virtual volume, the timestamp that upgrades mapping range in described target virtual volume is current time, and the data block of removing mapping in described target virtual volume is labeled as to idling-resource.
14. block devices that daily record is shone upon are according to claim 9 simplified collocation method, it is characterized in that, described two-stage checkpoint step also comprises:
Scanning step: the information of the mapping table of forward described in main memory is distributed by group, and described group is all mappings in described bunch, and described group is scanned;
Step toward division: if described group takes up room and be greater than the specified size of described grain, be split into multiple groups by described group, described group after division is scanned;
Combining step: if described group takes up room and be less than the specified size of described grain, described group is merged, described group after being combined scans;
Recording step: if described group take up room in the specified size of described grain distributes the buffer area of a described particle size, the mapping range in described group and current time stamp are recorded to described buffer area;
Discharge recycling step: if described forward mapping table belongs to described metadata volume, the map retrieval district of described hard disk head will be brushed under described buffer area, and the storage resources of the map record of recovery inefficacy, if described forward mapping table belongs to described virtual volume, will under described buffer area, brush described metadata volume, and releasing reverse mapping resource.
15. block devices that daily record is shone upon are according to claim 9 simplified collocation method, it is characterized in that, described dynamic log district management process also comprises:
Resource allocation step while writing: for writing time-division timing, from dynamic log area Resources allocation;
Dynamic log management process: while recovery for system, read correlation map information, carry out timestamp comparison, by the mapping information recovering of up-to-date timestamp in main memory.
16. block devices that daily record is shone upon are according to claim 9 simplified collocation method, it is characterized in that, described system recovering step also comprises:
Read retrieving information step: read the forward mapping table on hard disk, load described forward mapping table to main memory, read the dynamic log district retrieving information on described hard disk, carry out location, dynamic log district;
Scanning step: dynamic log district described in sequential scanning, read mapping amendment record, and according to described mapping amendment record inquiry forward mapping table, if the timestamp of described mapping amendment record is less than the timestamp of described forward mapping table, abandon described mapping amendment record, if the timestamp of described mapping amendment record is greater than the timestamp of described forward mapping table, described mapping amendment record is updated to described forward mapping table, and reclaims the storage resources of described mapping amendment record.
17. simplify collocation method according to the block devices of daily record mapping described in claim 15, it is characterized in that, described in while writing resource allocation step also comprise:
Query resource step: inquiry, when the storage resources of prevariety, if described bunch does not have idling-resource, is inquired about next described bunch;
Resources allocation step: if in described query resource step, inquiry is described when available free resource in prevariety, from described when memory allocated resource prevariety;
Check dynamic resource step: check total idling-resource quantity in current dynamic log district, if described idling-resource is less than certain threshold value, carry out described dynamic log management process.
18. simplify collocation method according to the block device of daily record mapping described in claim 15, it is characterized in that, described dynamic log management process also comprises:
Search idling-resource step: available free described bunch of sequential search in memory resource pool, if searched successfully, by the updating location information of described bunch to the dynamic log retrieval in main memory, the buffer area of monthly dynamics log searching section length;
Upgrade bunch step: in described buffer area, insert timestamp and described dynamic log district bunch relevant information, will under buffer area, brush the dynamic log retrieval district of all hard disks in storage pool, and upgrade the dynamic log district information in main memory;
Notifying process: described in search in idling-resource step, if search described bunch of idle failure, output notice user profile.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410218798.7A CN104035729B (en) | 2014-05-22 | 2014-05-22 | Block device thin-provisioning method for log mapping |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410218798.7A CN104035729B (en) | 2014-05-22 | 2014-05-22 | Block device thin-provisioning method for log mapping |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104035729A true CN104035729A (en) | 2014-09-10 |
| CN104035729B CN104035729B (en) | 2017-02-15 |
Family
ID=51466507
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410218798.7A Expired - Fee Related CN104035729B (en) | 2014-05-22 | 2014-05-22 | Block device thin-provisioning method for log mapping |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104035729B (en) |
Cited By (36)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104391954A (en) * | 2014-11-27 | 2015-03-04 | 北京国双科技有限公司 | Database log processing method and device |
| CN104461925A (en) * | 2014-11-14 | 2015-03-25 | 浪潮(北京)电子信息产业有限公司 | Automatic correcting method and device for aligning storage device addresses |
| CN104731517A (en) * | 2013-12-19 | 2015-06-24 | 中国移动通信集团四川有限公司 | Method and device for allocating capacity of storage pool |
| CN105095111A (en) * | 2015-07-17 | 2015-11-25 | 浪潮(北京)电子信息产业有限公司 | Method for improving safety of storage virtualization and storage virtualization device |
| CN105912277A (en) * | 2016-04-29 | 2016-08-31 | 上海交通大学 | Method for realizing single-writing log of file system in virtual machine mirror image |
| CN106293543A (en) * | 2016-08-25 | 2017-01-04 | 中国科学院计算技术研究所 | A kind of based on incidentally reclaim watt record a RAID write ordered method and device |
| CN106648457A (en) * | 2016-09-27 | 2017-05-10 | 华为数字技术（成都）有限公司 | Method of updating reverse mapping metadata and device |
| CN107003942A (en) * | 2014-10-27 | 2017-08-01 | 桑迪士克科技有限责任公司 | To for strengthening the performance of storage device and the processing of persistent unmapped order |
| CN107408022A (en) * | 2015-03-27 | 2017-11-28 | 英特尔公司 | Manage the previous version of the data of the logical address in storage device |
| CN107438829A (en) * | 2015-04-08 | 2017-12-05 | 华为技术有限公司 | Partitioned storage data set redoes log record |
| CN107436727A (en) * | 2016-05-26 | 2017-12-05 | 中兴通讯股份有限公司 | The memory space management and device of simplification volume |
| CN107515827A (en) * | 2017-08-21 | 2017-12-26 | 湖南国科微电子股份有限公司 | Storage method, device and the SSD of the self-defined daily records of PCIE SSD |
| CN107544894A (en) * | 2016-06-23 | 2018-01-05 | 中兴通讯股份有限公司 | The method, apparatus and server of a kind of log processing |
| CN107665156A (en) * | 2016-07-27 | 2018-02-06 | 西部数据技术公司 | Multithread daily record is replayed |
| WO2018040590A1 (en) * | 2016-09-05 | 2018-03-08 | 华为技术有限公司 | Space allocation method and apparatus for file system |
| CN108255740A (en) * | 2017-12-07 | 2018-07-06 | 深圳市中易通安全芯科技有限公司 | A kind of FLASH equilibriums erasing-writing method and system |
| CN109358818A (en) * | 2018-10-30 | 2019-02-19 | 深圳润迅数据通信有限公司 | A kind of block device I/O Request processing method of data center |
| WO2019048969A1 (en) * | 2017-09-05 | 2019-03-14 | International Business Machines Corporation | Asynchronous update of metadata tracks in response to a cache hit generated via an i/o operation over a bus interface |
| CN109901799A (en) * | 2019-02-28 | 2019-06-18 | 新华三信息安全技术有限公司 | A kind of log reading/writing method and device |
| TWI667573B (en) * | 2017-09-30 | 2019-08-01 | 英屬開曼群島商捷鼎創新股份有限公司 | Distributed storage device |
| CN110515861A (en) * | 2018-05-21 | 2019-11-29 | 北京忆芯科技有限公司 | Handle the storage device and method thereof of flash command |
| CN111125033A (en) * | 2018-10-31 | 2020-05-08 | 深信服科技股份有限公司 | Space recovery method and system based on full flash memory array |
| CN111176570A (en) * | 2019-12-27 | 2020-05-19 | 浪潮电子信息产业股份有限公司 | Thick backup roll creating method, device, equipment and medium |
| CN111190537A (en) * | 2019-12-10 | 2020-05-22 | 优刻得科技股份有限公司 | Method and system for managing sequential storage disks in write-addition scene |
| CN111273861A (en) * | 2020-01-15 | 2020-06-12 | 苏州浪潮智能科技有限公司 | Data volume Bitmap destaging method and system |
| CN111803917A (en) * | 2020-06-30 | 2020-10-23 | 北京金山云网络技术有限公司 | Resource processing method and device |
| CN111966655A (en) * | 2020-08-18 | 2020-11-20 | 中国银行股份有限公司 | Method and device for managing file objects in memory in log collection process |
| CN112162701A (en) * | 2020-09-18 | 2021-01-01 | 北京浪潮数据技术有限公司 | Storage space recovery method, device, equipment and computer storage medium |
| CN112667156A (en) * | 2020-12-25 | 2021-04-16 | 深圳创新科技术有限公司 | Method and device for realizing virtualization raid |
| CN112685337A (en) * | 2021-01-15 | 2021-04-20 | 浪潮云信息技术股份公司 | Method for hierarchically caching read and write data in storage cluster |
| CN112860594A (en) * | 2021-01-21 | 2021-05-28 | 华中科技大学 | Solid-state disk address remapping method and device and solid-state disk |
| CN113835639A (en) * | 2021-09-26 | 2021-12-24 | 深圳大普微电子科技有限公司 | I/O request processing method, device, equipment and readable storage medium |
| CN113885808A (en) * | 2021-10-28 | 2022-01-04 | 合肥兆芯电子有限公司 | Mapping information recording method, memory control circuit unit and memory device |
| CN115657963A (en) * | 2022-11-14 | 2023-01-31 | 苏州浪潮智能科技有限公司 | Sequential writing method and device based on solid state disk, electronic equipment and storage medium |
| CN117390072A (en) * | 2023-12-07 | 2024-01-12 | 深圳市云希谷科技有限公司 | Method for improving network request speed in embedded system |
| CN117785565A (en) * | 2024-02-23 | 2024-03-29 | 深圳市创世飞达科技有限公司 | Solid state storage hard disk data recovery method and solid state hard disk |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7711897B1 (en) * | 2005-06-10 | 2010-05-04 | American Megatrends, Inc. | Method, system, apparatus, and computer-readable medium for improving disk array performance |
| CN102722449B (en) * | 2012-05-24 | 2015-01-21 | 中国科学院计算技术研究所 | Key-Value local storage method and system based on solid state disk (SSD) |
| CN102968279B (en) * | 2012-11-13 | 2016-06-08 | 浪潮电子信息产业股份有限公司 | A kind of store the method that system simplifies configuration automatically |
| CN103077187A (en) * | 2012-12-20 | 2013-05-01 | 曙光信息产业(北京)有限公司 | Log management method and log management system |
-
2014
- 2014-05-22 CN CN201410218798.7A patent/CN104035729B/en not_active Expired - Fee Related
Non-Patent Citations (1)
| Title |
|---|
| 王彦龙等: "设备复制系统的日志机制研究", 《计算机工程与应用》 * |
Cited By (60)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104731517A (en) * | 2013-12-19 | 2015-06-24 | 中国移动通信集团四川有限公司 | Method and device for allocating capacity of storage pool |
| CN107003942B (en) * | 2014-10-27 | 2020-09-22 | 桑迪士克科技有限责任公司 | Processing of unmap commands to enhance performance and persistence of storage devices |
| CN107003942A (en) * | 2014-10-27 | 2017-08-01 | 桑迪士克科技有限责任公司 | To for strengthening the performance of storage device and the processing of persistent unmapped order |
| CN104461925A (en) * | 2014-11-14 | 2015-03-25 | 浪潮(北京)电子信息产业有限公司 | Automatic correcting method and device for aligning storage device addresses |
| CN104461925B (en) * | 2014-11-14 | 2017-10-13 | 浪潮(北京)电子信息产业有限公司 | A kind of method for automatically correcting and device of storage device address align |
| CN104391954A (en) * | 2014-11-27 | 2015-03-04 | 北京国双科技有限公司 | Database log processing method and device |
| CN104391954B (en) * | 2014-11-27 | 2019-04-09 | 北京国双科技有限公司 | The processing method and processing device of database journal |
| CN107408022B (en) * | 2015-03-27 | 2021-10-26 | 英特尔公司 | Managing previous versions of data for logical addresses in a storage device |
| CN107408022A (en) * | 2015-03-27 | 2017-11-28 | 英特尔公司 | Manage the previous version of the data of the logical address in storage device |
| CN107438829A (en) * | 2015-04-08 | 2017-12-05 | 华为技术有限公司 | Partitioned storage data set redoes log record |
| CN105095111B (en) * | 2015-07-17 | 2018-08-03 | 浪潮(北京)电子信息产业有限公司 | A kind of method and memory virtual equipment improving Storage Virtualization safety |
| CN105095111A (en) * | 2015-07-17 | 2015-11-25 | 浪潮(北京)电子信息产业有限公司 | Method for improving safety of storage virtualization and storage virtualization device |
| CN105912277B (en) * | 2016-04-29 | 2018-12-18 | 上海交通大学 | The method that file system list writes log is realized in virtual machine image |
| CN105912277A (en) * | 2016-04-29 | 2016-08-31 | 上海交通大学 | Method for realizing single-writing log of file system in virtual machine mirror image |
| CN107436727A (en) * | 2016-05-26 | 2017-12-05 | 中兴通讯股份有限公司 | The memory space management and device of simplification volume |
| CN107544894A (en) * | 2016-06-23 | 2018-01-05 | 中兴通讯股份有限公司 | The method, apparatus and server of a kind of log processing |
| US11182091B2 (en) | 2016-07-27 | 2021-11-23 | Western Digital Technologies, Inc. | Multi-stream journaled replay |
| CN107665156A (en) * | 2016-07-27 | 2018-02-06 | 西部数据技术公司 | Multithread daily record is replayed |
| CN106293543A (en) * | 2016-08-25 | 2017-01-04 | 中国科学院计算技术研究所 | A kind of based on incidentally reclaim watt record a RAID write ordered method and device |
| WO2018040590A1 (en) * | 2016-09-05 | 2018-03-08 | 华为技术有限公司 | Space allocation method and apparatus for file system |
| CN106648457B (en) * | 2016-09-27 | 2019-09-03 | 华为数字技术（成都）有限公司 | Update the method and device of back mapping metadata |
| CN106648457A (en) * | 2016-09-27 | 2017-05-10 | 华为数字技术（成都）有限公司 | Method of updating reverse mapping metadata and device |
| CN107515827A (en) * | 2017-08-21 | 2017-12-26 | 湖南国科微电子股份有限公司 | Storage method, device and the SSD of the self-defined daily records of PCIE SSD |
| CN107515827B (en) * | 2017-08-21 | 2021-07-27 | 湖南国科微电子股份有限公司 | PCIE SSD custom log storage method and device and SSD |
| GB2579754B (en) * | 2017-09-05 | 2020-12-02 | Ibm | Asynchronous update of metadata tracks in response to a cache hit generated via an I/O operation over a bus interface |
| GB2579754A (en) * | 2017-09-05 | 2020-07-01 | Ibm | Asynchronous update of metadata tracks in response to a cache hit generated via an I/O operation over a bus interface |
| US10565109B2 (en) | 2017-09-05 | 2020-02-18 | International Business Machines Corporation | Asynchronous update of metadata tracks in response to a cache hit generated via an I/O operation over a bus interface |
| WO2019048969A1 (en) * | 2017-09-05 | 2019-03-14 | International Business Machines Corporation | Asynchronous update of metadata tracks in response to a cache hit generated via an i/o operation over a bus interface |
| US11010295B2 (en) | 2017-09-05 | 2021-05-18 | International Business Machines Corporation | Asynchronous update of metadata tracks in response to a cache hit generated via an i/o operation over a bus interface |
| TWI667573B (en) * | 2017-09-30 | 2019-08-01 | 英屬開曼群島商捷鼎創新股份有限公司 | Distributed storage device |
| CN108255740A (en) * | 2017-12-07 | 2018-07-06 | 深圳市中易通安全芯科技有限公司 | A kind of FLASH equilibriums erasing-writing method and system |
| CN110515861A (en) * | 2018-05-21 | 2019-11-29 | 北京忆芯科技有限公司 | Handle the storage device and method thereof of flash command |
| CN109358818A (en) * | 2018-10-30 | 2019-02-19 | 深圳润迅数据通信有限公司 | A kind of block device I/O Request processing method of data center |
| CN111125033B (en) * | 2018-10-31 | 2024-04-09 | 深信服科技股份有限公司 | Space recycling method and system based on full flash memory array |
| CN111125033A (en) * | 2018-10-31 | 2020-05-08 | 深信服科技股份有限公司 | Space recovery method and system based on full flash memory array |
| CN109901799B (en) * | 2019-02-28 | 2022-08-19 | 新华三信息安全技术有限公司 | Log reading and writing method and device |
| CN109901799A (en) * | 2019-02-28 | 2019-06-18 | 新华三信息安全技术有限公司 | A kind of log reading/writing method and device |
| CN111190537A (en) * | 2019-12-10 | 2020-05-22 | 优刻得科技股份有限公司 | Method and system for managing sequential storage disks in write-addition scene |
| CN111190537B (en) * | 2019-12-10 | 2023-08-25 | 优刻得科技股份有限公司 | Method and system for managing sequential storage disk in additional writing scene |
| CN111176570A (en) * | 2019-12-27 | 2020-05-19 | 浪潮电子信息产业股份有限公司 | Thick backup roll creating method, device, equipment and medium |
| CN111273861A (en) * | 2020-01-15 | 2020-06-12 | 苏州浪潮智能科技有限公司 | Data volume Bitmap destaging method and system |
| CN111803917A (en) * | 2020-06-30 | 2020-10-23 | 北京金山云网络技术有限公司 | Resource processing method and device |
| CN111966655B (en) * | 2020-08-18 | 2023-10-24 | 中国银行股份有限公司 | Method and device for managing file objects in memory in log acquisition process |
| CN111966655A (en) * | 2020-08-18 | 2020-11-20 | 中国银行股份有限公司 | Method and device for managing file objects in memory in log collection process |
| CN112162701A (en) * | 2020-09-18 | 2021-01-01 | 北京浪潮数据技术有限公司 | Storage space recovery method, device, equipment and computer storage medium |
| CN112162701B (en) * | 2020-09-18 | 2023-12-22 | 北京浪潮数据技术有限公司 | Storage space recycling method, device, equipment and computer storage medium |
| CN112667156A (en) * | 2020-12-25 | 2021-04-16 | 深圳创新科技术有限公司 | Method and device for realizing virtualization raid |
| CN112685337B (en) * | 2021-01-15 | 2022-05-31 | 浪潮云信息技术股份公司 | Method for hierarchically caching read and write data in storage cluster |
| CN112685337A (en) * | 2021-01-15 | 2021-04-20 | 浪潮云信息技术股份公司 | Method for hierarchically caching read and write data in storage cluster |
| CN112860594A (en) * | 2021-01-21 | 2021-05-28 | 华中科技大学 | Solid-state disk address remapping method and device and solid-state disk |
| CN113835639A (en) * | 2021-09-26 | 2021-12-24 | 深圳大普微电子科技有限公司 | I/O request processing method, device, equipment and readable storage medium |
| CN113835639B (en) * | 2021-09-26 | 2024-03-19 | 深圳大普微电子科技有限公司 | I/O request processing method, device, equipment and readable storage medium |
| CN113885808B (en) * | 2021-10-28 | 2024-03-15 | 合肥兆芯电子有限公司 | Mapping information recording method, memory control circuit unit and memory device |
| CN113885808A (en) * | 2021-10-28 | 2022-01-04 | 合肥兆芯电子有限公司 | Mapping information recording method, memory control circuit unit and memory device |
| CN115657963A (en) * | 2022-11-14 | 2023-01-31 | 苏州浪潮智能科技有限公司 | Sequential writing method and device based on solid state disk, electronic equipment and storage medium |
| WO2024103870A1 (en) * | 2022-11-14 | 2024-05-23 | 苏州元脑智能科技有限公司 | Sequential writing method and apparatus based on solid state disk, and electronic device and storage medium |
| CN117390072A (en) * | 2023-12-07 | 2024-01-12 | 深圳市云希谷科技有限公司 | Method for improving network request speed in embedded system |
| CN117390072B (en) * | 2023-12-07 | 2024-03-26 | 深圳市云希谷科技有限公司 | Method for improving network request speed in embedded system |
| CN117785565A (en) * | 2024-02-23 | 2024-03-29 | 深圳市创世飞达科技有限公司 | Solid state storage hard disk data recovery method and solid state hard disk |
| CN117785565B (en) * | 2024-02-23 | 2024-05-03 | 深圳市创世飞达科技有限公司 | Solid state storage hard disk data recovery method and solid state hard disk |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104035729B (en) | 2017-02-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104035729A (en) | Block device thin-provisioning method for log mapping | |
| US10649910B2 (en) | Persistent memory for key-value storage | |
| US10303596B2 (en) | Read-write control method for memory, and corresponding memory and server | |
| CN102436353B (en) | Semiconductor memory controlling device | |
| US9304904B2 (en) | Hierarchical flash translation layer | |
| US8880544B2 (en) | Method of adapting a uniform access indexing process to a non-uniform access memory, and computer system | |
| CN102981963B (en) | A kind of implementation method of flash translation layer (FTL) of solid-state disk | |
| CN111026329B (en) | Key value storage system based on host management tile record disk and data processing method | |
| CN103577121B (en) | A kind of highly reliable linear file access method based on Nand Flash | |
| CN110347336A (en) | A kind of key assignments storage system based on NVM with SSD mixing storage organization | |
| CN114546296B (en) | ZNS solid state disk-based full flash memory system and address mapping method | |
| CN103577574B (en) | High-reliability linear file system based on nand flash | |
| CN110825748A (en) | High-performance and easily-expandable key value storage method utilizing differential index mechanism | |
| CN100426259C (en) | Virtual access method of storage document data | |
| US10572171B2 (en) | Storage system | |
| CN106708427A (en) | Storage method suitable for key value pair data | |
| CN111522507B (en) | Low-delay file system address space management method, system and medium | |
| CN109582593B (en) | FTL address mapping reading and writing method based on calculation | |
| CN102722449A (en) | Key-Value local storage method and system based on solid state disk (SSD) | |
| CN106502587A (en) | Data in magnetic disk management method and magnetic disk control unit | |
| CN105955664B (en) | A kind of reading/writing method of watt record conversion layer based on segment structure | |
| CN106815152A (en) | A kind of method for optimizing page level flash translation layer (FTL) | |
| CN107015763A (en) | Mix SSD management methods and device in storage system | |
| CN107992269A (en) | A kind of affairs wiring method based on duplicate removal SSD | |
| CN110968269A (en) | SCM and SSD-based key value storage system and read-write request processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20170215 |