CN102968279B - A kind of store the method that system simplifies configuration automatically - Google Patents
A kind of store the method that system simplifies configuration automatically Download PDFInfo
- Publication number
- CN102968279B CN102968279B CN201210453207.5A CN201210453207A CN102968279B CN 102968279 B CN102968279 B CN 102968279B CN 201210453207 A CN201210453207 A CN 201210453207A CN 102968279 B CN102968279 B CN 102968279B
- Authority
- CN
- China
- Prior art keywords
- metadata
- space
- memory element
- equipment
- management
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 31
- 230000006870 function Effects 0.000 claims abstract description 12
- 230000007246 mechanism Effects 0.000 claims description 42
- 238000004891 communication Methods 0.000 claims description 19
- 230000008569 process Effects 0.000 claims description 18
- 238000013507 mapping Methods 0.000 claims description 13
- 238000004064 recycling Methods 0.000 claims description 12
- 238000005516 engineering process Methods 0.000 claims description 9
- 238000011084 recovery Methods 0.000 claims description 9
- 238000013461 design Methods 0.000 claims description 6
- 238000001514 detection method Methods 0.000 claims description 5
- 238000012544 monitoring process Methods 0.000 claims description 4
- 230000008520 organization Effects 0.000 claims description 4
- 230000002301 combined effect Effects 0.000 claims description 3
- 230000000694 effects Effects 0.000 claims description 3
- 238000005096 rolling process Methods 0.000 claims 1
- 230000004899 motility Effects 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 6
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- NHDHVHZZCFYRSB-UHFFFAOYSA-N pyriproxyfen Chemical compound C=1C=CC=NC=1OC(C)COC(C=C1)=CC=C1OC1=CC=CC=C1 NHDHVHZZCFYRSB-UHFFFAOYSA-N 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention provides a kind of and stores the method that system simplifies configuration automatically, automatically simplifies configuration and is generally used in storage system, it is intended to provide jumbo virtual drive to operating system. Automatic reduction techniques virtual offer can exceed the volume space of reality to upper strata, namely allows operating system think operable capacity more than the actual capacity being actually provided to it. This is primarily due to the data in enterprise and is being gradually increased often, rather than has so many at the very start, and configuration is simplified in employing automatically can improve the utilization rate of memory capacity. Automatically simplify configuration storage scheme and can provide jumbo virtual drive to operating system. Operating system thinks there is so big disk space when distributing disk space, and therefore along with the increase of business data amount, enterprise can consider that situation substep increases storage device, it is not necessary to operating system is adjusted. This is equivalent to the warm connection function being to achieve storage device, it is possible to improve storage efficiency and the motility of disk unit.
Description
Technical field
The present invention relates to server storage system, specifically a kind of store the method that system simplifies configuration automatically.
Background technology
Actual physical storage device is likely to so not many, and allows operating system think and have so big memory space. Therefore, automatically simplify configuration and allow for supporting dynamic capacity-expanding, before the actual physical storage device device space takes, use the dynamic capacity-expanding method automatically simplifying configuration to expand actual memory space, the normal use of guarantee custom system. Otherwise, operating system is because of it and does not know actual physics capacity situation, and uses and non-existent memory space, will result in immeasurable serious problems.
Automatically simplify at present to be arranged in the external storage system storing manufacturer such as EMC, IBM, HDS, HP3PAR and extensively use, but domestic manufacturers are still slightly aobvious weak in this field.
Automatically the configuration of simplifying of HP3PAR has many good qualities on the whole, such as its unit storage unit is 16KB, this be a kind of very fine granularity significantly high automatically simplify configuration, compare some more coarse implementations, automatically the configuration capacity saving of simplifying of 3PAR reaches to maximize, can there is the more obvious impact of performance, and it simplifies configuration automatically is full automatic automatically simplify configuration, do not use in the storage environment automatically simplifying configuration traditional, the reason of some application programs configuration bulk redundancy capacity (3 times or more be usually actually needed) would generally be given, namely it is in order to avoid manually repeating complexity and the interruption that configuration behavior brings. if automatically simplifying configuration still to be unable to do without and manually set, substantial amounts of original complexity just still can be retained.
But, HP3PAR from simplifying in realization, especially reclaim on, the asic chip assist process of its uniqueness.
Certainly simplifying of EMC and IBM realizes feature, have some relations with two company's software engineering outstanding abilities in distribution and early warning etc., to POOL and LUN(or be Volume) process on, it is all based on Intel Virtualization Technology, readwrite performance equilibrium has one's own knack, but in space reclamation, all there is a relatively larger difficult problem, realizing also is all realize in other additional software applications, rather than provides general reclaim mechanism.
The dynamically pre-configured of HDS is not completely automatic. Need nonetheless remain for manually going to create array group and LDEV, and assign them to HDP pond. In full automatic implementation; storage system can create array group or HDP pond volume automatically by suitable RAID protection level or capabilities service level; and can be real-time these be rolled up adds in HDP ponds, and whole process is not completely by the interference of anthropic factor. Meanwhile, the dynamic pre-configured volume (HDP) of HDS is with 42MB for memory element. Comparing 3PAR and provide the unit storage pool of 16KB, this size is very coarse.
Another one aspect, big memory element partition capacity again means that the less metadata of needs describes a virtual volume, at this moment, contributes to automatically simplifying allocation list and reveals good performance. But this rough automatically simplifying configures and may bring certain threat in practical operation, because automatically simplifying the space not taking into account that this partial memory cell time configuration distributes memory element is formatted or original state, therefore, when application needs to format some memory spaces dispensed, excessive memory element can cause potential operation risk. If the memory space of these unformatted original states spreads all over whole volume, the space that they will consume in more memory element.
In these foreign vendors, the maximum of 3PAR is distinctive in that, automatically the simplifying to configure of 3PAR is implemented on bottom architecture entirely, is supplied to storage system not as an additional software function. Thus more transparent, also automatization more. And other manufacturer, such as NetApp, HDS and HP, the method automatically simplifying configuration is mostly create memory resource pool in the middle of RAID group, and user distributes LUNs and volume as before, but actually system is simply at deception hardware. Increasing when actual disk uses, storage administrator must adjust the distribution of their volume, this automatically simplify configuration to may result in manual operation more, and metadata processes more numerous and diverse.
Technical monopoly and block in the face of foreign vendor, the present invention provides one automatically to simplify configuration system architecture, this framework adopts exclusive chip assist process unlike HP3PAR, Intel Virtualization Technology is adopted to go to realize, adopt double; two B+ tree to carry out metadata management, amendment storage agreement provides dynamic capacity to reclaim function, and it is modified utilizing the agreement of SSD, early warning mechanism is provided in operating system aspect, realize automatization and automatically simplify configuration, not only there is high-performance, and having recovery function, memory element granularity can regulate and control according to practical application.
Summary of the invention
It is an object of the invention to provide and a kind of store the method that system simplifies configuration automatically.
It is an object of the invention to realize in the following manner, this system includes 5 functional modules, and they are respectively as follows: pond molded tissue block 1); Simplify distribution module 2); Simplify recycling module 3); Dynamic capacity-expanding module 4); Warning module 5), each functional module illustrates as follows:
1) pond molded tissue block:
Pond tissue is exactly record and the management of the memory element metadata to each distribution, in native system, takes two equipment of management in logic: data equipment and metadata equipment;
In the molded tissue block of pond, the management of metadata is more complicated, because automatically simplifying configuration to need each data block on management data equipment, but also to have structure to describe storage pool and simplification volume, so needing substantial amounts of metadata to manage these contents, therefore, individually allocate a device and store the information of metadata, this equipment is called metadata equipment, the metadata information that metadata equipment produces in order to the distribution information and storage pool operation storing memory element in data equipment, and data equipment, it is then carry out, according to memory element granularity, the physical storage devices cut, without metadata information in this equipment,
As can be seen here, the management of pond molded tissue block metadata needs two equipment of management, metadata equipment and data equipment, therefore, it is required for the two equipment to manage respectively, here it is accomplished by a set of space and maps the administrative mechanism of space-map to record the mapping situation of each memory element, the management of auxiliary metadata, simultaneously, this set mechanism not only can assist the management realizing metadata, can also serving as the allotter of new memory element, the granularity of memory element also defines in the molded tissue block of pond;
The administrative mechanism that the space of metadata management maps comprises two kinds of implementations:
The implementation of management physical device data equipment; The implementation of management metadata Metadata equipment; In the molded tissue block of pond, metadata management comprises both implementations, and the management of metadata needs both implementation combineds effect, comprehensively manages thus realizing pond organization metadata;
2) distribution module is simplified:
In simplifying distribution module, system uses distribution technique when writing, physical disk capacity is only just allocated when application program description real data is to logical volume, and simple understanding is exactly only when there being write request, just on-demand can distribute new block to this request from idle resource pool; In simplifying distribution module, when there being write request, the memory element for corresponding to this request: first, the metadata information preserved in metadata to be searched, be divided into two kinds of situations here:
A) find, illustrate that this memory element completes to map, therefore directly map that to logical volume;
B) do not find, illustrate that this memory element is not fully complete mapping, then need this request is carried out delay disposal, in simplifying distribution main thread, complete the operation of this request subsequently;
For situation about not finding, also need this write request of subsequent treatment, mainly distribute new storage list, complete the mapping of this memory element, and can in upper process once because it has completed to map, so the operation of this write request just can be completed;
3) recycling module is simplified:
File system layer learns that space release conditions circularizes bottom storage system, and bottom storage receives release notice, abandons corresponding memory element;
Simplify recovery and include two aspects: the file system layer at application program place; The storage system layer at real data place, and this one of them important issue is that the communication mechanism problem of file system and storage system, file system does not have " consciousness " simplified, when a space is no longer used, ready-made mechanism is not had to go to circulate a notice of these situations, and the storage system layer of bottom is without actively reclaiming space, unless you " tell " it, therefore it is accomplished by the communication mechanism designing a set of file system with storage system;
Problem for communication mechanism, design document system is wanted to drive with bottom block, by supporting that discard realizes the communication of file system and storage system, at present, discard supports it is for optimizing the linux kernel support to SSD, and is a set of mechanism added in ext4 and xfs file system;
Discard is the term of linux, it is no longer store valid data for notice these sectors of storage device, its essence is exactly by discard, file system tells which sector of SSD dish of bottom can be deleted, similar this communication mechanism is used to remove amendment SCSI protocol, that can better support automatically to simplify configuration simplifies recovery function, and need not take third party's instrument;
Therefore, design the similar discard communication mechanism supported, simplifying recycling module, after file system deletes certain file, notify that one piece of region that bottom storage device this document is corresponding no longer stores valid data by discard, reclaim at bottom;
4) dynamic capacity-expanding module:
Realize the expansion of storage pool capacity; Dynamic capacity-expanding main logic:
A) according to the map information that the incoming dilatation parameter in upper strata or dilatation are complete, the quantity of the complete memory element of dilatation is calculated;
B) use space mapping mechanism to complete the realization of dilatation, namely realize storage pool, namely pond molded tissue block 1) in the dilatation of data equipment;
C) quantity that bottom storage pool needs the memory element of dilatation is calculated;
Circulation gradually processes the initialization information of each memory element, mainly distributes the capacity-enlarging information that new unit records data block maps here in data equipment metadata; Then the information of dilatation is preserved;
After dilatation loop ends, the amount of capacity of amendment data equipment;
The metadata submitting storage pool to refreshes to disk;
5) space warning module:
The detection early warning information of bottom or kernel state, has the monitoring early warning information of then notice client layer or User space, client layer or User space;
Automatically simplify configuration space warning module mainly to realize having reached early warning threshold values when spatial content, bottom or kernel state report to the police to client layer or User space;
Space warning module to be divided into two parts, respectively kernel state and User space, it is necessary to realizes communicating of kernel state and User space; Space warning module main logic is:
1) at kernel state, whether detection spatial content reaches early warning threshold values;
2) when reaching early warning threshold values, early warning information is sent to User space;
3) User space monitoring and warning information, receives early warning information, notifies manager;
But how by the information of kernel state, after seizure, notify User space, be the most basic technology point of space warning module.
Thin provisioning extends memory management functions, although the physical capacity of actual allocated is little, but can dynamically provide the virtual memory space of vast capacity for operating system. Along with the data of application write get more and more, the physical memory space can also extend in time, without extended by hands. In other words, automatically simplify configuration and provide " operation time space ", it is possible to substantially reduce and distribute but untapped memory space.
The invention has the beneficial effects as follows: adopt this framework not need to adopt the distribution of exclusive chip assist process as HP3PAR and reclaim, Intel Virtualization Technology is adopted to go to realize, adopt double; two B+ tree to carry out metadata management, amendment storage agreement provides dynamic capacity to reclaim function, and it is modified utilizing the agreement of SSD, early warning mechanism is provided, it is achieved automatization simplifies configuration automatically, not only has high-performance in operating system aspect, and having recovery function, memory element granularity can regulate and control according to practical application.
Automatically simplify configuration storage scheme and can provide jumbo virtual drive to operating system. Operating system thinks there is so big disk space when distributing disk space, and therefore along with the increase of business data amount, enterprise can consider that situation substep increases storage device, it is not necessary to operating system is adjusted. This is equivalent to the warm connection function being to achieve storage device, it is possible to improve storage efficiency and the motility of disk unit.
Accompanying drawing explanation
Fig. 1 simplifies configuration schematic diagram automatically;
Fig. 2 is framework main modular schematic diagram;
Fig. 3 is system architecture overview flow chart;
Fig. 4 is that space map information differentiation processes;
Fig. 5 is discard communication mechanism schematic diagram;
The dilatation flow process that Fig. 6 is General Logic volume dilatation and the present invention contrasts;
Fig. 7 is space warning module main logic schematic diagram.
Detailed description of the invention
With reference to Figure of description, the method for the present invention is described in detail below.
The technology of the need solution that the present invention is main includes:
1) virtual address maps
The virtual address space mapping mechanism to applied host machine, thus under not interrupting data access scenarios, dynamically promoting storage system and use space;
2) capacity recycling function
Automatically simplify configuration and can dynamically reclaim allocated space, but how to reclaim the physical space dispensed, and space is redistributed re-use the difficult problem being automatically simplify configuration;
3) virtual memory space management
The storage pool of dynamic expansion needs to keep the efficient of storage access performance, it is thus desirable to the Data Migration algorithm of the research data distribution algorithms of performance optimization, lightweight and efficient data balancing algorithm, reduce the impact that the extension concurrent access of storage pool causes to greatest extent, reduce the number of times that data move, it is achieved the highly-parallel that system accesses;
4) the dynamic alert mechanism of space distribution
Many factors need to be comprehensively utilized, the cost etc. consumed during including the ruuning situation of application program, the Expenditure Levels of memory space, the type of organization storing system and initialization time, file system format, change the alarm mechanism automatically simplifying configuration adaptively, it is to avoid the problem of fixed allocation alarm mechanism or the assignment constraint encountered motility of mechanism and safety deficiency.
This framework is based entirely on software and realizes automatically simplifying configuration, it is not necessary to by special exclusive chip, and supports that dynamically recovery and memory element granularity are dynamically adjustable, and provides space early warning.
This framework mainly by 5 functional modules, as it can be seen, they be respectively as follows: pond molded tissue block, simplify distribution module, simplify recycling module, dynamic capacity-expanding module and warning module composition.
Can be seen that from this framework, only dilatation module, simplify distribution module directly can interact with user with simplifying recycling module, user can initiate the instruction relevant to this 3 modules, carry out corresponding from simplifying control, and for warning module, user is only passive receives whether capacity reaches early warning, it is possible to inquiry, but the work of this module uncontrollable. For pond molded tissue block, belong to this framework nucleus module, user is transparent.
As it is shown on figure 3, for the request from user, this framework first determines whether whether this request relates to reclaiming, if relating to reclaiming, then inquiry pond molded tissue block, calls the handling process simplifying recycling module, reclaims capacity and updates the metadata of pond tissue; If this request is not related to reclaim, for normal read-write requests, then distinguish read-write, read request in this way, then by simplifying the distribution module polls concrete memory element in the tissue of pond, process, if write request, then determine whether to exceed the setting of warning module, if exceeded, warning module then can directly issue the user with alarm, if without departing from early warning threshold values, then submit to and simplify allocation process, simplify distribution module and from the molded tissue block of pond, then distribute concrete memory element process.
Each functional module illustrates as follows:
1) pond molded tissue block:
Pond tissue is exactly to the record of memory element (or being designated as block) metadata of each distribution and management. In this framework, take two equipment of management in logic: data equipment and metadata equipment;
In the molded tissue block of pond, the management of metadata is more complicated, because automatically simplifying configuration to need each data block on management data equipment, but also to have structure to describe storage pool and simplification volume, so needing substantial amounts of metadata to manage these contents. Therefore, the present invention individually allocates a device and stores the information of metadata, and this equipment is called metadata equipment. The metadata equipment metadata information in order to store the distribution information of memory element in data equipment and storage pool operation produces, and data equipment, then be carry out, according to memory element granularity, the physical storage devices cut, without metadata information in this equipment;
As can be seen here, the management of pond molded tissue block metadata needs two equipment (metadata equipment and data equipment) of management, it is therefore desirable to manage respectively for the two equipment.Here it is accomplished by a set of space and maps the administrative mechanism of (space-map) to record the mapping situation of each memory element, the management of auxiliary metadata. Meanwhile, this set mechanism not only can assist the management realizing metadata, additionally it is possible to serves as the allotter of new memory element. The granularity of memory element also defines in the molded tissue block of pond;
The administrative mechanism that the space of metadata management maps comprises two kinds of implementations:
The implementation of management physical device data equipment; The implementation of management metadata Metadata equipment;
In the molded tissue block of pond, metadata management comprises both implementations, and the management of metadata needs both implementation combineds effect, comprehensively manages thus realizing pond organization metadata. Fig. 4 simple declaration logic signal on disk;
2) distribution module is simplified:
In simplifying distribution module, the present invention uses distribution technique when writing, and physical disk capacity is only just allocated when application program description real data is to logical volume. Simple understanding is exactly only when there being write request, just on-demand can distribute new block to this request from idle resource pool;
In simplifying distribution module, when there being write request, the memory element for corresponding to this request:
First, the metadata information preserved in metadata to be searched, be divided into two kinds of situations here:
A) find, illustrate that this memory element completes to map, therefore directly map that to logical volume;
B) do not find, illustrate that this memory element is not fully complete mapping, then need this request is carried out delay disposal, in simplifying distribution main thread, complete the operation of this request subsequently;
For situation about not finding, also need this write request of subsequent treatment, mainly distribute new storage list, complete the mapping of this memory element, and can in upper process once because it has completed to map, so the operation of this write request just can be completed;
3) recycling module is simplified:
File system layer learns that space release conditions circularizes bottom storage system. Bottom storage receives release notice, abandons corresponding memory element;
Simplify recovery and include two aspects: the file system layer at application program place; The storage system layer at real data place. And this one of them important issue is that the communication mechanism problem of file system and storage system. File system does not have " consciousness " simplified, when a space is no longer used, it does not have ready-made mechanism goes to circulate a notice of these situations. And the storage system layer of bottom is without actively reclaiming space, unless you " tell " it. Therefore the communication mechanism designing a set of file system with storage system it is accomplished by;
For the problem of communication mechanism, in framework of the present invention, design document system and bottom block are driven through the communication supporting discard to realize file system and storage system. At present, discard supports it is for optimizing the linux kernel support to SSD, and in a set of mechanism that ext4 and xfs file system is added;
Discard is the term of linux, is no longer store valid data for notice these sectors of storage device. Its essence is exactly by discard, file system tells which sector of SSD dish of bottom can be deleted, similar this communication mechanism is used to remove amendment SCSI protocol, that can better support automatically to simplify configuration simplifies recovery function, and need not take third party's instrument;
Therefore, the present invention designs the similar discard communication mechanism supported, is simplifying recycling module, after file system deletes certain file, notify that one piece of region that bottom storage device this document is corresponding no longer stores valid data by discard, it is possible to reclaim at bottom;
4) dynamic capacity-expanding module:
Realize the expansion of storage pool capacity;
Dynamic capacity-expanding main logic:
A) according to the dilatation parameter (map information that dilatation is complete) that upper strata is incoming, the quantity of the complete memory element of dilatation is calculated;
B) use space mapping mechanism to complete the realization of dilatation, namely realize storage pool, the namely dilatation of data equipment in module 1;
C) quantity that bottom storage pool needs the memory element of dilatation is calculated;
Circulation gradually processes the initialization information of each memory element. Here in data equipment metadata, mainly distribute the capacity-enlarging information that new unit records data block maps; Then the information of dilatation is preserved;
After dilatation loop ends, the amount of capacity of amendment data equipment;
The metadata submitting storage pool to refreshes to disk;
The dilatation flow process that Fig. 6 is General Logic volume dilatation schematic diagram and the present invention contrasts:
5) space warning module:
Bottom (kernel state) detects early warning information, has, and notifies client layer (User space). Client layer (User space) monitors early warning information;
Automatically simplify configuration space warning module mainly to realize having reached early warning threshold values when spatial content, bottom (kernel state) report to the police to client layer (User space).
Space warning module to be divided into two parts, respectively kernel state and User space, it is necessary to realizing communicating of kernel state and User space, space warning module main logic is:
1) at kernel state, whether detection spatial content reaches early warning threshold values;
2) when reaching early warning threshold values, early warning information is sent to User space;
3) User space monitoring and warning information, receives early warning information, notifies manager;
Main logic schematic diagram is as shown in Figure 7:
The main logic of space warning module as shown above, but how by the information of kernel state, after seizure, notifies User space, is the most basic technology point of space warning module.
Except the technical characteristic described in description, it is the known technology of those skilled in the art.
Claims (1)
1. one kind stores the method that system simplifies configuration automatically, it is characterised in that system includes 5 functional modules, and they are respectively as follows: pond molded tissue block; Simplify distribution module; Simplify recycling module; Dynamic capacity-expanding module; Space warning module, each functional module illustrates as follows:
(1) pond molded tissue block:
Pond tissue is exactly record and the management of the memory cell data to each distribution, in native system, takes two equipment of management in logic: data equipment and metadata equipment;
In the molded tissue block of pond, the management of metadata is more complicated, because automatically simplifying configuration to need each data block on management data equipment, but also to have structure to describe storage pool and simplification volume, so needing substantial amounts of metadata to manage these contents, therefore, individually allocate a device and store the information of metadata, this equipment is called metadata equipment, the metadata information that metadata equipment produces in order to the distribution information and storage pool operation storing memory element in data equipment, and data equipment, it is then carry out, according to memory element granularity, the physical storage devices cut, without metadata information in this equipment,
As can be seen here, the management of pond molded tissue block metadata needs two equipment of management, metadata equipment and data equipment, therefore, it is required for the two equipment to manage respectively, here it is accomplished by a set of space and maps the administrative mechanism of space-map to record the mapping situation of each memory element, the management of auxiliary metadata, simultaneously, this set mechanism not only can assist the management realizing metadata, can also serving as the allotter of new memory element, the granularity of memory element also defines in the molded tissue block of pond;
The administrative mechanism that the space of metadata management maps comprises two kinds of implementations:
The implementation of management physical device data equipment; The implementation of management metadata Metadata equipment; In the molded tissue block of pond, metadata management comprises both implementations, and the management of metadata needs both implementation combineds effect, comprehensively manages thus realizing pond organization metadata;
(2) distribution module is simplified:
In simplifying distribution module, system uses distribution technique when writing, physical disk capacity is only just allocated when application program description real data is to logical volume, and simple understanding is exactly only when there being write request, just on-demand can distribute new memory element from storage pool to this request; In simplifying distribution module, when there being write request, the memory element for corresponding to this request: first, the metadata information preserved in metadata to be searched, be divided into two kinds of situations here:
A) find, illustrate that this memory element completes to map, therefore directly map that to logical volume;
B) do not find, illustrate that this memory element is not fully complete mapping, then need this request is carried out delay disposal, in simplifying distribution main thread, complete the operation of this request subsequently;
For situation about not finding, also need this write request of subsequent treatment, mainly distribute new memory element, complete the mapping of this memory element, and can in upper process once because it has completed to map, so the operation of this write request just can be completed;
(3) recycling module is simplified:
File system layer learns that space release conditions circularizes lower floor's block device, the logical volume that in the corresponding present invention, storage pool produces, and logical volume receives release notice, abandons corresponding memory element;
Simplify recovery and include two aspects: the file system layer at application program place; The logic volume layer at real data place, and this one of them important issue is that the communication mechanism problem of file system and logical volume, file system does not have " consciousness " simplified, when a space is no longer used, ready-made mechanism is not had to go to circulate a notice of these situations, and logical volume is without actively reclaiming space, unless you " tell " it, therefore it is accomplished by the communication mechanism designing a set of file system with logical volume;
Problem for communication mechanism, design document system is wanted to drive with bottom block, by supporting that discard realizes the communication of file system and logical volume, at present, discard supports it is for optimizing the linux kernel support to SSD, and is a set of mechanism added in ext4 and xfs file system;
Discard is the term of linux, it is no longer store valid data for notice these sectors of storage device, its essence is exactly by discard, file system tells which sector of SSD dish of bottom can be deleted, similar this communication mechanism is used to remove amendment SCSI protocol, that can better support automatically to simplify configuration simplifies recovery function, and need not take third party's instrument;
Therefore, design the similar discard communication mechanism supported, simplify recycling module, after file system deletes certain file, the one piece of region rolling up this document corresponding by discard notification logic no longer stores valid data, reclaims at the storage pool layer belonging to logical volume;
(4) dynamic capacity-expanding module:
Realize the expansion of storage pool capacity; Dynamic capacity-expanding main logic:
A) according to the map information that the incoming dilatation parameter in upper strata or dilatation are complete, the quantity of the complete memory element of dilatation is calculated;
B) use space mapping mechanism to complete the realization of dilatation, namely realize storage pool, the namely dilatation of data equipment in the molded tissue block of pond;
C) quantity that bottom storage pool needs the memory element of dilatation is calculated;
Circulation gradually processes the initialization information of each memory element, mainly distributes the capacity-enlarging information that new unit records data block maps here in data equipment metadata; Then the information of dilatation is preserved;
After dilatation loop ends, the amount of capacity of amendment data equipment;
The metadata submitting storage pool to refreshes to disk;
(5) space warning module:
Bottom or kernel state detection early warning information, have then notice client layer or User space, client layer or User space to monitor early warning information;
Space warning module mainly realizes working as spatial content and has reached early warning threshold values, bottom or kernel state report to the police to client layer or User space;
Space warning module to be divided into two parts, respectively kernel state and User space, it is necessary to realizes communicating of kernel state and User space; Space warning module main logic is:
1) at kernel state, whether detection spatial content reaches early warning threshold values;
2) when reaching early warning threshold values, early warning information is sent to User space;
3) User space monitoring and warning information, receives early warning information, notifies manager;
But how by the information of kernel state, after seizure, notify User space, be the most basic technology point of space warning module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210453207.5A CN102968279B (en) | 2012-11-13 | 2012-11-13 | A kind of store the method that system simplifies configuration automatically |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210453207.5A CN102968279B (en) | 2012-11-13 | 2012-11-13 | A kind of store the method that system simplifies configuration automatically |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102968279A CN102968279A (en) | 2013-03-13 |
| CN102968279B true CN102968279B (en) | 2016-06-08 |
Family
ID=47798444
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210453207.5A Active CN102968279B (en) | 2012-11-13 | 2012-11-13 | A kind of store the method that system simplifies configuration automatically |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102968279B (en) |
Families Citing this family (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9003135B2 (en) | 2013-01-15 | 2015-04-07 | International Business Machines Corporation | Efficient allocation and reclamation of thin-provisioned storage |
| CN103345417A (en) * | 2013-06-06 | 2013-10-09 | 华为技术有限公司 | Method and device for recovering storage space |
| CN103631537B (en) * | 2013-11-29 | 2016-09-14 | 华为技术有限公司 | The management method of a kind of virtual disk and device |
| CN103744792B (en) * | 2014-01-10 | 2017-10-31 | 浪潮电子信息产业股份有限公司 | It is a kind of to realize the system that the automatic simplify configuration of storage system is fully distributed |
| CN103838524B (en) * | 2014-03-13 | 2017-01-18 | 深圳市迪菲特科技股份有限公司 | Implement method and storage system of self-adaption thin provisioning |
| CN104035729B (en) * | 2014-05-22 | 2017-02-15 | 中国科学院计算技术研究所 | Block device thin-provisioning method for log mapping |
| CN103973497A (en) * | 2014-05-23 | 2014-08-06 | 浪潮电子信息产业股份有限公司 | Method and device for realizing multi-path concurrent storage based on high-density micro server |
| CN104820575B (en) * | 2015-04-27 | 2017-08-15 | 西北工业大学 | Realize the method that storage system is simplified automatically |
| US9697079B2 (en) | 2015-07-13 | 2017-07-04 | International Business Machines Corporation | Protecting data integrity in de-duplicated storage environments in combination with software defined native raid |
| CN105159616A (en) * | 2015-09-11 | 2015-12-16 | 浪潮(北京)电子信息产业有限公司 | Disk space management method and device |
| US9846538B2 (en) | 2015-12-07 | 2017-12-19 | International Business Machines Corporation | Data integrity and acceleration in compressed storage environments in combination with software defined native RAID |
| CN106919342A (en) * | 2015-12-28 | 2017-07-04 | 成都华为技术有限公司 | Storage resource distribution method and device based on automatic simplify configuration |
| CN106202350A (en) * | 2016-07-05 | 2016-12-07 | 浪潮(北京)电子信息产业有限公司 | A kind of distributed file system simplifies the method and system of configuration automatically |
| CN107239283B (en) * | 2017-05-31 | 2020-11-20 | 苏州浪潮智能科技有限公司 | Storage system management software adaptation method and device |
| CN107506142A (en) * | 2017-08-18 | 2017-12-22 | 郑州云海信息技术有限公司 | The distribution method and device of a kind of volume space |
| CN108073723A (en) * | 2018-01-03 | 2018-05-25 | 郑州云海信息技术有限公司 | A kind of file in distributed type assemblies storage is from compressing method and equipment |
| CN109358976A (en) * | 2018-10-24 | 2019-02-19 | 郑州云海信息技术有限公司 | It is a kind of to simplify characteristic test method, device, equipment and storage medium certainly |
| CN109445701B (en) * | 2018-10-26 | 2021-02-23 | 北京计算机技术及应用研究所 | Automatic simplifying configuration synchronization method for double-control disk array |
| CN110007868B (en) * | 2019-04-12 | 2022-07-22 | 苏州浪潮智能科技有限公司 | SSD disc metadata storage method, device, controller and storage medium |
| CN110647294B (en) * | 2019-09-09 | 2022-03-25 | Oppo广东移动通信有限公司 | Storage block recovery method and device, storage medium and electronic equipment |
| CN111459880B (en) * | 2020-03-27 | 2023-01-06 | 苏州浪潮智能科技有限公司 | Method and system for modifying total space of self-simplified pre-allocation space compatible system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101013996A (en) * | 2007-02-16 | 2007-08-08 | 华为技术有限公司 | Block storage service method, block storage service system and block storage service client |
| US7747835B1 (en) * | 2005-06-10 | 2010-06-29 | American Megatrends, Inc. | Method, system, and apparatus for expanding storage capacity in a data storage system |
| CN102591789A (en) * | 2011-12-26 | 2012-07-18 | 成都市华为赛门铁克科技有限公司 | Storage space recovery method and storage space recovery device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7162600B2 (en) * | 2005-03-29 | 2007-01-09 | Hitachi, Ltd. | Data copying method and apparatus in a thin provisioned system |
-
2012
- 2012-11-13 CN CN201210453207.5A patent/CN102968279B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7747835B1 (en) * | 2005-06-10 | 2010-06-29 | American Megatrends, Inc. | Method, system, and apparatus for expanding storage capacity in a data storage system |
| CN101013996A (en) * | 2007-02-16 | 2007-08-08 | 华为技术有限公司 | Block storage service method, block storage service system and block storage service client |
| CN102591789A (en) * | 2011-12-26 | 2012-07-18 | 成都市华为赛门铁克科技有限公司 | Storage space recovery method and storage space recovery device |
Non-Patent Citations (1)
| Title |
|---|
| 存储的自动精简配置技术应用研究;邱红飞;《电信科学》;20101115(第11期);第12-17页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102968279A (en) | 2013-03-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102968279B (en) | A kind of store the method that system simplifies configuration automatically | |
| US11663029B2 (en) | Virtual machine storage controller selection in hyperconverged infrastructure environment and storage system | |
| US8756392B2 (en) | Storage control apparatus and storage system comprising multiple storage control apparatuses | |
| US9811465B2 (en) | Computer system and cache control method | |
| CN104636080B (en) | Storage system and the method for it | |
| CN104317742B (en) | Thin provisioning method for optimizing space management | |
| US8225039B2 (en) | Storage controller and virtual volume control method | |
| US9292218B2 (en) | Method and apparatus to manage object based tier | |
| CN111158587B (en) | Distributed storage system based on storage pool virtualization management and data read-write method | |
| JP4975399B2 (en) | Resource management method in logical partitioning storage system | |
| CN102207830B (en) | Cache dynamic allocation management method and device | |
| US20130097377A1 (en) | Method for assigning storage area and computer system using the same | |
| US20190004724A1 (en) | Memory system and method for controlling nonvolatile memory | |
| US20120246386A1 (en) | Storage system and storage area allocation method | |
| US9760292B2 (en) | Storage system and storage control method | |
| US20150199129A1 (en) | System and Method for Providing Data Services in Direct Attached Storage via Multiple De-clustered RAID Pools | |
| CN105808157A (en) | Storage architecture creating method, storage access method and storage system | |
| US20140297909A1 (en) | Storage apparatus and hierarchy control method | |
| US8650358B2 (en) | Storage system providing virtual volume and electrical power saving control method including moving data and changing allocations between real and virtual storage areas | |
| CN104731517A (en) | Method and device for allocating capacity of storage pool | |
| US10082968B2 (en) | Preferred zone scheduling | |
| US10496278B1 (en) | Inline compression support using discrete sized containers for backing store | |
| US11314416B1 (en) | Defragmentation of striped volume in data storage system | |
| CN104238955B (en) | A kind of device and method of storage resource virtualization distribution according to need | |
| JP2013101665A (en) | Storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |