CN102123171A - Storage virtualization method for mass storage management - Google Patents
Storage virtualization method for mass storage management Download PDFInfo
- Publication number
- CN102123171A CN102123171A CN2011100253618A CN201110025361A CN102123171A CN 102123171 A CN102123171 A CN 102123171A CN 2011100253618 A CN2011100253618 A CN 2011100253618A CN 201110025361 A CN201110025361 A CN 201110025361A CN 102123171 A CN102123171 A CN 102123171A
- Authority
- CN
- China
- Prior art keywords
- file
- node
- storage resources
- storage
- memory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention discloses a storage virtualization method for mass storage management, which belongs to the technical field of computer storage and solves the problems of highly-complicated and large-scale storage resources in the conventional mass heterogeneous storage so as to improve the utilization ratio of the storage resources and to reduce the management cost of a storage system. The storage virtualization method for the mass storage management comprises the following steps of: (1) discovery of the storage resources; (2) agent of the storage resources; and (3) allocation of the storage resources. In the method, the utilization ratio of the storage resources is improved by the reasonable allocation of storage capacity and storage performance in a whole storage space, data transmission performance is improved by independent control and data transmission paths, and the management cost of the storage system is reduced by providing centralized management for scattered enterprise storage. The storage virtualization method for the mass storage management is applied to virtualized management for a mass heterogeneous storage system.
Description
Technical field
The invention belongs to computer memory technical field, be specifically related to a kind of Storage Virtualization method towards the mass memory management.
Background technology
The mass storage system (MSS) of current application is the mass memory network that is made of memory node, management node, user node.Each memory node is made of the memory device and the storing software of different vendor, has incompatible phenomenon between mutually, and functional area is divided complicated, equipment, protocol type are various, memory node quantity is huge, and various application load height mix, and cause system management memory complexity, the wasting of resources.Though propose the demand of the standardization of products in the industry already, but from realistic situation, integration, management for the isomery storage environment are still very undesirable, thereby cause a large amount of wastes and the overlapping investment of storage resources, but also influenced the consistency visit of data and all many-sides such as backup disaster tolerance of entire system.
Simultaneously, the magnanimity heterogeneous storage system has following characteristics: (1) isomerism height, and storage resources information high complexity, the visit expense is big; (2) in large scale, surpass single volume controller disposal ability, need many high performance volume controller equiments of configuration.
Storage virtualization technology has usually that device level is virtual, the subsystem irrespective of size is virtual, tape is virtual, the piece level is virtual and the virtual several types of file-level, implementation method then comprises in the band and is with outer two kinds, be meant the same path of transfer of data and control transmission in the band, be with the reportedly defeated different paths of outer index with control transmission.Wherein device level and the virtual scale of subsystem irrespective of size are less, how are realized the virtual standby system of being more common in of tape, general virtual two types of the virtual and file-level of piece level that adopts in mass storage system (MSS) by firmware or hardware mode.Traditional Storage Virtualization method adopts form in the band more, be convenient to centralized control and management, storage volume controller with IBM is representative, its piece level Storage Virtualization solution adopts in-band method more, can the better tissues storage resources in the isomorphism storage networking environment of certain scale, simplification is to the management of these resources, provide unified virtual memory facilities to application, but because its all virtual data all will be handled by virtual node, therefore virtual node is easy to become system bottleneck, the expansion of system also is subjected to the restriction of virtual node processing ability on the other hand, in conjunction with realizing cost, autgmentability, many-sided factor such as storage device performance considers that existing scheme will be difficult to effectively realization to the management of magnanimity isomerism storage resources.
At above problem, the user needs the Storage Virtualization method of more effectively managing towards mass memory badly, and the magnanimity heterogeneous storage system is carried out dynamic management, realizes that the out-band method piece level of intelligence is virtual, and using for storage provides flexible, uniform platform.
Summary of the invention
The present invention proposes a kind of Storage Virtualization method towards mass memory management, solves storage resources high complexity in the existing magnanimity isomery storage, problem in large scale, to improve utilization ratio of storage resources and to reduce the management cost of storage system.
A kind of Storage Virtualization method towards the mass memory management of the present invention comprises the following steps:
(1) storage resources is found step, moves on each memory node, and order is carried out following substep:
(1.1) with the attribute of this memory node of XML file description, form the storage resources file; The storage resources file comprises memory node title, memory capacity, bandwidth, the network address and ID number;
(1.2) judging whether operate as normal of this memory node second every T, is then the storage resources file to be sent to management node; Otherwise do not send the storage resources file;
(2) storage resources is acted on behalf of step, moves having on the user node of storage resource demands, and order is carried out following substep:
(2.1) with XML file description user's storage resource demands, form the resource requirement file; The resource requirement file comprises the operating system of user node title, the network address, use and ID number, the title of required virtual memory facilities, capacity, bandwidth and disk array grade;
(2.2) the resource requirement file is sent to management node, the needed storage resources of application user;
(2.3) wait for the resource allocation file of management node, receive the resource allocation file after, create virtual memory facilities according to the resource allocation file for the user;
(2.4) check whether virtual memory facilities creates success, is then to use XML file description storage resources operating position, form resource and use file, send to management node; Otherwise, send to management node with the error message of XML file description; The operating system of the user node title that resource is used file to comprise to distribute virtual memory facilities, the network address, use and ID number, distribute storing virtual device name, memory capacity, bandwidth and ID number, memory node memory capacity, the network address, bandwidth, residual capacity and ID number of using of all memory node titles, memory capacity, bandwidth, the network address and ID number, virtual memory facilities;
(3) the storage resources allocation step is moved on management node, comprises following substep:
(3.1) judging whether to receive the storage resources file that sends from each memory node second every T, is then the storage resources file that is received to be updated in the database; Otherwise think that this memory node lost efficacy, the storage resources file that deletion is corresponding, T=3,4,5,6,7 or 8;
Described renewal is meant if there is old storage resources file in the database, then replaces this old storage resources file, does not have old storage resources file in the database, then preserves this storage resources file;
(3.2) receive the resource requirement file that sends from each user node, the storage resources file in the searching database carries out substep (3.3);
(3.3) seek storage resources, use the XML file description, form the resource allocation file, carry out substep (3.4); Memory node ID number of comprising that the virtual memory facilities title, memory capacity, use memory node quantity, disk array grade of distribution and user node ID number, virtual memory facilities use of resource allocation file, memory capacity, the network address, bandwidth and use capacity;
(3.4) the resource allocation file is sent to user node, carry out substep (3.5);
(3.5) receive the resource that sends from each user node and use file, be updated in the database.
In the described storage resources allocation step, substep (3.3) order is carried out following process:
A. the storage resources file in the scan database, whether the bandwidth of judging each storage resources in the storage resources file is more than or equal to the bandwidth of resource requirement documentation requirements, be then with satisfactory storage resources, arrange from small to large by bandwidth and generate chained list L, turn over journey B; Otherwise select storage resources, generate chained list L, turn over journey B near the bandwidth of resource requirement documentation requirements;
B. from the gauge outfit of chained list L, whether the memory capacity of judging node among the chained list L is then to choose this node more than or equal to the memory capacity of resource requirement documentation requirements, turns over journey C; Otherwise the memory capacity of former resource requirement documentation requirements is deducted the memory capacity of the memory capacity of this node as the resource requirement documentation requirements,, turn over journey B the next node of former chained list L gauge outfit node gauge outfit as chained list L;
C. with memory capacity, bandwidth and the network address of the node chosen at last among the chained list L, use the XML file description, form the resource allocation file.
Storage resources is found step: at system's run duration, the system manager gathers the attribute information of the storage resources of the new adding of needs system, periodically to management node report memory node survival information, is used for storage resources is joined system; Whether management node is stopped using because of failure and other reasons by survival information detection of stored resource, avoids the wrong storage resources that uses.
Storage resources is acted on behalf of step: the user sends to management node with the storage demand of oneself, and application can be satisfied the storage resources of user's needs; Behind the feedback information that receives management node, read resource allocation file in the feedback information, create virtual memory facilities by the resource allocation file for the user.
Storage resources allocation step: be used for concentrating the storage resources information and the virtual unit information of preserving and safeguarding mass storage system (MSS), and carry out storage resources and distribute.The transmission path of management node and user node is that control path and data path are independent, can alleviate the transfer of data pressure of virtual memory facilities like this, improves the performance of virtual memory facilities.
The present invention has improved utilization ratio of storage resources by the reasonable distribution of memory capacity in the whole memory space and memory property; By independently controlling and data transfer path, improved the performance of transfer of data; By for the enterprise's storage that disperses provides concentrated management, reduced the management cost of storage system, be applicable to towards the virtual management of magnanimity heterogeneous storage system.
Description of drawings
Fig. 1 is the schematic diagram of the embodiment of the invention;
The storage system schematic diagram that Fig. 2 uses for the embodiment of the invention;
Fig. 3 finds the step schematic diagram for storage resources;
Fig. 4 is a storage resources Files XML form schematic diagram;
Fig. 5 acts on behalf of the step schematic diagram for storage resources;
Fig. 6 is a resource requirement Files XML form schematic diagram;
Fig. 7 uses Files XML form schematic diagram for resource;
Fig. 8 is a storage resources allocation step schematic diagram;
Fig. 9 is a resource allocation Files XML form schematic diagram;
Figure 10 is substep (3.3) schematic diagram of storage resources allocation step.
Embodiment
As shown in Figure 1, the present invention includes: (1) storage resources is found step; (2) storage resources is acted on behalf of step; (3) storage resources allocation step.
As shown in Figure 2, suppose that the storage system that the embodiment of the invention is used comprises memory node, user node and management node, memory node is fibre channel storage, internet small computer system interface memory device, infinite bandwidth memory device, represent with memory node D1, D2 and D3 respectively, user node comprises user node H1, H2 and H3, management node is M1, and user node H1 need create virtual memory facilities LV1 and cross over memory node D1 and D2.
The embodiment of the invention comprises:
(1) storage resources is found step, moves on each memory node, and as shown in Figure 3, order is carried out following substep:
(1.1) with the attribute of XML file description memory node D1, D2 and D3, comprise memory capacity, bandwidth, the network address, form the storage resources file, as shown in Figure 4;
(1.2) judging self whether operate as normal every 5 seconds memory node D1, D2 and D3, is then the storage resources file to be sent to management node; Otherwise do not send the storage resources file;
(2) storage resources is acted on behalf of step, moves on user node H1, and as shown in Figure 5, order is carried out following substep:
(2.1) with the storage resource demands of XML file description user node H1, form the resource requirement file, as shown in Figure 6;
(2.2) the resource requirement file is sent to management node M1, the needed storage resources of application user node H1;
(2.3) wait for the resource allocation file of management node M1, receive the resource allocation file after, be that user node H1 creates virtual memory facilities LV1 according to the resource allocation file;
(2.4) check whether virtual memory facilities LV1 creates success, is then to use XML file description storage resources operating position, form resource and use file, send to management node M1; Otherwise, send to management node M1 with the error message of XML file description; Resource is used file as shown in Figure 7;
(3) storage resources allocation step is moved on management node M1, and as shown in Figure 8, order is carried out following substep:
(3.1) judging every 5 seconds whether receive the storage resources file that sends from memory node D1, D2, D3, is then the storage resources file that is received to be updated in the database; Otherwise think that this memory node lost efficacy, the storage resources file that deletion is corresponding;
Described renewal is meant if there is old storage resources file in the database, then replaces this old storage resources file, does not have old storage resources file in the database, then preserves this storage resources file;
(3.2) receive the resource requirement file that sends from user node H1, the storage resources file in the searching database,
(3.3) seek storage resources, use the XML file description, form the resource allocation file, as shown in Figure 9;
(3.4) the resource allocation file is sent to user node H1;
(3.5) receive the resource that sends from user node H1, H2, H3 and use file, be updated in the database.
As shown in figure 10, the substep of storage resources allocation step (3.3) order is carried out following process:
A. the storage resources file in the scan database, whether the bandwidth of judging each storage resources in the storage resources file is more than or equal to the bandwidth of resource requirement documentation requirements, be then with satisfactory storage resources, arrange from small to large by bandwidth and generate chained list L, turn over journey B; Otherwise select storage resources, generate chained list L, turn over journey B near the bandwidth of resource requirement documentation requirements;
B. from the gauge outfit of chained list L, whether the memory capacity of judging node among the chained list L is then to choose this node more than or equal to the memory capacity of resource requirement documentation requirements, turns over journey C; Otherwise the memory capacity of former resource requirement documentation requirements is deducted the memory capacity of the memory capacity of this node as the resource requirement documentation requirements,, turn over journey B the next node of former chained list L gauge outfit node gauge outfit as chained list L;
C. with memory capacity, bandwidth and the network address of the node chosen at last among the chained list L, use the XML file description, form the resource allocation file.
Fig. 4 is a storage resources Files XML form schematic diagram: element LVML defined file is a kind of XML formatted file of the Storage Virtualization method towards mass memory management, wherein attribute UUID defines ID number of this XML formatted file, attribute description defines the descriptor of this XML formatted file, and attribute version defines the version number of this XML formatted file; It is the storage resources file that element DiskInfo defines this XML formatted file; Element Disk defines a memory node, wherein, and the title of attribute DiskName definition memory node, the capacity of attribute DiskSize definition memory node, the network address of attribute DiskIP definition memory node, attribute Bandwidth defines bandwidth information, and attribute UUID defines ID number of memory node.
Fig. 6 is a resource requirement Files XML form schematic diagram: it is the resource requirement file that element LvmReq defines this XML formatted file; Element Host defines a user node, wherein attribute Format defines the operating system that user node uses, value is 1 to be (SuSE) Linux OS, value is 2 to be Windows operating system, the network address of attribute HostIP definition user node, the title of attribute HostName definition user node, attribute UUID defines ID number of user node; The virtual memory facilities of element LogicVolume definition user node demand, the title of attribute LVName defining virtual memory device wherein, the capacity of attribute LVSize defining virtual memory device, the disk array grade of attribute Raid defining virtual memory device, value is the-1st, non-disk array, value is 0 for RAID0, and value is 1 to be RAID1, and value is 5 to be RAID5.
Fig. 7 uses Files XML form schematic diagram for resource: element LvmInfo defines this XML formatted file and uses file as resource; The virtual memory facilities that element LogicVolume definition is created, wherein user node ID number of this virtual memory facilities of attribute HostUUID definition use, attribute UUID defining virtual memory device I D number; Memory node of element Disk, the wherein untapped capacity of attribute FreeSize definition memory node
Fig. 9 is a resource allocation Files XML form schematic diagram: it is the resource allocation file that element DiskAllo defines this XML formatted file; The virtual memory facilities that element LogicVolume definition distributes, wherein the memory node number of attribute DiskNum defining virtual memory device use; The memory node that element Disk definition distributes, wherein the memory node capacity that virtual memory facilities uses is distributed in attribute UsedSize definition.
Claims (2)
1. the Storage Virtualization method towards the mass memory management comprises the following steps:
(1) storage resources is found step, moves on each memory node, and order is carried out following substep:
(1.1) with the attribute of this memory node of XML file description, form the storage resources file; The storage resources file comprises memory node title, memory capacity, bandwidth, the network address and ID number;
(1.2) judging whether operate as normal of this memory node second every T, is then the storage resources file to be sent to management node; Otherwise do not send the storage resources file;
(2) storage resources is acted on behalf of step, moves having on the user node of storage resource demands, and order is carried out following substep:
(2.1) with XML file description user's storage resource demands, form the resource requirement file; The resource requirement file comprises the operating system of user node title, the network address, use and ID number, the title of required virtual memory facilities, capacity, bandwidth and disk array grade;
(2.2) the resource requirement file is sent to management node, the needed storage resources of application user;
(2.3) wait for the resource allocation file of management node, receive the resource allocation file after, create virtual memory facilities according to the resource allocation file for the user;
(2.4) check whether virtual memory facilities creates success, is then to use XML file description storage resources operating position, form resource and use file, send to management node; Otherwise, send to management node with the error message of XML file description; The operating system of the user node title that resource is used file to comprise to distribute virtual memory facilities, the network address, use and ID number, distribute storing virtual device name, memory capacity, bandwidth and ID number, memory node memory capacity, the network address, bandwidth, residual capacity and ID number of using of all memory node titles, memory capacity, bandwidth, the network address and ID number, virtual memory facilities;
(3) the storage resources allocation step is moved on management node, comprises following substep:
(3.1) judging whether to receive the storage resources file that sends from each memory node second every T, is then the storage resources file that is received to be updated in the database; Otherwise think that this memory node lost efficacy, the storage resources file that deletion is corresponding, T=3,4,5,6,7 or 8;
Described renewal is meant if there is old storage resources file in the database, then replaces this old storage resources file, does not have old storage resources file in the database, then preserves this storage resources file;
(3.2) receive the resource requirement file that sends from each user node, the storage resources file in the searching database carries out substep (3.3);
(3.3) seek storage resources, use the XML file description, form the resource allocation file, carry out substep (3.4); Memory node ID number of comprising that the virtual memory facilities title, memory capacity, use memory node quantity, disk array grade of distribution and user node ID number, virtual memory facilities use of resource allocation file, memory capacity, the network address, bandwidth and use capacity;
(3.4) the resource allocation file is sent to user node, carry out substep (3.5);
(3.5) receive the resource that sends from each user node and use file, be updated in the database.
2. the Storage Virtualization method towards the mass memory management as claimed in claim 1 is characterized in that:
In the described storage resources allocation step, substep (3.3) order is carried out following process:
A. the storage resources file in the scan database, whether the bandwidth of judging each storage resources in the storage resources file is more than or equal to the bandwidth of resource requirement documentation requirements, be then with satisfactory storage resources, arrange from small to large by bandwidth and generate chained list L, turn over journey B; Otherwise select storage resources, generate chained list L, turn over journey B near the bandwidth of resource requirement documentation requirements;
B. from the gauge outfit of chained list L, whether the memory capacity of judging node among the chained list L is then to choose this node more than or equal to the memory capacity of resource requirement documentation requirements, turns over journey C; Otherwise the memory capacity of former resource requirement documentation requirements is deducted the memory capacity of the memory capacity of this node as the resource requirement documentation requirements,, turn over journey B the next node of former chained list L gauge outfit node gauge outfit as chained list L;
C. with memory capacity, bandwidth and the network address of the node chosen at last among the chained list L, use the XML file description, form the resource allocation file.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN 201110025361 CN102123171B (en) | 2011-01-24 | 2011-01-24 | Storage virtualization method for mass storage management |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN 201110025361 CN102123171B (en) | 2011-01-24 | 2011-01-24 | Storage virtualization method for mass storage management |
Publications (2)
Publication Number | Publication Date |
---|---|
CN102123171A true CN102123171A (en) | 2011-07-13 |
CN102123171B CN102123171B (en) | 2013-06-05 |
Family
ID=44251622
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN 201110025361 Active CN102123171B (en) | 2011-01-24 | 2011-01-24 | Storage virtualization method for mass storage management |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN102123171B (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103036952A (en) * | 2012-11-30 | 2013-04-10 | 中国电力科学研究院 | Enterprise-level heterogeneous fusion memory management system |
CN107636598A (en) * | 2015-07-15 | 2018-01-26 | 思科技术公司 | Bid/charge agreement in NVMe storage devices extending transversely |
CN111399761A (en) * | 2019-11-19 | 2020-07-10 | 杭州海康威视系统技术有限公司 | Storage resource allocation method, device and equipment and storage medium |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1955912A (en) * | 2006-10-13 | 2007-05-02 | 清华大学 | Method for distributing resource in large scale storage system |
CN101496005A (en) * | 2005-12-29 | 2009-07-29 | 亚马逊科技公司 | Distributed replica storage system with web services interface |
-
2011
- 2011-01-24 CN CN 201110025361 patent/CN102123171B/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101496005A (en) * | 2005-12-29 | 2009-07-29 | 亚马逊科技公司 | Distributed replica storage system with web services interface |
CN1955912A (en) * | 2006-10-13 | 2007-05-02 | 清华大学 | Method for distributing resource in large scale storage system |
Non-Patent Citations (3)
Title |
---|
《Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing, 2006. SNPD 2006. Seventh ACIS International Conference on》 20060620 Jie Xiao,et al Flexible Metadata Management for Block-level Storage System 第127-132页 , * |
JIE XIAO,ET AL: "Flexible Metadata Management for Block–level Storage System", 《SOFTWARE ENGINEERING, ARTIFICIAL INTELLIGENCE, NETWORKING, AND PARALLEL/DISTRIBUTED COMPUTING, 2006. SNPD 2006. SEVENTH ACIS INTERNATIONAL CONFERENCE ON》 * |
冯 丹,叶 飞,施 展: "VISA 存储网络系统元数据管理的研究", 《华 中 科 技 大 学 学 报(自然科学版)》 * |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103036952A (en) * | 2012-11-30 | 2013-04-10 | 中国电力科学研究院 | Enterprise-level heterogeneous fusion memory management system |
CN103036952B (en) * | 2012-11-30 | 2016-04-20 | 国家电网公司 | A kind of enterprise-level isomery merges storage management system |
CN107636598A (en) * | 2015-07-15 | 2018-01-26 | 思科技术公司 | Bid/charge agreement in NVMe storage devices extending transversely |
CN107636598B (en) * | 2015-07-15 | 2021-06-01 | 思科技术公司 | Bid/ask agreement in laterally extended NVMe storage |
CN111399761A (en) * | 2019-11-19 | 2020-07-10 | 杭州海康威视系统技术有限公司 | Storage resource allocation method, device and equipment and storage medium |
Also Published As
Publication number | Publication date |
---|---|
CN102123171B (en) | 2013-06-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20210263658A1 (en) | Data system with flush views | |
US9170743B2 (en) | Management method of virtual storage system and remote copy system | |
CN103797770B (en) | A kind of method and system of shared storage resources | |
CN101997918B (en) | Method for allocating mass storage resources according to needs in heterogeneous SAN (Storage Area Network) environment | |
KR101137213B1 (en) | System and method for recovery units in databases | |
JP6056453B2 (en) | Program, data management method, and information processing apparatus | |
US20180232308A1 (en) | Data system with data flush mechanism | |
CN102123176B (en) | Space distribution and management method and device for network storage system | |
US20230054415A1 (en) | Managing containers on a data storage system | |
AU2017387062A1 (en) | Data storage system with redundant internal networks | |
JP2004110218A (en) | Virtual volume creation/management method for dbms | |
US10871911B2 (en) | Reducing data amplification when replicating objects across different sites | |
US11262918B1 (en) | Data storage system with uneven drive wear reduction | |
AU2017387063A1 (en) | Data storage system with multiple durability levels | |
CN102937964B (en) | Intelligent data service method based on distributed system | |
CN102868727B (en) | Method for realizing high availability of logical volume | |
WO2021057108A1 (en) | Data reading method, data writing method, and server | |
US11853587B2 (en) | Data storage system with configurable durability | |
WO2019148841A1 (en) | Distributed storage system, data processing method and storage node | |
CN103067488A (en) | Implement method of unified storage | |
CN104050248A (en) | File storage system and storage method | |
JP2021524104A (en) | Master / Standby Container System Switching | |
KR101719116B1 (en) | Efficient resource utilization in data centers | |
CN102982182A (en) | Data storage planning method and device | |
CN102123171B (en) | Storage virtualization method for mass storage management |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant |