CN102123171B - Storage virtualization method for mass storage management - Google Patents
Storage virtualization method for mass storage management Download PDFInfo
- Publication number
- CN102123171B CN102123171B CN 201110025361 CN201110025361A CN102123171B CN 102123171 B CN102123171 B CN 102123171B CN 201110025361 CN201110025361 CN 201110025361 CN 201110025361 A CN201110025361 A CN 201110025361A CN 102123171 B CN102123171 B CN 102123171B
- Authority
- CN
- China
- Prior art keywords
- file
- node
- storage resources
- storage
- memory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention discloses a storage virtualization method for mass storage management, which belongs to the technical field of computer storage and solves the problems of highly-complicated and large-scale storage resources in the conventional mass heterogeneous storage so as to improve the utilization ratio of the storage resources and to reduce the management cost of a storage system. The storage virtualization method for the mass storage management comprises the following steps of: (1) discovery of the storage resources; (2) agent of the storage resources; and (3) allocation of the storage resources. In the method, the utilization ratio of the storage resources is improved by the reasonable allocation of storage capacity and storage performance in a whole storage space, data transmission performance is improved by independent control and data transmission paths, and the management cost of the storage system is reduced by providing centralized management for scattered enterprise storage. The storage virtualization method for the mass storage management is applied to virtualized management for a mass heterogeneous storage system.
Description
Technical field
The invention belongs to computer memory technical field, be specifically related to a kind of Storage Virtualization method towards the mass memory management.
Background technology
The mass storage system (MSS) of current application is the mass memory network that is made of memory node, management node, user node.Each memory node is made of memory device and the storing software of different vendor, has incompatible phenomenon between mutually, and functional regional division is complicated, equipment, protocol type are various, memory node quantity is huge, and various application load height mix, and causes system management memory complexity, the wasting of resources.Although propose already in the industry the demand of the standardization of products, but from realistic situation, integration, management for the isomery storage environment are still very undesirable, thereby cause a large amount of wastes and the overlapping investment of storage resources, but also affected the consistency access of data and all many-sides such as backup disaster tolerance of entire system.
Simultaneously, the magnanimity heterogeneous storage system has following characteristics: (1) isomerism is high, storage resources information high complexity, and the access expense is large; (2) in large scale, surpass single volume controller disposal ability, need many high performance volume controller equiments of configuration.
Storage virtualization technology has usually that device level is virtual, the subsystem irrespective of size is virtual, tape is virtual, the piece level is virtual and the virtual several types of file-level, implementation method comprises two kinds of band and out-of-bands, refer to transfer of data in band and control transmission same path, band refers to that outward transfer of data transmits different paths with control.Wherein device level and the virtual scale of subsystem irrespective of size are less, how are realized the virtual standby system of being more common in of tape, general virtual two types of the virtual and file-level of piece level that adopts in mass storage system (MSS) by firmware or hardware mode.traditional Storage Virtualization method adopts form in band more, be convenient to centralized control and management, take the storage volume controller of IBM as representative, its piece level Storage Virtualization solution adopts in-band method more, can organize preferably storage resources in the isomorphism storage networking environment of certain scale, the management of simplification to these resources, provide unified virtual memory facilities to application, but because its all virtual data all will be processed by virtual node, therefore virtual node is easy to become system bottleneck, the expansion of system also is subject to the restriction of virtual node processing ability on the other hand, in conjunction with realizing cost, autgmentability, the many factors such as storage device performance are considered, existing scheme will be difficult to the effectively management of realization to the magnanimity isomerism storage resources.
For above problem, the user needs the Storage Virtualization method of more effectively managing towards mass memory badly, and the magnanimity heterogeneous storage system is carried out dynamic management, realizes that the out-band method piece level of intelligence is virtual, and using for storage provides flexible, unified platform.
Summary of the invention
The present invention proposes a kind of Storage Virtualization method towards mass memory management, solves storage resources high complexity in existing magnanimity isomery storage, problem in large scale, to improve utilization ratio of storage resources and to reduce the management cost of storage system.
A kind of Storage Virtualization method towards the mass memory management of the present invention comprises the following steps:
(1) storage resources is found step, moves on each memory node, and order is carried out following substep:
(1.1) describe the attribute of this memory node with the XML file, form the storage resources file; The storage resources file comprises memory node title, memory capacity, bandwidth, the network address and No. ID;
(1.2) judge whether this memory node works, and is the storage resources file to be sent to management node second every T; Otherwise do not send the storage resources file;
(2) storage resource broker step is moved having on the user node of storage resource demands, and order is carried out following substep:
(2.1) describe user's storage resource demands with the XML file, form the resource requirement file; The resource requirement file comprises the operating system of user node title, the network address, use and No. ID, the title of required virtual memory facilities, capacity, bandwidth and disk array grade;
(2.2) the resource requirement file is sent to management node, the needed storage resources of application user;
(2.3) wait for the resource allocate file of management node, after receiving the resource allocate file, create virtual memory facilities according to the resource allocate file for the user;
(2.4) check whether virtual memory facilities creates successfully, is to describe the storage resources operating position with the XML file, form resource and use file, send to management node; Otherwise describe error message with the XML file, send to management node; The operating system of the user node title that resource is used file to comprise to distribute virtual memory facilities, the network address, use and No. ID, the storing virtual device name of distributing, memory capacity, bandwidth and No. ID, all memory node titles, memory capacity, bandwidth, the network address and No. ID, the memory node memory capacity that virtual memory facilities uses, the network address, bandwidth, residual capacity and No. ID;
(3) the storage resources allocation step, move on management node, comprises following substep:
(3.1) judge whether second to receive the storage resources file that sends from each memory node every T, be with the storage resources file update that receives in database; Otherwise think that this memory node lost efficacy, the storage resources file that deletion is corresponding, T=3,4,5,6,7 or 8;
Described renewal refers to if there is old storage resources file in database, replaces this old storage resources file, does not have old storage resources file in database, preserves this storage resources file;
(3.2) receive the resource requirement file that sends from each user node, the storage resources file in searching database carries out substep (3.3);
(3.3) seek storage resources, describe with the XML file, form the resource allocate file, carry out substep (3.4); The resource allocate file comprises the virtual memory facilities title, memory capacity, use memory node quantity, disk array grade of distribution and user node No. ID, No. ID, memory node, memory capacity, the network address, bandwidth and use capacity that virtual memory facilities uses;
(3.4) the resource allocate file is sent to user node, carry out substep (3.5);
(3.5) receive the resource that sends from each user node and use file, be updated in database.
In described storage resources allocation step, substep (3.3) order is carried out following process:
A. the storage resources file in scan database, whether the bandwidth that judges each storage resources in the storage resources file is more than or equal to the bandwidth of resource requirement documentation requirements, be with satisfactory storage resources, arrange from small to large by bandwidth and generate chained list L, turn over journey B; Otherwise select the storage resources near the bandwidth of resource requirement documentation requirements, generate chained list L, turn over journey B;
B. from the gauge outfit of chained list L, judge that the memory capacity of node in chained list L whether more than or equal to the memory capacity of resource requirement documentation requirements, is to choose this node, turn over journey C; Otherwise the memory capacity of former resource requirement documentation requirements is deducted the memory capacity of this node as the memory capacity of resource requirement documentation requirements, with the next node of the former chained list L gauge outfit node gauge outfit as chained list L, turn over journey B;
C. with memory capacity, bandwidth and the network address of the node chosen at last in chained list L, describe with the XML file, form the resource allocate file.
Storage resources is found step: at system's run duration, the system manager gathers the attribute information that need to newly add the storage resources of system, periodically to management node report memory node survival information, is used for storage resources is joined system; Whether management node is stopped using because of failure and other reasons by survival information detection of stored resource, avoids the wrong storage resources that uses.
The storage resource broker step: the user sends to management node with the storage demand of oneself, and application can be satisfied the storage resources of user's needs; After the feedback information that receives management node, read resource allocate file in feedback information, create virtual memory facilities by the resource allocate file for the user.
Storage resources allocation step: be used for concentrating storage resources information and the virtual unit information of preserving and safeguarding mass storage system (MSS), and carry out storage resources and distribute.The transmission path of management node and user node is to control the path, and data path is independent, can alleviate like this transfer of data pressure of virtual memory facilities, improves the performance of virtual memory facilities.
The present invention has improved utilization ratio of storage resources by the reasonable distribution of the memory capacity in whole memory space and memory property; By independently controlling and data transfer path, improved the performance of transfer of data; By for the enterprise's storage that disperses provides concentrated management, reduced the management cost of storage system, be applicable to towards the virtual management of magnanimity heterogeneous storage system.
Description of drawings
Fig. 1 is the schematic diagram of the embodiment of the present invention;
Fig. 2 is the storage system schematic diagram that the embodiment of the present invention is used;
Fig. 3 is that storage resources is found the step schematic diagram;
Fig. 4 is storage resources Files XML form schematic diagram;
Fig. 5 is storage resource broker step schematic diagram;
Fig. 6 is resource requirement Files XML form schematic diagram;
Fig. 7 is that resource is used Files XML form schematic diagram;
Fig. 8 is storage resources allocation step schematic diagram;
Fig. 9 is resource allocate file XML form schematic diagram;
Figure 10 is substep (3.3) schematic diagram of storage resources allocation step.
Embodiment
As shown in Figure 1, the present invention includes: (1) storage resources is found step; (2) storage resource broker step; (3) storage resources allocation step.
As shown in Figure 2, suppose that the storage system that the embodiment of the present invention is used comprises memory node, user node and management node, memory node is fibre channel storage, internet small computer system interface memory device, infinite bandwidth memory device, represent with memory node D1, D2 and D3 respectively, user node comprises user node H1, H2 and H3, management node is M1, and user node H1 need to create virtual memory facilities LV1 and cross over memory node D1 and D2.
The embodiment of the present invention comprises:
(1) storage resources is found step, moves on each memory node, and as shown in Figure 3, order is carried out following substep:
(1.1) describe the attribute of memory node D1, D2 and D3 with the XML file, comprise memory capacity, bandwidth, the network address, form the storage resources file, as shown in Figure 4;
(1.2) judge every 5 seconds memory node D1, D2 and D3 whether self works, and is the storage resources file to be sent to management node; Otherwise do not send the storage resources file;
(2) storage resource broker step is moved on user node H1, and as shown in Figure 5, order is carried out following substep:
(2.1) describe the storage resource demands of user node H1 with the XML file, form the resource requirement file, as shown in Figure 6;
(2.2) the resource requirement file is sent to management node M1, the needed storage resources of application user node H1;
(2.3) waiting for the resource allocate file of management node M1, after receiving the resource allocate file, is that user node H1 creates virtual memory facilities LV1 according to the resource allocate file;
(2.4) check whether virtual memory facilities LV1 creates successfully, is to describe the storage resources operating position with the XML file, form resource and use file, send to management node M1; Otherwise describe error message with the XML file, send to management node M1; Resource is used file as shown in Figure 7;
(3) storage resources allocation step is moved on management node M1, and as shown in Figure 8, order is carried out following substep:
(3.1) whether judgement received the storage resources file that sends from memory node D1, D2, D3 every 5 seconds, be with the storage resources file update that receives in database; Otherwise think that this memory node lost efficacy, the storage resources file that deletion is corresponding;
Described renewal refers to if there is old storage resources file in database, replaces this old storage resources file, does not have old storage resources file in database, preserves this storage resources file;
(3.2) receive the resource requirement file that sends from user node H1, the storage resources file in searching database,
(3.3) seek storage resources, describe with the XML file, form the resource allocate file, as shown in Figure 9;
(3.4) the resource allocate file is sent to user node H1;
(3.5) receive the resource that sends from user node H1, H2, H3 and use file, be updated in database.
As shown in figure 10, the substep of storage resources allocation step (3.3) order is carried out following process:
A. the storage resources file in scan database, whether the bandwidth that judges each storage resources in the storage resources file is more than or equal to the bandwidth of resource requirement documentation requirements, be with satisfactory storage resources, arrange from small to large by bandwidth and generate chained list L, turn over journey B; Otherwise select the storage resources near the bandwidth of resource requirement documentation requirements, generate chained list L, turn over journey B;
B. from the gauge outfit of chained list L, judge that the memory capacity of node in chained list L whether more than or equal to the memory capacity of resource requirement documentation requirements, is to choose this node, turn over journey C; Otherwise the memory capacity of former resource requirement documentation requirements is deducted the memory capacity of this node as the memory capacity of resource requirement documentation requirements, with the next node of the former chained list L gauge outfit node gauge outfit as chained list L, turn over journey B;
C. with memory capacity, bandwidth and the network address of the node chosen at last in chained list L, describe with the XML file, form the resource allocate file.
Fig. 4 is storage resources Files XML form schematic diagram: element LVML defined file is a kind of XML formatted file of the Storage Virtualization method towards mass memory management, wherein attribute UUID defines No. ID of this XML formatted file, attribute description defines the descriptor of this XML formatted file, and attribute version defines the version number of this XML formatted file; It is the storage resources file that element DiskInfo defines this XML formatted file; Element Disk defines a memory node, wherein, and the title of attribute DiskName definition memory node, the capacity of attribute DiskSize definition memory node, the network address of attribute DiskIP definition memory node, attribute Bandwidth defines bandwidth information, and attribute UUID defines No. ID of memory node.
Fig. 6 is resource requirement Files XML form schematic diagram: it is the resource requirement file that element LvmReq defines this XML formatted file; Element Host defines a user node, wherein attribute Format defines the operating system that user node uses, value is 1 to be (SuSE) Linux OS, value is 2 to be Windows operating system, the network address of attribute HostIP definition user node, the title of attribute HostName definition user node, attribute UUID defines No. ID of user node; The virtual memory facilities of element LogicVolume definition user node demand, the title of attribute LVName defining virtual memory device wherein, the capacity of attribute LVSize defining virtual memory device, the disk array grade of attribute Raid defining virtual memory device, value is the-1st, non-disk array, value is 0 for RAID0, and value is 1 to be RAID1, and value is 5 to be RAID5.
Fig. 7 is that resource is used Files XML form schematic diagram: element LvmInfo defines this XML formatted file and uses file as resource; The virtual memory facilities that element LogicVolume definition creates, wherein No. ID, the user node of this virtual memory facilities is used in attribute HostUUID definition, No. ID of attribute UUID defining virtual memory device; Memory node of element Disk, the wherein untapped capacity of attribute FreeSize definition memory node
Fig. 9 is resource allocate file XML form schematic diagram: it is the resource allocate file that element DiskAllo defines this XML formatted file; The virtual memory facilities that element LogicVolume definition distributes, wherein the memory node number of attribute DiskNum defining virtual memory device use; The memory node that element Disk definition distributes, wherein the memory node capacity that virtual memory facilities uses is distributed in attribute UsedSize definition.
Claims (2)
1. the Storage Virtualization method towards the mass memory management, comprise the following steps:
(1) storage resources is found step, moves on each memory node, and order is carried out following substep:
(1.1) describe the attribute of this memory node with the XML file, form the storage resources file; The storage resources file comprises memory node title, memory capacity, bandwidth, the network address and No. ID;
(1.2) judge whether this memory node works, and is the storage resources file to be sent to management node second every T; Otherwise do not send the storage resources file; T=3,4,5,6,7 or 8;
(2) storage resource broker step is moved having on the user node of storage resource demands, and order is carried out following substep:
(2.1) describe user's storage resource demands with the XML file, form the resource requirement file; The resource requirement file comprises the operating system of user node title, the network address, use and No. ID, the title of required virtual memory facilities, capacity, bandwidth and disk array grade;
(2.2) the resource requirement file is sent to management node, the needed storage resources of application user;
(2.3) wait for the resource allocate file of management node, after receiving the resource allocate file, create virtual memory facilities according to the resource allocate file for the user;
(2.4) check whether virtual memory facilities creates successfully, is to describe the storage resources operating position with the XML file, form resource and use file, send to management node; Otherwise describe error message with the XML file, send to management node; The operating system of the user node title that resource is used file to comprise to distribute virtual memory facilities, the network address, use and No. ID, the storing virtual device name of distributing, memory capacity, bandwidth and No. ID, all memory node titles, memory capacity, bandwidth, the network address and No. ID, the memory node memory capacity that virtual memory facilities uses, the network address, bandwidth, residual capacity and No. ID;
(3) the storage resources allocation step, move on management node, comprises following substep:
(3.1) judge whether second to receive the storage resources file that sends from each memory node every T, be with the storage resources file update that receives in database; Otherwise think that this memory node lost efficacy, the storage resources file that deletion is corresponding;
Described renewal refers to if there is old storage resources file in database, replaces this old storage resources file, does not have old storage resources file in database, preserves this storage resources file;
(3.2) receive the resource requirement file that sends from each user node, the storage resources file in searching database carries out substep (3.3);
(3.3) seek storage resources, describe with the XML file, form the resource allocate file, carry out substep (3.4); The resource allocate file comprises the virtual memory facilities title, memory capacity, use memory node quantity, disk array grade of distribution and user node No. ID, No. ID, memory node, memory capacity, the network address, bandwidth and use capacity that virtual memory facilities uses;
(3.4) the resource allocate file is sent to user node, carry out substep (3.5);
(3.5) receive the resource that sends from each user node and use file, be updated in database.
2. the Storage Virtualization method towards mass memory management as claimed in claim 1 is characterized in that:
In described storage resources allocation step, substep (3.3) order is carried out following process:
A. the storage resources file in scan database, whether the bandwidth that judges each storage resources in the storage resources file is more than or equal to the bandwidth of resource requirement documentation requirements, be with satisfactory storage resources, arrange from small to large by bandwidth and generate chained list L, turn over journey B; Otherwise select the storage resources near the bandwidth of resource requirement documentation requirements, generate chained list L, turn over journey B;
B. from the gauge outfit of chained list L, judge that the memory capacity of node in chained list L whether more than or equal to the memory capacity of resource requirement documentation requirements, is to choose this node, turn over journey C; Otherwise the memory capacity of former resource requirement documentation requirements is deducted the memory capacity of this node as the memory capacity of resource requirement documentation requirements, with the next node of the former chained list L gauge outfit node gauge outfit as chained list L, turn over journey B;
C. with memory capacity, bandwidth and the network address of the node chosen at last in chained list L, describe with the XML file, form the resource allocate file.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110025361 CN102123171B (en) | 2011-01-24 | 2011-01-24 | Storage virtualization method for mass storage management |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110025361 CN102123171B (en) | 2011-01-24 | 2011-01-24 | Storage virtualization method for mass storage management |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102123171A CN102123171A (en) | 2011-07-13 |
| CN102123171B true CN102123171B (en) | 2013-06-05 |
Family
ID=44251622
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201110025361 Active CN102123171B (en) | 2011-01-24 | 2011-01-24 | Storage virtualization method for mass storage management |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102123171B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103036952B (en) * | 2012-11-30 | 2016-04-20 | 国家电网公司 | A kind of enterprise-level isomery merges storage management system |
| US10778765B2 (en) * | 2015-07-15 | 2020-09-15 | Cisco Technology, Inc. | Bid/ask protocol in scale-out NVMe storage |
| CN111399761B (en) * | 2019-11-19 | 2023-06-30 | 杭州海康威视系统技术有限公司 | Storage resource allocation method, device and equipment, and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1955912A (en) * | 2006-10-13 | 2007-05-02 | 清华大学 | Method for distributing resource in large scale storage system |
| CN101496005A (en) * | 2005-12-29 | 2009-07-29 | 亚马逊科技公司 | Distributed replica storage system with web services interface |
-
2011
- 2011-01-24 CN CN 201110025361 patent/CN102123171B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101496005A (en) * | 2005-12-29 | 2009-07-29 | 亚马逊科技公司 | Distributed replica storage system with web services interface |
| CN1955912A (en) * | 2006-10-13 | 2007-05-02 | 清华大学 | Method for distributing resource in large scale storage system |
Non-Patent Citations (3)
| Title |
|---|
| Flexible Metadata Management for Block–level Storage System;Jie Xiao,et al;《Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing, 2006. SNPD 2006. Seventh ACIS International Conference on》;20060620;第127-132页 * |
| JieXiao et al.Flexible Metadata Management for Block–level Storage System.《Software Engineering |
| 冯 丹，叶 飞，施 展.VISA 存储网络系统元数据管理的研究.《华 中 科 技 大 学 学 报(自然科学版)》.2007,第35卷(第3期),第49-52页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102123171A (en) | 2011-07-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7118230B2 (en) | VM/container and volume allocation determination method and storage system in HCI environment | |
| US20210263658A1 (en) | Data system with flush views | |
| US11438411B2 (en) | Data storage system with redundant internal networks | |
| US10521135B2 (en) | Data system with data flush mechanism | |
| US10628043B1 (en) | Systems and methods for implementing a horizontally federated heterogeneous cluster | |
| CN103797770B (en) | A kind of method and system of shared storage resources | |
| CN102082692B (en) | Method and equipment for migrating virtual machines based on network data flow direction, and cluster system | |
| CN102123176B (en) | Space distribution and management method and device for network storage system | |
| US11789840B2 (en) | Managing containers on a data storage system | |
| JP2018022529A (en) | Scalable distributed storage architecture | |
| US11262918B1 (en) | Data storage system with uneven drive wear reduction | |
| AU2017387063A1 (en) | Data storage system with multiple durability levels | |
| US10871911B2 (en) | Reducing data amplification when replicating objects across different sites | |
| JP2004110218A (en) | Virtual volume creation/management method for dbms | |
| CN101997918A (en) | Method for allocating mass storage resources according to needs in heterogeneous SAN (Storage Area Network) environment | |
| US8386707B2 (en) | Virtual disk management program, storage device management program, multinode storage system, and virtual disk managing method | |
| JP2014123218A (en) | Program, data management method, and information processing device | |
| US11182096B1 (en) | Data storage system with configurable durability | |
| CN105095317A (en) | Distributive database service management system | |
| CN102868727B (en) | Method for realizing high availability of logical volume | |
| WO2021057108A1 (en) | Data reading method, data writing method, and server | |
| JP2021524104A (en) | Master / Standby Container System Switching | |
| CN110119308B (en) | System for managing large-scale container applications | |
| CN102123171B (en) | Storage virtualization method for mass storage management | |
| US11681443B1 (en) | Durable data storage with snapshot storage space optimization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |