CN102664923A - Method for realizing shared storage pool by utilizing Linux global file system - Google Patents
Method for realizing shared storage pool by utilizing Linux global file system Download PDFInfo
- Publication number
- CN102664923A CN102664923A CN2012100887961A CN201210088796A CN102664923A CN 102664923 A CN102664923 A CN 102664923A CN 2012100887961 A CN2012100887961 A CN 2012100887961A CN 201210088796 A CN201210088796 A CN 201210088796A CN 102664923 A CN102664923 A CN 102664923A
- Authority
- CN
- China
- Prior art keywords
- file system
- storage
- logical
- client
- global
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a method for realizing a shared storage pool by utilizing a Linux global file system. A Linux global file system (GFS) is arranged in an SAN storage network, thereby realizing a small-scale cluster shared storage pool. According to the method, a shared storage pool for unified management, parallel reading and writing, data protection and elastic expansion can be provided for cluster nodes; SAN equipment-based cluster storage file system configures a physical storage raw device into a flexible logical storage volume group; storage space that is shared globally and is oriented to a node group is established on the logical volume group; a client mounts a GFS to a local end to realize parallel reading and writing operation on the file system; and a distributed log management system and a fault node isolation mechanism provide functions of data protection and disaster recovery for a shared storage pool.
Description
Technical field
The present invention relates to cluster and share field of storage, specifically a kind of method of utilizing the Linux global file system to realize sharing storage pool.
Background technology
Shared storage pool is a kind of for cluster provides file-sharing, concurrent reading and concurrent writing, the cluster storage system of unified management and data disaster tolerance ability.
The conventional cluster node adopts local unit file system usually, and a plurality of like this clustered node shared files need upgrade and cause the waste of storage resources like this at each node storage data trnascription not only bad for the communication of information.The unit file system is confined to the maintenance management of this region, lacks the uniform dispatching of cluster resource, and management cost is high, and the unit file system also is confined to the data disaster tolerance ability of single-point, lacks the fault transfer/recovery capability of concentrating type.
Universal day by day in cluster application, under the huge day by day development trend of scale, the file system of traditional local OS level is not suitable for storage demand already.The shared storage pool way of this invention can effectively solve the technical bottleneck of conventional store way.The unified named file of Linux overall situation system towards cluster client terminal provides the efficient data of network level to share, and supports a plurality of users reading and writing of files concurrently; Support online flexibly capacity extension and data balancing distribution technique; The unified resource model of the overall situation is convenient to the holistic management maintenance more; The file quotas administered of global level realizes unified scheduling of resource, and distributed log management system and Fault Isolation mechanism realize the effective protection and the disaster recovery of data.
Summary of the invention
The purpose of this invention is to provide a kind of Linux of utilization global file system and realize sharing the method for storage pool.
The objective of the invention is to realize by following mode; At first adopt the logical volume management instrument to set up logical storage volumes at SAN memory device piece; Then storage volume is formatted into the GFS global file system; Form the global file system of the unified NameSpace of nodes sharing, each client is mounted to this locality with the partitioned file system as required, concurrently file is carried out read-write operation;
The storage raw device of physics is configured to elasticity logical storage volumes group flexibly; On the logical volume group, set up the memory space of sharing towards the overall situation of node cluster; Client is file system mounted to local side with the overall situation; Concurrently file system is carried out read-write operation; Coordinate the conflict situations of a plurality of user writable identical file in addition through dynamic lock management mechanism, allow each carry client dynamic-configuration separately the metadata log record come data protection and fault recovery function to be provided to whole file system, wherein:
1) described logical volume management comprises physical storage medium is configured to the physical storage volume group, and again by a plurality of physical volume group construction logic storage volume groups, a plurality of logical volume groups form the partition holding unit of logical partition as global file system.This processing with tangible physics hardware logic volumeization helps the equiblibrium mass distribution and the flexible online expansion of later stage of data;
2) deployment of described global file system; Comprise logical partition is formatd; File system is at the carry of client; User's quotas administered, daily record configuration management and adopt dynamic lock management mechanism to coordinate of the read-write of a plurality of nodes, the inconsistent and wrong initialized situation of the data that the system accident of the avoiding machine of delaying brings to identical file;
3) described metadata daily record system and the malfunctioning node isolation mech isolation test that is distributed in each client is included in client and is global file system and dynamically add daily record data protection mechanism is provided; The real-time isolation mech isolation test of malfunctioning node realizes the fault recovery function, and behind Single Point of Faliure, in the consistency detection and the recovery of all the other working nodes completion data, the fault machine recovers the initialization correctly of back data like this.
The invention has the beneficial effects as follows: adopt the logical volume management instrument to set up logical storage volumes at SAN memory device piece; Then storage volume is formatted into the GFS global file system; Form the global file system of the unified NameSpace of nodes sharing; Each client is mounted to this locality with the partitioned file system as required, concurrently file is carried out read-write operation.The storage raw device of physics is configured to elasticity logical storage volumes group flexibly; On the logical volume group, set up the memory space of sharing towards the overall situation of node cluster; Client is file system mounted to local side with the overall situation, can carry out read-write operation to file system concurrently.Coordinate the conflict situations of a plurality of user writable identical file in addition through dynamic lock management mechanism, each carry client can dynamic-configuration separately the metadata log record come data protection and disaster recovery functionality to be provided to whole file system.
Description of drawings
Accompanying drawing 1 is to share the storage architecture sketch map;
Accompanying drawing 2 is global file system function structure sketch mapes.
Embodiment
Explanation at length below with reference to Figure of description method of the present invention being done.
What Fig. 1 showed is the deployment of global file system on the storage networking framework of traditional SAN formula, and global file system is deployed in shared memory block equipment, and each client node is mounted to the local storage pool function that realizes sharing with global file system.
Fig. 2 is the design way of showing the shared storage pool of realization that utilizes global file system by different level, is divided into system configuration, these 3 zones of entity instrument and functional characteristic.
The design way that the Linux global file system is deployed in the shared storage pool of storage SAN equipment realization cluster of the present invention; At first under clustered node Linux environment, adopt the logical volume management instrument to set up logical storage volumes at SAN memory device piece; Then storage volume is formatted into the GFS global file system; And when setting up file system the cluster-specific title; Form the cluster memory file system of the shared unified NameSpace of cluster interior nodes, each client node is mounted to this locality with the partitioned file system as required, concurrently file is carried out read-write operation.
The method for designing that this invention provides based on the cluster memory file system of SAN equipment; The storage raw device of physics is configured to elasticity logical storage volumes group flexibly; On the logical volume group, set up the memory space of sharing towards the overall situation of node cluster; Client is file system mounted to local side with the overall situation, can carry out read-write operation to file system concurrently.Coordinate the conflict situations of a plurality of user writable identical file in addition through dynamic lock management mechanism, avoid causing the file data mistake.Each client has separately metadata log record to constitute distributed log management system to come to whole file system data protection and disaster recovery functionality to be provided.When individual nodes is delayed machine, can pass through log information restore data information like this at other nodes.
Except that the described technical characterictic of specification, be the known technology of those skilled in the art.
Claims (2)
1. method of utilizing the Linux global file system realize to share storage pool; It is characterized in that; At first adopt the logical volume management instrument to set up logical storage volumes, then storage volume is formatted into the GFS global file system, form the global file system of the unified NameSpace of nodes sharing at SAN memory device piece; Each client is mounted to this locality with the partitioned file system as required, concurrently file is carried out read-write operation;
The storage raw device of physics is configured to elasticity logical storage volumes group flexibly; On the logical volume group, set up the memory space of sharing towards the overall situation of node cluster; Client is file system mounted to local side with the overall situation; Concurrently file system is carried out read-write operation; Coordinate the conflict situations of a plurality of user writable identical file in addition through dynamic lock management mechanism, allow each carry client dynamic-configuration separately the metadata log record come data protection and fault recovery function to be provided to whole file system, wherein:
1) described logical volume management; Comprise physical storage medium is configured to the physical storage volume group; Again by a plurality of physical volume group construction logic storage volume groups; A plurality of logical volume groups form the partition holding unit of logical partition as global file system, and this processing with tangible physics hardware logic volumeization helps the equiblibrium mass distribution and the flexible online expansion of later stage of data;
2) deployment of described global file system; Comprise logical partition is formatd; File system is at the carry of client; User's quotas administered, daily record configuration management and adopt dynamic lock management mechanism to coordinate of the read-write of a plurality of nodes, the inconsistent and wrong initialized situation of the data that the system accident of the avoiding machine of delaying brings to identical file;
3) described metadata daily record system and the malfunctioning node isolation mech isolation test that is distributed in each client is included in client and is global file system and dynamically add daily record data protection mechanism is provided; The real-time isolation mech isolation test of malfunctioning node realizes the fault recovery function, and behind Single Point of Faliure, in the consistency detection and the recovery of all the other working nodes completion data, the fault machine recovers the initialization correctly of back data like this.
2. method according to claim 1, but it is characterized in that utilizing the logical volume management instrument physical store raw device to be configured to the logical storage volumes group of resilient expansion; Dispose global file system in logical volume group aspect, set up, will file system mountedly arrive this locality in local nodes towards the cluster storage system of the unified NameSpace of node cluster; Distributed log management system and malfunctioning node isolation mech isolation test provide the data protection of file system and fault recovery function.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012100887961A CN102664923A (en) | 2012-03-30 | 2012-03-30 | Method for realizing shared storage pool by utilizing Linux global file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012100887961A CN102664923A (en) | 2012-03-30 | 2012-03-30 | Method for realizing shared storage pool by utilizing Linux global file system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102664923A true CN102664923A (en) | 2012-09-12 |
Family
ID=46774332
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2012100887961A Pending CN102664923A (en) | 2012-03-30 | 2012-03-30 | Method for realizing shared storage pool by utilizing Linux global file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102664923A (en) |
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103051681A (en) * | 2012-12-06 | 2013-04-17 | 华中科技大学 | Collaborative type log system facing to distribution-type file system |
| CN103229487A (en) * | 2012-12-27 | 2013-07-31 | 华为技术有限公司 | Partition balance method, device and server in distributed storage system |
| CN103797770A (en) * | 2012-12-31 | 2014-05-14 | 华为技术有限公司 | Method and system for sharing storage resources |
| CN104883380A (en) * | 2014-05-12 | 2015-09-02 | 陈杰 | Method and device for storage volume configuration |
| CN106201357A (en) * | 2016-07-18 | 2016-12-07 | 浪潮(北京)电子信息产业有限公司 | The construction method of a kind of storage pool and system |
| WO2016197867A1 (en) * | 2015-06-11 | 2016-12-15 | 阿里巴巴集团控股有限公司 | Data processing method and device applicable to computing device |
| CN106777394A (en) * | 2017-02-28 | 2017-05-31 | 郑州云海信息技术有限公司 | A kind of cluster file system |
| CN106919477A (en) * | 2017-03-03 | 2017-07-04 | 上海爱数信息技术股份有限公司 | The analytic method and system of a kind of virtual disk |
| US9733848B2 (en) | 2012-12-31 | 2017-08-15 | Huawei Technologies Co., Ltd. | Method and system for pooling, partitioning, and sharing network storage resources |
| CN108848136A (en) * | 2018-05-30 | 2018-11-20 | 安徽四创电子股份有限公司 | A kind of shared storage method of cloud service cluster |
| CN108984299A (en) * | 2018-06-29 | 2018-12-11 | 郑州云海信息技术有限公司 | A kind of optimization method of distributed type assemblies, device, system and readable storage medium storing program for executing |
| CN109032521A (en) * | 2018-07-25 | 2018-12-18 | 星环信息科技（上海）有限公司 | Storage volume creation method, device, server and storage medium |
| CN109271457A (en) * | 2018-11-19 | 2019-01-25 | 郑州云海信息技术有限公司 | A kind of method of data synchronization and device |
| CN109284270A (en) * | 2018-11-14 | 2019-01-29 | 郑州云海信息技术有限公司 | A kind of disposition optimization method and device of distributed file system memory module |
| CN109460285A (en) * | 2018-11-01 | 2019-03-12 | 郑州云海信息技术有限公司 | The hanging method and relevant apparatus of storage system in a kind of virtual platform |
| CN110515778A (en) * | 2019-08-30 | 2019-11-29 | 星辰天合（北京）数据科技有限公司 | Method, device and system for data protection based on shared logical volume |
| WO2019228217A1 (en) * | 2018-06-01 | 2019-12-05 | 阿里巴巴集团控股有限公司 | File system data access method and file system |
| CN111212141A (en) * | 2020-01-02 | 2020-05-29 | 中国科学院计算技术研究所 | Shared storage system |
| CN111857577A (en) * | 2020-06-29 | 2020-10-30 | 烽火通信科技股份有限公司 | Method and device for managing physical hard disk in distributed storage system |
| CN112148229A (en) * | 2020-10-12 | 2020-12-29 | 北京计算机技术及应用研究所 | Method for accelerating nonvolatile memory read-write efficiency under Feiteng server platform |
| CN112416862A (en) * | 2020-11-30 | 2021-02-26 | 中国航空工业集团公司西安航空计算技术研究所 | File management system and method based on partition operating system architecture |
| CN112463306A (en) * | 2020-12-03 | 2021-03-09 | 南京机敏软件科技有限公司 | Method for sharing disk data consistency in virtual machine |
| US10972542B2 (en) | 2017-03-29 | 2021-04-06 | Huawei Technologies Co., Ltd. | Data storage method and apparatus |
| CN113177025A (en) * | 2021-03-25 | 2021-07-27 | 北京城建智控科技有限公司 | Data operation method and system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101841526A (en) * | 2010-03-04 | 2010-09-22 | 清华大学 | Cluster streaming media server system applied to large-scale user demand |
| CN102164177A (en) * | 2011-03-11 | 2011-08-24 | 浪潮(北京)电子信息产业有限公司 | Method, device and system for sharing storage pool by cluster |
-
2012
- 2012-03-30 CN CN2012100887961A patent/CN102664923A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101841526A (en) * | 2010-03-04 | 2010-09-22 | 清华大学 | Cluster streaming media server system applied to large-scale user demand |
| CN102164177A (en) * | 2011-03-11 | 2011-08-24 | 浪潮(北京)电子信息产业有限公司 | Method, device and system for sharing storage pool by cluster |
Cited By (36)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103051681A (en) * | 2012-12-06 | 2013-04-17 | 华中科技大学 | Collaborative type log system facing to distribution-type file system |
| CN103051681B (en) * | 2012-12-06 | 2015-06-17 | 华中科技大学 | Collaborative type log system facing to distribution-type file system |
| CN103229487B (en) * | 2012-12-27 | 2015-09-09 | 华为技术有限公司 | Partition balancing method, device and server in distributed memory system |
| CN103229487A (en) * | 2012-12-27 | 2013-07-31 | 华为技术有限公司 | Partition balance method, device and server in distributed storage system |
| CN103797770B (en) * | 2012-12-31 | 2015-12-02 | 华为技术有限公司 | A kind of method and system of shared storage resources |
| CN103797770A (en) * | 2012-12-31 | 2014-05-14 | 华为技术有限公司 | Method and system for sharing storage resources |
| US11042311B2 (en) | 2012-12-31 | 2021-06-22 | Huawei Technologies Co., Ltd. | Cluster system with calculation and storage converged |
| US9733848B2 (en) | 2012-12-31 | 2017-08-15 | Huawei Technologies Co., Ltd. | Method and system for pooling, partitioning, and sharing network storage resources |
| US10082972B2 (en) | 2012-12-31 | 2018-09-25 | Huawei Technologies Co., Ltd. | Method and system for pooling, partitioning, and sharing network storage resources |
| US10481804B2 (en) | 2012-12-31 | 2019-11-19 | Huawei Technologies Co., Ltd. | Cluster system with calculation and storage converged |
| CN104883380A (en) * | 2014-05-12 | 2015-09-02 | 陈杰 | Method and device for storage volume configuration |
| CN104883380B (en) * | 2014-05-12 | 2018-06-05 | 陈杰 | The method and device of storage volume configuration |
| WO2016197867A1 (en) * | 2015-06-11 | 2016-12-15 | 阿里巴巴集团控股有限公司 | Data processing method and device applicable to computing device |
| CN106201357A (en) * | 2016-07-18 | 2016-12-07 | 浪潮(北京)电子信息产业有限公司 | The construction method of a kind of storage pool and system |
| CN106777394A (en) * | 2017-02-28 | 2017-05-31 | 郑州云海信息技术有限公司 | A kind of cluster file system |
| CN106919477A (en) * | 2017-03-03 | 2017-07-04 | 上海爱数信息技术股份有限公司 | The analytic method and system of a kind of virtual disk |
| CN106919477B (en) * | 2017-03-03 | 2020-05-22 | 上海爱数信息技术股份有限公司 | Virtual disk analysis method and system |
| US11575748B2 (en) | 2017-03-29 | 2023-02-07 | Huawei Technologies Co., Ltd. | Data storage method and apparatus for combining different data distribution policies |
| US10972542B2 (en) | 2017-03-29 | 2021-04-06 | Huawei Technologies Co., Ltd. | Data storage method and apparatus |
| CN108848136A (en) * | 2018-05-30 | 2018-11-20 | 安徽四创电子股份有限公司 | A kind of shared storage method of cloud service cluster |
| CN108848136B (en) * | 2018-05-30 | 2021-08-10 | 安徽四创电子股份有限公司 | Shared storage method of cloud service cluster |
| WO2019228217A1 (en) * | 2018-06-01 | 2019-12-05 | 阿里巴巴集团控股有限公司 | File system data access method and file system |
| CN108984299A (en) * | 2018-06-29 | 2018-12-11 | 郑州云海信息技术有限公司 | A kind of optimization method of distributed type assemblies, device, system and readable storage medium storing program for executing |
| CN109032521A (en) * | 2018-07-25 | 2018-12-18 | 星环信息科技（上海）有限公司 | Storage volume creation method, device, server and storage medium |
| CN109460285A (en) * | 2018-11-01 | 2019-03-12 | 郑州云海信息技术有限公司 | The hanging method and relevant apparatus of storage system in a kind of virtual platform |
| CN109284270A (en) * | 2018-11-14 | 2019-01-29 | 郑州云海信息技术有限公司 | A kind of disposition optimization method and device of distributed file system memory module |
| CN109284270B (en) * | 2018-11-14 | 2021-09-17 | 郑州云海信息技术有限公司 | Deployment optimization method and device for distributed file system storage module |
| CN109271457A (en) * | 2018-11-19 | 2019-01-25 | 郑州云海信息技术有限公司 | A kind of method of data synchronization and device |
| CN110515778A (en) * | 2019-08-30 | 2019-11-29 | 星辰天合（北京）数据科技有限公司 | Method, device and system for data protection based on shared logical volume |
| CN111212141A (en) * | 2020-01-02 | 2020-05-29 | 中国科学院计算技术研究所 | Shared storage system |
| CN111857577A (en) * | 2020-06-29 | 2020-10-30 | 烽火通信科技股份有限公司 | Method and device for managing physical hard disk in distributed storage system |
| CN112148229A (en) * | 2020-10-12 | 2020-12-29 | 北京计算机技术及应用研究所 | Method for accelerating nonvolatile memory read-write efficiency under Feiteng server platform |
| CN112148229B (en) * | 2020-10-12 | 2024-01-30 | 北京计算机技术及应用研究所 | Method for accelerating read-write efficiency of nonvolatile memory under Feiteng server platform |
| CN112416862A (en) * | 2020-11-30 | 2021-02-26 | 中国航空工业集团公司西安航空计算技术研究所 | File management system and method based on partition operating system architecture |
| CN112463306A (en) * | 2020-12-03 | 2021-03-09 | 南京机敏软件科技有限公司 | Method for sharing disk data consistency in virtual machine |
| CN113177025A (en) * | 2021-03-25 | 2021-07-27 | 北京城建智控科技有限公司 | Data operation method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102664923A (en) | Method for realizing shared storage pool by utilizing Linux global file system | |
| US11636015B2 (en) | Storage system and control software deployment method | |
| CN102164177A (en) | Method, device and system for sharing storage pool by cluster | |
| CN102255962B (en) | Distributive storage method, device and system | |
| EP3627358B1 (en) | Method, system, and apparatus for cloud application redundancy | |
| Wang et al. | Burstmem: A high-performance burst buffer system for scientific applications | |
| CN101952814B (en) | Method and system for implementing virtual storage pool in virtual environment | |
| CN107734026B (en) | Method, device and equipment for designing network additional storage cluster | |
| CN103379159B (en) | A kind of method that distributed Web station data synchronizes | |
| CN103098015A (en) | Storage system | |
| CN102413172B (en) | Parallel data sharing method based on cluster technology and apparatus thereof | |
| CN105740248B (en) | A kind of method of data synchronization, apparatus and system | |
| CN105518641A (en) | Method, device, and system for peer-to-peer data replication and method, device, and system for master node switching | |
| WO2012101540A1 (en) | Application recovery in file system | |
| CN106686099A (en) | Method of realizing active-active mode across machine rooms of OracleRAC database based on infiniband network | |
| CN102938778A (en) | Method for realizing multi-node disaster tolerance in cloud storage | |
| CN104202359A (en) | NVMe SSD virtualization design method based on blade server | |
| US8990815B1 (en) | Synchronizing allocated blocks of virtual disk files across primary and secondary volumes by excluding unused blocks | |
| CN102820998A (en) | Dual-fault-tolerant service system applicable to office applications and data storage method of dual-fault-tolerant service system | |
| CN103209219A (en) | Distributed cluster file system | |
| CN102541473A (en) | Method for setting storage equipment of blade server and blade server by utilizing method | |
| CN202856789U (en) | Data resource storage system integrating NAS and SAN | |
| CN203054824U (en) | Server storage system | |
| CN109831502A (en) | Novel website cloud fictitious host computer framework | |
| CN206948385U (en) | A kind of big data intelligent analysis arithmetic processing system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20120912 |