CN105471989B - A kind of date storage method - Google Patents
A kind of date storage method Download PDFInfo
- Publication number
- CN105471989B CN105471989B CN201510817619.6A CN201510817619A CN105471989B CN 105471989 B CN105471989 B CN 105471989B CN 201510817619 A CN201510817619 A CN 201510817619A CN 105471989 B CN105471989 B CN 105471989B
- Authority
- CN
- China
- Prior art keywords
- storage
- nfs
- data
- configuration file
- ganesha
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
- G06F16/1824—Distributed file systems implemented using Network-attached Storage [NAS] architecture
- G06F16/183—Provision of network file services by network file servers, e.g. by using NFS, CIFS
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention provides a kind of date storage method, is related to a kind of data-storage system, data-storage system includes client, NFS-GANESHA and Ceph RBD；Date storage method includes the following steps：S1, NFS-GANESHA server-side write configuration file；S2, NFS-GANESHA parse configuration file, and the storage in carry configuration file；S3, judge configuration file parsing and storage carry whether simultaneously successfully；If it is not, then exiting process and prompting mistake；If so, entering step S4；S4, carry Ceph RBD, and data read-write operation is executed to Ceph RBD.Storage and customer side are connected, Ceph RBD provide high-performance storage, and NFS-GANESHA provides high performance file-sharing function, to realize more efficient distributed storage.Ceph RBD stablize the most as storage, and snapshot management may be implemented in RBD, are accidentally deleted if any file or other problems, so that it may to restore snapshot manually；And a point domain may be implemented with Ceph RBD storage VMware virtual machines to preserve, can according to empty machine it is important whether, realize that different copies preserve.
Description
Technical field
The present invention relates to field of distributed storage, more particularly to a kind of date storage method.
Background technology
With the arriving in big data epoch, more and more distributed systems are come into being.And distributed storage is distribution
The final of formula system wants link.Big data, cloud computing final data are all to preserve, and the best approach of preservation is exactly to be distributed
Formula storage system, such as present Ceph, gluster etc. is the distributed memory system of mainstream.And give multiple use storage carry
It is exactly a problem to be solved that family, which uses,.The problem of so this relates to how to deposit, how to take.Distributed storage can be with
Data are preserved well, and present distributed memory system is all to carry more copies.Storage is set to be guaranteed safely in this way.
Ceph file system has high scalability, high reliability and high-performance.RBD is supports of the Ceph to block device, it
Support snapshot, the operations such as clone.
The development of internet makes the main body that data are created be increasingly turned to personal user by enterprise, and caused by individual absolutely
Most of data are the unstructured datas such as picture, document, video；Enterprise's office workflow more by real-time performance, list,
Bill etc. all realizes the archive of the digitlization based on unstructured；Meanwhile the structural data based on database application is still
It is play an important role in enterprise, stores a large amount of core information.
Data service sharply increases, and the single SAN storage modes of tradition have been not suitable with business development needs.SAN is deposited
Storage：It is of high cost, be not suitable for PB grades of large-scale storage systems.Data sharing is bad, can not support multi-user's file-sharing.
The storage of data can be protected, and the readings of data has that is to be done, if facing multi-user, how to accomplish to count
According to High Availabitity.Multi-user access same data, at this moment network just become bottleneck.The shared of file is exactly extremely important
's.
NFS file system is presently the most successful Network File System, there is outstanding performance in file-sharing field,
The especially appearance of NFS-GANESHA so that NFS file system has in performance and scale to be substantially improved, and has been its band more
For wide application space.Meanwhile why NFS is concerned, and also resides in its key effect in NAS field of storage.
Invention content
In view of the foregoing deficiencies of prior art, it the purpose of the present invention is to provide a kind of date storage method, is used for
It is low to solve distributed storage efficiency in the prior art, the bad problem of the shared performance of data and file.
In order to achieve the above objects and other related objects, the present invention provides a kind of date storage method, is related to a kind of data
Storage system, the data-storage system include client, the first server for supporting NFS-GANESHA and for supporting
The second server of Ceph RBD；The date storage method includes the following steps：S1, in the NFS- of the first server
GANESHA server-sides write configuration file；S2, the NFS-GANESHA parse the configuration file, and text is configured described in carry
Storage in part；S3, judge in the configuration file storage whether the carry of legal and described storage whether simultaneously successfully；If
It is no, then it exits process and prompts mistake；If so, entering step S4；Ceph RBD described in S4, carry, and to the Ceph
RBD executes data read-write operation.
In one embodiment of the invention, the step S1 further includes step：S11, the configuration configuration file format are wanted
It asks；The Ceph pool of carry are needed in S12, the configuration configuration file.
In one embodiment of the invention, the step S2 further includes step：S21, load described program；S22, by pre-
If instructions query stores information.
In one embodiment of the invention, the preset instructions are showmount-e localhost orders.
In one embodiment of the invention, the S4 further includes step：S41, operation mount-t nfs source directory carry mesh
It records, Ceph RBD described in carry；Authority configuration in S42, the reading configuration file；S43, data are carried out according to the permission
Read-write operation.
In one embodiment of the invention, further include in the step S43：S431, general letter is written in described program
Number, S432, the NFS-GANESHA call the general purpose function；S433, the read-write behaviour for carrying out data is limited according to the permission
Make.
Further include step after the step S4 in one embodiment of the invention：S5, the Ceph RBD are carried out soon
According to processing.
Further include step after the step S5 in one embodiment of the invention：S6, the detection data-storage system
Whether break down.
In one embodiment of the invention, the step S4 further includes：After the Ceph RBD execute data read-write operation,
It is readable in all clients in the file that the client for having write permission generates.
As described above, the date storage method of the present invention, has the advantages that：Storage has been connected with customer side
Come, Ceph RBD provide high-performance storage, and NFS-GANESHA provides high performance file-sharing function, more efficient to realize
Distributed storage.Ceph RBD stablize the most as storage, and snapshot management may be implemented in RBD, can according to user demand
To add snapshot and restore snapshot；And a point domain may be implemented with Ceph RBD storage VMware virtual machines to preserve, it can basis
Empty machine it is important whether, realize that different copies preserve.
Description of the drawings
Fig. 1 is shown as the module block diagram in an embodiment of data-storage system of the present invention.
Fig. 2 is shown as the process blocks schematic diagram in an embodiment of date storage method of the present invention.
Component label instructions
10 clients
20 first servers
30 second servers
Specific implementation mode
Illustrate that embodiments of the present invention, those skilled in the art can be by this specification below by way of specific specific example
Disclosed content understands other advantages and effect of the present invention easily.The present invention can also pass through in addition different specific realities
The mode of applying is embodied or practiced, the various details in this specification can also be based on different viewpoints with application, without departing from
Various modifications or alterations are carried out under the spirit of the present invention.It should be noted that in the absence of conflict, following embodiment and implementation
Feature in example can be combined with each other.
It should be noted that the diagram provided in following embodiment only illustrates the basic structure of the present invention in a schematic way
Think, component count, shape and size when only display is with related component in the present invention rather than according to actual implementation in schema then
Draw, when actual implementation kenel, quantity and the ratio of each component can be a kind of random change, and its assembly layout kenel
It is likely more complexity.
It please refers to Fig.1 with shown in 2, the module box that Fig. 1 is shown as in an embodiment of data-storage system of the present invention shows
It is intended to.Fig. 2 is shown as the process blocks schematic diagram in an embodiment of date storage method of the present invention.The present invention provides one kind
Date storage method, as shown in Fig. 2, and be related to a kind of data-storage system as shown in Figure 1, the data-storage system includes
Client, the first server 20 for supporting NFS-GANESHA and the second server 30 for supporting Ceph RBD；Mainly
It is to be supplied to NFS-GANESHA to call the librbd interfaces in Ceph.NFS-GANESHA is got well than kernel state NFS performances, institute
It is exactly one FSAL (File System Abstraction Layer) of addition in this project with the present invention, makes NFS-
GANESHA supports Ceph RBD.NFS-GANESHA is managed collectively all rear end storages there are one level of abstraction, and NFS can be simultaneously
Support that a variety of storages, the present invention are to add a kind of method of standby storage to NFS-GANESHA.The date storage method includes
Following steps：S1, respective profiles are write in the NFS-GANESHA server-sides of the first server 10；The present invention's
In one preferred embodiment, the step S1 further includes step：S11, the configuration configuration file format requirement；Described in S12, configuration
The Ceph pool of carry are needed in configuration file.The code for compiling invention, if NFS-GANESHA server ends first
Ceph clustered nodes, and program is dynamically loaded into NFS-GANESHA by program execution module.
S2, the NFS-GANESHA parse the configuration file, and the storage in configuration file described in carry；In this hair
In a bright preferred embodiment, shown step S2 further includes：S21, load described program；S22, storage is inquired by preset instructions
Information inquires all storages of NFS-GANESHA by preset instructions.Wherein, the preset instructions are showmount-e
Localhost orders.The configuration file for writing the NFS-GANESHA for supporting Ceph RBD, matching for the Ceph RBD to be stored
It sets in file, configuration file can be parsed automatically when NFS-GANESHA is run, and the storage carry in configuration file is gone out
Come.After module loading success, so that it may to allow the customer side of NFS that this storage is mounted to corresponding catalogue by mount orders
Under.Then the storage for needing carry is checked with showmount-e localhost orders.
S3, whether simultaneously successfully to judge the carry of configuration file parsing and the storage；If it is not, then exiting process simultaneously
Prompt mistake；If so, entering step S4；
Ceph RBD described in S4, carry, and data read-write operation is executed to the Ceph RBD.One in the present invention is preferred
In embodiment, the S4 further includes step：S41, operation mount-t nfs source directory mount directory, Ceph RBD described in carry；
Authority configuration in S42, the reading configuration file；S43, the read-write operation that data are carried out according to the permission.Further,
Further include in the step S43：S431, general purpose function is written in described program, S432, the NFS-GANESHA call institute
State general purpose function；S433, the read-write operation for carrying out data is limited according to the permission.After carry success, so that it may with any one
Operation mount-t nfs source directory mount directory on a customer sides NFS, this RBD of carry, and can be right in configuration file
User carries out permission limitation.
In addition, failure or other problems, date storage method of the present invention further include step in order to prevent：It is S5, right
The Ceph RBD carry out snapshot processing.Further include step S6 after S5, detect whether the data-storage system breaks down.
Above-mentioned steps all after the completion of, usually further include in the step S4：The Ceph RBD execute reading and writing data
It is readable in all clients in the file that the client for having write permission generates after operation.I.e. as there is not any problem, table
Show the carry that succeeded, in the file that any one has the generation of write permission client, other customer sides are all visible.
In conclusion the present invention will store and customer side connects, Ceph RBD provide high-performance storage, NFS-
GANESHA provides high performance file-sharing function, to realize more efficient distributed storage.Ceph RBD are used as storage most
For stabilization, and snapshot management may be implemented in RBD；And a point domain may be implemented with Ceph RBD storage VMware virtual machines to protect
Deposit, can according to empty machine it is important whether, realize that different copies preserve.So the present invention effectively overcomes in the prior art kind
It plants disadvantage and has high industrial utilization.
The above-described embodiments merely illustrate the principles and effects of the present invention, and is not intended to limit the present invention.It is any ripe
The personage for knowing this technology can all carry out modifications and changes to above-described embodiment without violating the spirit and scope of the present invention.Cause
This, institute is complete without departing from the spirit and technical ideas disclosed in the present invention by those of ordinary skill in the art such as
At all equivalent modifications or change, should by the present invention claim be covered.
Claims (9)
1. a kind of date storage method, which is characterized in that be related to a kind of data-storage system, the data-storage system includes visitor
Family end, the first server for supporting NFS-GANESHA and the second server for supporting Ceph RBD；The data are deposited
Method for storing includes the following steps：
S1, configuration file is write in the NFS-GANESHA server-sides of the first server；
S2, the configuration file, and the storage in configuration file described in carry are parsed by the NFS-GANESHA；
S3, whether simultaneously successfully to judge the carry of configuration file parsing and the storage；If it is not, then exiting process and prompting
Mistake；
If so, entering step S4；
Ceph RBD described in S4, carry, and data read-write operation is executed to the Ceph RBD.
2. date storage method according to claim 1, which is characterized in that the step S1 further includes step：
S11, the configuration configuration file format requirement；
The Ceph pool of carry are needed in S12, the configuration configuration file.
3. date storage method according to claim 2, which is characterized in that the step S2 further includes step：
S21, loading procedure；
S22, storage information is inquired by preset instructions.
4. date storage method according to claim 3, which is characterized in that the preset instructions are showmount-
Elocalhost orders.
5. date storage method according to claim 1, which is characterized in that the S4 further includes step：
S41, operation mount-t nfs source directory mount directory, Ceph RBD described in carry；
Authority configuration in S42, the reading configuration file；
S43, the read-write operation that data are carried out according to the permission.
6. date storage method according to claim 5, which is characterized in that further include in the step S43：
S431, general purpose function is written in a program；
S432, the NFS-GANESHA call the general purpose function；
S433, the read-write operation that data are carried out according to the limitation of the permission.
7. date storage method according to claim 1, which is characterized in that further include step after the step S4：
S5, snapshot processing is carried out to the Ceph RBD.
8. date storage method according to claim 7, which is characterized in that further include step after the step S5：
Whether S6, the detection data-storage system break down.
9. according to claim 1-8 any one of them date storage methods, which is characterized in that the step S4 further includes：Institute
State after Ceph RBD execute data read-write operation, the file that the client for having write permission generates all clients can
It reads.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510817619.6A CN105471989B (en) | 2015-11-23 | 2015-11-23 | A kind of date storage method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510817619.6A CN105471989B (en) | 2015-11-23 | 2015-11-23 | A kind of date storage method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105471989A CN105471989A (en) | 2016-04-06 |
| CN105471989B true CN105471989B (en) | 2018-11-02 |
Family
ID=55609252
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510817619.6A Expired - Fee Related CN105471989B (en) | 2015-11-23 | 2015-11-23 | A kind of date storage method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105471989B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106445563A (en) * | 2016-11-24 | 2017-02-22 | 合肥中科云巢科技有限公司 | Operation system mirror image manufacturing method and device |
| CN107179929B (en) * | 2017-05-15 | 2020-02-07 | 苏州浪潮智能科技有限公司 | WRITE _ SAME function optimization implementation method and device |
| CN107197022B (en) * | 2017-06-02 | 2020-01-10 | 华南理工大学 | OpenStack storage optimization method and system |
| CN107609133A (en) * | 2017-09-18 | 2018-01-19 | 郑州云海信息技术有限公司 | Journal file dump method, device, equipment and its computer-readable recording medium |
| CN108132815A (en) * | 2017-12-25 | 2018-06-08 | 郑州云海信息技术有限公司 | A kind of method, apparatus and storage medium of configuration file dynamic load |
| CN108646985A (en) * | 2018-05-16 | 2018-10-12 | 广东睿江云计算股份有限公司 | A kind of resource constraint and distribution method of Ceph distributed memory systems |
| CN112511578B (en) * | 2019-09-16 | 2022-06-24 | 大唐移动通信设备有限公司 | Data storage method and device |
| CN111125050B (en) * | 2019-12-26 | 2023-08-22 | 浪潮云信息技术股份公司 | File storage method based on CephFS to provide NFS protocol in openstack environment |
| CN111159105B (en) * | 2019-12-30 | 2023-11-28 | 浪潮云信息技术股份公司 | System and method for supporting NFS protocol and setting ACL based on mangia sharing mode |
| CN111787113B (en) * | 2020-07-03 | 2021-09-03 | 北京大道云行科技有限公司 | Node fault processing method and device, storage medium and electronic equipment |
| CN112579549A (en) * | 2020-12-14 | 2021-03-30 | 浪潮云信息技术股份公司 | CephFS file protocol sharing system, construction method and implementation method |
| CN112733189A (en) * | 2021-01-14 | 2021-04-30 | 浪潮云信息技术股份公司 | System and method for realizing file storage server side encryption |
| CN113342282A (en) * | 2021-06-28 | 2021-09-03 | 深圳市瑞驰信息技术有限公司 | Method for migrating data backup of android system to rbd storage |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103561101A (en) * | 2013-11-06 | 2014-02-05 | 中国联合网络通信集团有限公司 | Network file system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI676898B (en) * | 2013-12-09 | 2019-11-11 | 安然國際科技有限公司 | Decentralized memory disk cluster storage system operation method |
-
2015
- 2015-11-23 CN CN201510817619.6A patent/CN105471989B/en not_active Expired - Fee Related
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103561101A (en) * | 2013-11-06 | 2014-02-05 | 中国联合网络通信集团有限公司 | Network file system |
Non-Patent Citations (1)
| Title |
|---|
| "面向大数据分析的分布式文件系统关键技术";周江等;《计算机研究与发展》;20141231;382-394页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105471989A (en) | 2016-04-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105471989B (en) | A kind of date storage method | |
| US11455217B2 (en) | Transaction consistency query support for replicated data from recovery log to external data stores | |
| US8555018B1 (en) | Techniques for storing data | |
| JP6188732B2 (en) | Computer-implemented method, computer program product, and system for managing tenant-specific data sets in a multi-tenant environment | |
| US9779127B2 (en) | Integrating database management system and external cache | |
| US11249983B2 (en) | Transaction change data forwarding | |
| US9772911B2 (en) | Pooling work across multiple transactions for reducing contention in operational analytics systems | |
| US9990225B2 (en) | Relaxing transaction serializability with statement-based data replication | |
| US20140040294A1 (en) | Manipulation of multi-tenancy database | |
| EP2556444B1 (en) | System and method for sharing data between occasionally connected devices and remote global database | |
| US20120054182A1 (en) | Systems and methods for massive structured data management over cloud aware distributed file system | |
| US20140324917A1 (en) | Reclamation of empty pages in database tables | |
| CN102779185A (en) | High-availability distribution type full-text index method | |
| Tan et al. | Diff-Index: Differentiated Index in Distributed Log-Structured Data Stores. | |
| US8615488B2 (en) | Physical replication of database subset to improve availability and reduce resource cost in a cloud environment | |
| US20180150540A1 (en) | Synchronization Mechanism for Serialized Data Log Replay in Database Systems | |
| US10698921B2 (en) | Persistence and initialization of synchronization state for serialized data log replay in database systems | |
| CN110427364A (en) | A kind of data processing method, device, electronic equipment and storage medium | |
| Chandra et al. | A study on cloud database | |
| US10769034B2 (en) | Caching DML statement context during asynchronous database system replication | |
| US8639657B2 (en) | Reorganizing table-based data objects | |
| WO2015094195A1 (en) | Transaction query engine | |
| US10127270B1 (en) | Transaction processing using a key-value store | |
| US9229659B2 (en) | Identifying and accessing reference data in an in-memory data grid | |
| US20210141759A1 (en) | Synchronous object placement for information lifecycle management |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20181102 Termination date: 20191123 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |