CN109413130A - A kind of cloud storage system - Google Patents
A kind of cloud storage system Download PDFInfo
- Publication number
- CN109413130A CN109413130A CN201711500129.9A CN201711500129A CN109413130A CN 109413130 A CN109413130 A CN 109413130A CN 201711500129 A CN201711500129 A CN 201711500129A CN 109413130 A CN109413130 A CN 109413130A
- Authority
- CN
- China
- Prior art keywords
- data
- server
- storage
- file
- storage system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
- H04L67/61—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources taking into account QoS or priority requirements
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Storage Device Security (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of cloud storage systems, including meta data server, data storage server and client.Wherein, meta data server, data storage server, client pass through TCP/IP network protocol respectively and connect one to one；Meta data server is constructed using the framework of centralization, the metadata information for All Files in storage and management cloud storage system；Data storage server is constructed using the framework of concentrating type, is used for storage file data block, the read-write service to clients providing data；Client for user for inputting operation instruction, to realize the write-in, reading and deletion of file.The control stream and data stream separation of file are supported the load balancing and data redundancy strategy of storage resource by technical solution of the present invention, improve system performance.
Description
Technical field
The present invention relates to field of computer technology more particularly to a kind of cloud storage systems.
Background technique
Cloud storage be in the conceptive extension of cloud computing (cloud computing) and a new concept developed,
It is a kind of emerging Network storage technology, refers to through functions such as cluster application, network technology or distributed file systems, by net
A large amount of various types of storage equipment gather collaborative work by application software in network, and the common data that externally provide are deposited
The system of storage and business access function.
Traditional cloud storage system utilizes close coupling symmetrical architecture, and the design of this framework aims to solve the problem that HPC (high-performance meter
Calculate, super operation) problem, it is extending to the outside as cloud storage to meet the market demand quickly presented.But it is traditional
System does not completely classify " control stream " and " data flow ", is easy to produce data redundancy, is unfavorable for load balancing.
Summary of the invention
The embodiment of the present invention proposes a kind of cloud storage system, by the control stream and data stream separation of file, supports storage money
The load balancing and data redundancy strategy in source improve system performance.
The embodiment of the present invention provides a kind of cloud storage system, comprising: meta data server, data storage server and client
End；
Wherein, the meta data server, data storage server, client pass through TCP/IP network protocol one by one respectively
It is correspondingly connected with；
The meta data server is constructed using the framework of centralization, for cloud storage system described in storage and management
The metadata information of interior All Files；The metadata information is by: file global identifier, filename, file size, storage position
Set mapping, access authority, creation and modification time and copy information composition；
The data storage server is constructed using the framework of concentrating type, is used for storage file data block, Xiang Suoshu
The read-write service of clients providing data；
The client for user for inputting operation instruction, to realize the write-in, reading and deletion of file.
Further, the meta data server is also used to resolve in the virtualization path of file metadata, and by institute
Metadata is stated to be mapped on the data storage in corresponding data block.
Further, the meta data server is provided with authority management module；
The authority management module, which is used to access user, carries out Authority Verification and matching.
Further, the meta data server is provided with QOS control module, load balancing module, node administration module
With replica management module；
The meta data server passes through QOS control, load balancing control, node administration and replica management, monitoring and pipe
Manage the global resource service condition of the cloud storage system.
Further, the data storage server is also used to replica management, concurrently access management and reading and writing data
QOS control.
Further, the data storage server includes several sub- storage servers；
Each sub- storage server is isomerism server, and has unified memory interface.
The implementation of the embodiments of the present invention has the following beneficial effects:
Cloud storage system provided in an embodiment of the present invention, including meta data server, data storage server and client.
Wherein, meta data server, data storage server, client pass through TCP/IP network protocol respectively and connect one to one；Member
Data server is constructed using the framework of centralization, the metadata for All Files in storage and management cloud storage system
Information；Data storage server is constructed using the framework of concentrating type, is used for storage file data block, provides number to client
According to read-write service；Client for user for inputting operation instruction, to realize the write-in, reading and deletion of file.Compared to
The cloud storage system of prior art control stream and data flow mixing, technical solution of the present invention is by the control stream and data flow point of file
From supporting the load balancing and data redundancy strategy of storage resource, improve system performance.
Detailed description of the invention
Fig. 1 is a kind of structural schematic diagram of embodiment of cloud storage system provided by the invention；
Fig. 2 is the structural schematic diagram of another embodiment of cloud storage system provided by the invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
It is a kind of structural schematic diagram of embodiment of cloud storage system provided by the invention, the cloud stocking system referring to Fig. 1
It include: meta data server 1, data storage server 2 and client 3；Wherein, meta data server 1, data storage server
2, client 3 is connected one to one by TCP/IP network protocol respectively.
Meta data server 1 is constructed using the framework of centralization, for owning in storage and management cloud storage system
The metadata information of file.Metadata information by: file global identifier, filename, file size, storage location mapping, visit
Ask permission, creation and modification time and copy information composition.
Data storage server 2 is constructed using the framework of concentrating type, storage file data block is used for, to client 3
The read-write service of data is provided.
Client 3 for user for inputting operation instruction, to realize the write-in, reading and deletion of file.
In the present embodiment, access entrance of the meta data server 1 as cloud storage system.Data storage server 2
It is responsible for the specific file data blocks of storage, is not related to the control information of file.Client 3 be cloud storage system using platform and
Entrance provides data storage and service access interface for user.
In the present embodiment, meta data server 1 is also used to the virtualization path of file resolving to metadata, and will be first
Data are mapped on data storage 2 in corresponding data block.
In the present embodiment, meta data server 1 is provided with authority management module.Authority management module is used to visit user
It asks and carries out Authority Verification and matching.
In the present embodiment, data server 1 be provided with QOS control module, load balancing module, node administration module and
Replica management module.Meta data server passes through QOS control, load balancing control, node administration and replica management, monitoring and pipe
Manage the global resource service condition of cloud storage system.
In the present embodiment, data storage server 2 is also used to replica management, concurrently access management and reading and writing data
QOS control.
In the present embodiment, data storage server 2 includes several sub- storage servers.Each sub- storage server is different
Structure sex service device, and there is unified memory interface.Isomerism is that different types of server may have different bottom texts
Part system, the respective performance of server also difference.
Cloud storage system in order to better illustrate the present invention, referring to fig. 2, Fig. 2 are cloud storage systems provided by the invention
Another embodiment structural schematic diagram.As shown in Fig. 2, entire cloud storage system is set up using master-slave mode.Metadata Service
Device includes remote call service and communication interface, authority management module, QOS control module, load balancing module, node administration mould
Block, replica management module, NameSpace layer, cache layer and data persistence layer.Data storage server include: unified memory interface,
Information monitoring module, QOS control module, memory management module, replica management module, block storage equipment and object storage device.
The file format that data storage server is stored is EXT3, FAT32, NTFS etc..As it can be seen that cloud storage system provided in this embodiment
The control stream and data stream separation of file will be distributed over the discrete resource in multiple data storages and are integrated into a system by system
One virtual storage resource pond.The cloud storage system framework has elasticity well, can be expanded by hot plug mode dynamic linear
Memory space is opened up, while the cluster mode of storage server can support the load balancing and data redundancy of storage resource well
Strategy, it is ensured that the qos requirement that is available reliable and meeting user service of data file.
In conclusion cloud storage system provided in an embodiment of the present invention includes: meta data server 1, data storage service
Device 2 and client 3.Wherein, meta data server 1, data storage server 2, client 3 pass through TCP/IP network protocol respectively
It connects one to one；Meta data server 1 is constructed using the framework of centralization, in storage and management cloud storage system
The metadata information of All Files；Data storage server 2 is constructed using the framework of concentrating type, is used for storage file data
Block provides the read-write service of data to client 3；Client 3 for user for inputting operation instruction, to realize writing for file
Enter, read and deletes.The cloud storage system of control stream and data flow mixing compared with the prior art, technical solution of the present invention will be literary
The control stream and data stream separation of part support the load balancing and data redundancy strategy of storage resource, improve system performance.
Those of ordinary skill in the art will appreciate that realizing all or part of the process in above-described embodiment method, being can be with
Relevant hardware is instructed to complete by computer program, the program can be stored in a computer-readable storage medium
In, the program is when being executed, it may include such as the process of the embodiment of above-mentioned each method.Wherein, the storage medium can be magnetic
Dish, CD, read-only memory (Read-Only Memory, ROM) or random access memory (Random Access
Memory, RAM) etc..
The above is a preferred embodiment of the present invention, it is noted that for those skilled in the art
For, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications are also considered as
Protection scope of the present invention.
Claims (6)
1. a kind of cloud storage system characterized by comprising meta data server, data storage server and client；
Wherein, the meta data server, data storage server, client pass through TCP/IP network protocol respectively and correspond
Connection；
The meta data server is constructed using the framework of centralization, for institute in cloud storage system described in storage and management
Documentary metadata information；The metadata information by: file global identifier, filename, file size, storage location are reflected
It penetrates, access authority, creation and modification time and copy information composition；
The data storage server is constructed using the framework of concentrating type, is used for storage file data block, Xiang Suoshu client
End provides the read-write service of data；
The client for user for inputting operation instruction, to realize the write-in, reading and deletion of file.
2. cloud storage system according to claim 1, which is characterized in that the meta data server is also used to file
Virtualization path resolves to metadata, and the metadata is mapped on the data storage in corresponding data block.
3. cloud storage system according to claim 1, which is characterized in that the meta data server is provided with rights management
Module；
The authority management module, which is used to access user, carries out Authority Verification and matching.
4. cloud storage system according to claim 1, which is characterized in that the meta data server is provided with QOS control
Module, load balancing module, node administration module and replica management module；
The meta data server monitors and manages institute by QOS control, load balancing control, node administration and replica management
State the global resource service condition of cloud storage system.
5. cloud storage system according to claim 1, which is characterized in that the data storage server is also used to copy pipe
Reason, concurrently access management and the QOS control of reading and writing data.
6. according to right want 1 described in cloud storage system, the data storage server includes several sub- storage servers；
Each sub- storage server is isomerism server, and has unified memory interface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711500129.9A CN109413130A (en) | 2017-12-30 | 2017-12-30 | A kind of cloud storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711500129.9A CN109413130A (en) | 2017-12-30 | 2017-12-30 | A kind of cloud storage system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109413130A true CN109413130A (en) | 2019-03-01 |
Family
ID=65463375
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711500129.9A Pending CN109413130A (en) | 2017-12-30 | 2017-12-30 | A kind of cloud storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109413130A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110769025A (en) * | 2019-09-06 | 2020-02-07 | 江苏中云科技有限公司 | Method for accelerating data index of multi-tenant-oriented cloud storage system |
| CN111694791A (en) * | 2020-04-01 | 2020-09-22 | 新华三大数据技术有限公司 | Data access method and device in distributed basic framework |
| CN112416892A (en) * | 2020-12-10 | 2021-02-26 | 四川警察学院 | Emergency video data cloud storage system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102170396A (en) * | 2011-05-06 | 2011-08-31 | 浙江大学 | QoS control method of cloud storage system based on differentiated service |
| CN102855294A (en) * | 2012-08-13 | 2013-01-02 | 北京联创信安科技有限公司 | Intelligent hash data layout method, cluster storage system and method thereof |
| CN103439629A (en) * | 2013-08-05 | 2013-12-11 | 国家电网公司 | Power distribution network fault diagnosis system based on data grid |
| CN103793534A (en) * | 2014-02-28 | 2014-05-14 | 苏州博纳讯动软件有限公司 | Distributed file system and implementation method for balancing storage loads and access loads of metadata |
-
2017
- 2017-12-30 CN CN201711500129.9A patent/CN109413130A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102170396A (en) * | 2011-05-06 | 2011-08-31 | 浙江大学 | QoS control method of cloud storage system based on differentiated service |
| CN102855294A (en) * | 2012-08-13 | 2013-01-02 | 北京联创信安科技有限公司 | Intelligent hash data layout method, cluster storage system and method thereof |
| CN103439629A (en) * | 2013-08-05 | 2013-12-11 | 国家电网公司 | Power distribution network fault diagnosis system based on data grid |
| CN103793534A (en) * | 2014-02-28 | 2014-05-14 | 苏州博纳讯动软件有限公司 | Distributed file system and implementation method for balancing storage loads and access loads of metadata |
Non-Patent Citations (1)
| Title |
|---|
| 许信: "云存储系统服务质量控制与可靠性技术研究", 《中国硕士期刊全文库》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110769025A (en) * | 2019-09-06 | 2020-02-07 | 江苏中云科技有限公司 | Method for accelerating data index of multi-tenant-oriented cloud storage system |
| CN110769025B (en) * | 2019-09-06 | 2022-04-22 | 江苏中云科技有限公司 | Method for accelerating data index of multi-tenant-oriented cloud storage system |
| CN111694791A (en) * | 2020-04-01 | 2020-09-22 | 新华三大数据技术有限公司 | Data access method and device in distributed basic framework |
| CN112416892A (en) * | 2020-12-10 | 2021-02-26 | 四川警察学院 | Emergency video data cloud storage system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101272313B (en) | Intermediate device for achieving virtualization of file level, file server system and relay method | |
| Zeng et al. | Research on cloud storage architecture and key technologies | |
| Hupfeld et al. | The XtreemFS architecture—a case for object‐based file systems in Grids | |
| US9558194B1 (en) | Scalable object store | |
| US9342529B2 (en) | Directory-level referral method for parallel NFS with multiple metadata servers | |
| US20090063556A1 (en) | Root node for carrying out file level virtualization and migration | |
| US10740005B1 (en) | Distributed file system deployment on a data storage system | |
| JP2016505935A (en) | Separation of content and metadata in a distributed object storage ecosystem | |
| US11106625B2 (en) | Enabling a Hadoop file system with POSIX compliance | |
| TW200422855A (en) | A distributed storage system for data-sharing among client computers running different operating system types | |
| US10852985B2 (en) | Persistent hole reservation | |
| US11086551B2 (en) | Freeing and utilizing unused inodes | |
| US11907261B2 (en) | Timestamp consistency for synchronous replication | |
| CN106293490A (en) | Data storage, the method read, Apparatus and system | |
| US20200301886A1 (en) | Inofile management and access control list file handle parity | |
| US10152493B1 (en) | Dynamic ephemeral point-in-time snapshots for consistent reads to HDFS clients | |
| CN109542861A (en) | File management method, device and system | |
| CN109413130A (en) | A kind of cloud storage system | |
| WO2012164617A1 (en) | Data management method for nas | |
| CN108604162A (en) | Optimize the access to creation data | |
| US10558373B1 (en) | Scalable index store | |
| CN103853612A (en) | Method for reading data based on digital family content under distributed storage | |
| CN105637471B (en) | Method and apparatus for being monitored and controlling to storage environment | |
| CN105022779A (en) | Method for realizing HDFS file access by utilizing Filesystem API | |
| CN110362590A (en) | Data managing method, device, system, electronic equipment and computer-readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190301 |
|
| RJ01 | Rejection of invention patent application after publication |