CN107291633A - A kind of new cluster-based storage write buffer grading management method - Google Patents
A kind of new cluster-based storage write buffer grading management method Download PDFInfo
- Publication number
- CN107291633A CN107291633A CN201710575755.8A CN201710575755A CN107291633A CN 107291633 A CN107291633 A CN 107291633A CN 201710575755 A CN201710575755 A CN 201710575755A CN 107291633 A CN107291633 A CN 107291633A
- Authority
- CN
- China
- Prior art keywords
- write buffer
- cluster
- accounting
- based storage
- accounting area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F12/0893—Caches characterised by their organisation or structure
- G06F12/0897—Caches characterised by their organisation or structure with two or more cache hierarchy levels
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/06—Addressing a physical block of locations, e.g. base addressing, module addressing, memory dedication
- G06F12/0646—Configuration or reconfiguration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
- G06F2212/10—Providing a specific technical effect
- G06F2212/1016—Performance improvement
- G06F2212/1024—Latency reduction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
- G06F2212/26—Using a specific storage system architecture
- G06F2212/263—Network storage, e.g. SAN or NAS
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention relates to a kind of new cluster-based storage write buffer grading management method, this method sets rank that is high, neutralizing low three kinds different write buffer accountings by the write buffer to cluster-based storage memory node, the frequency of three kinds of different synchronous rules of three kinds of different write buffers of write buffer accounting rank correspondence, the frequency highest of the synchronous rule of write buffer during write buffer high accounting, in write buffer during accounting write buffer synchronization rule frequency moderate, the frequency of the synchronous rule of write buffer is minimum during write buffer low accounting.
Description
Technical field
Patent of the present invention is related to computer cluster field of storage.
Background technology
Cluster-based storage is a kind of new storage architecture for being different from traditional SAN and NAS.Traditional SAN and NAS is carried respectively
What is supplied is the storage architecture of data block and two different stages of file, and cluster-based storage is the storage system for being mainly directed towards file-level
System.Cluster-based storage is that the memory space in many storage devices is aggregated into one to provide unified access to application server
The storage pool of interface and administration interface, using can pellucidly be accessed by the access interface and using in all storage devices
Disk, can give full play to the performance and disk utilization of storage device.Data will be stored according to certain rule from many
Store and read in equipment, to obtain higher concurrent access performance.Once running into the bottleneck of storage system, two kinds of choosings are just had
Select：One is：Using the more powerful single storage system of hardware；Two are：Using the storage system of several common performances come group
Into " cluster of storage "." cluster " constitutes " cluster as an architecture being widely used if used in storage
Storage ", so that it may performance, capacity, reliability and the availability for the storage resource being scaling up are provided, stand-alone device is breached
A variety of limitations.
General cluster-based storage extends very convenient, the extension stored as playing with building blocks.Especially for that
A little users to the more difficult prediction of data growth trend, can first buy a part of storage, when having demand, add at any time,
Use without influenceing existing storage.The operation of cluster-based storage divides all via distributed operating system United Dispatching and distribution
It is scattered on cluster-based storage each memory node and completes.Do not appoint between having the advantage that each node using distributed operating system
What is distinguished, and the difference without primary and secondary, functionally, all memory node functions are completely the same, so just can be truly realized performance most
It is excellent.Cluster-based storage can all realize load balancing by the effect of distributed operating system in front-end and back-end.Front-end access cluster
The operation of storage, by several load balancings, is distributed to accessing on each memory node of cluster-based storage.Back-end access
Data, by open framework and back-end network, data can be distributed on all nodes and be deposited and be read.
Patent of invention content
Patent of the present invention is related to a kind of new cluster-based storage write buffer grading management method, and this method to cluster-based storage by storing
The write buffer of node sets rank that is high, neutralizing low three kinds different write buffer accountings, three kinds of different write buffer accounting ranks
The frequency of the synchronous rule of three kinds of different write buffers of correspondence, the frequency highest of the synchronous rule of write buffer, writes during write buffer high accounting
In caching during accounting write buffer synchronization rule frequency moderate, the frequency of the synchronous rule of write buffer is minimum during write buffer low accounting.
The write buffer of cluster-based storage memory node is therefore the industry in order to lift response efficiency when business should send write request
Business, which should send write request, can be first written to write buffer, then by the synchronous rule of write buffer to physical volume, cluster-based storage memory node
In order to lift the efficiency of synchronous rule, can using merge multiple business should send write request and together synchronization rule to physics
Volume, to reduce the addressing time and stand-by period of physical volume.
Brief description of the drawings
Fig. 1 is a kind of new cluster-based storage write buffer grading management method structural representation of patent of the present invention.
Embodiment
In order that the object, technical solution and advantage of patent of the present invention are more clearly understood, below in conjunction with accompanying drawing and implementation
Example, patent of the present invention is further elaborated.It should be appreciated that specific embodiment described herein is only to explain
Patent of the present invention, is not intended to limit the present invention patent.
Referring to Fig. 1, Fig. 1 is a kind of new cluster-based storage write buffer grading management method structural representation of patent of the present invention
Figure.
A kind of new cluster-based storage write buffer grading management method, it is characterised in that methods described is accounted for comprising write buffer height
Compare area（10）, accounting area in write buffer（11）, the low accounting area of write buffer（12）, cluster-based storage memory node physical volume（13）, collection
Group's storage memory node a（14a）, cluster-based storage memory node b（14b）, cluster-based storage memory node c（14c）, cluster-based storage deposits
Store up inter-node communication network（15）, this method cluster-based storage memory node a（14a）, cluster-based storage memory node b（14b）And collection
Group's storage memory node c（14c）Pass through communication network between cluster-based storage memory node（15）Cluster-based storage is constituted, and cluster is deposited
Store up memory node a（14a）Write buffer by write buffer accounting different demarcation be the high accounting area of write buffer（10）, accounting in write buffer
Area（11）With the low accounting area of write buffer（12）, the high accounting area of write buffer（10）, accounting area in write buffer（11）With write buffer is low accounts for
Compare area（12）Three kinds of different write buffer accountings correspond to different synchronous rules to cluster-based storage memory node physical volume（13）Frequently
Rate.
A kind of new cluster-based storage write buffer grading management method, it is characterised in that the high accounting area of this method write buffer
（10）, accounting area in write buffer（11）With the low accounting area of write buffer（12）Accounting ratio can with self-defined setting, wherein
The high accounting area of write buffer（10）Setting range is accounting area in 50 percent to a hundred per cent, write buffer（11）Setting range is
80 30 percent to percent, the low accounting area of write buffer（12）Setting range is less than 50 percent, the rule of setting
It is the high accounting area of write buffer（10）More than accounting area in write buffer（11）, accounting area in write buffer（11）More than the low accounting of write buffer
Area（12）, such as high accounting area of write buffer（10）71 percent is set in a hundred per cent, the low accounting area of write buffer（12）If
Less than 30 percent is scheduled on, then accounting area in write buffer（11）It is set as 70 31 percent to percent.
A kind of new cluster-based storage write buffer grading management method, it is characterised in that this method is to the high accounting area of write buffer
（10）, accounting area in write buffer（11）With the low accounting area of write buffer（12）Synchronous rule frequency setting it is different, accounted in write buffer
Compare area（11）For normal speed Frequency Synchronization rule, the low accounting area of write buffer（12）For normal Half Speed Frequency Synchronization rule,
The high accounting area of write buffer（10）For normal two speeds Frequency Synchronization rule.
The preferred embodiment of patent of the present invention is the foregoing is only, patent is not intended to limit the invention, it is all at this
Any modifications, equivalent substitutions and improvements made within the spirit and principle of patent of invention etc., should be included in patent of the present invention
Protection domain within.
Claims (3)
1. a kind of new cluster-based storage write buffer grading management method, it is characterised in that methods described includes the high accounting of write buffer
Area（10）, accounting area in write buffer（11）, the low accounting area of write buffer（12）, cluster-based storage memory node physical volume（13）, cluster
Store memory node a（14a）, cluster-based storage memory node b（14b）, cluster-based storage memory node c（14c）, cluster-based storage storage
Inter-node communication network（15）, this method cluster-based storage memory node a（14a）, cluster-based storage memory node b（14b）And cluster
Store memory node c（14c）Pass through communication network between cluster-based storage memory node（15）Cluster-based storage is constituted, and to cluster-based storage
Memory node a（14a）Write buffer by write buffer accounting different demarcation be the high accounting area of write buffer（10）, accounting area in write buffer
（11）With the low accounting area of write buffer（12）, the high accounting area of write buffer（10）, accounting area in write buffer（11）With the low accounting of write buffer
Area（12）Three kinds of different write buffer accountings correspond to different synchronous rules to cluster-based storage memory node physical volume（13）Frequency.
2. a kind of new cluster-based storage write buffer grading management method according to claim 1, it is characterised in that this method
The high accounting area of write buffer（10）, accounting area in write buffer（11）With the low accounting area of write buffer（12）Accounting ratio be to make by oneself
Justice setting, the wherein high accounting area of write buffer（10）Setting range is accounting area in 50 percent to a hundred per cent, write buffer
（11）Setting range is 80 30 percent to percent, the low accounting area of write buffer（12）Setting range be 50 percent with
Under, the rule of setting is the high accounting area of write buffer（10）More than accounting area in write buffer（11）, accounting area in write buffer（11）Greatly
In the low accounting area of write buffer（12）, such as high accounting area of write buffer（10）71 percent is set in a hundred per cent, write buffer
Low accounting area（12）Less than 30 percent is set in, then accounting area in write buffer（11）It is set as 31 percent to percentage
70.
3. a kind of new cluster-based storage write buffer grading management method according to claim 1, it is characterised in that this method
To the high accounting area of write buffer（10）, accounting area in write buffer（11）With the low accounting area of write buffer（12）Synchronous rule frequency setting
Accounting area in difference, write buffer（11）For normal speed Frequency Synchronization rule, the low accounting area of write buffer（12）For normal half
Fast Frequency Synchronization rule, the high accounting area of write buffer（10）For normal two speeds Frequency Synchronization rule.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710575755.8A CN107291633A (en) | 2017-07-14 | 2017-07-14 | A kind of new cluster-based storage write buffer grading management method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710575755.8A CN107291633A (en) | 2017-07-14 | 2017-07-14 | A kind of new cluster-based storage write buffer grading management method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107291633A true CN107291633A (en) | 2017-10-24 |
Family
ID=60100433
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710575755.8A Pending CN107291633A (en) | 2017-07-14 | 2017-07-14 | A kind of new cluster-based storage write buffer grading management method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107291633A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109582244A (en) * | 2018-12-05 | 2019-04-05 | 广东浪潮大数据研究有限公司 | A kind of metadata rule method, apparatus, terminal and computer readable storage medium |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100077154A1 (en) * | 2008-09-24 | 2010-03-25 | Sun Microsystems, Inc. | Method and system for optimizing processor performance by regulating issue of pre-fetches to hot cache sets |
| CN102170460A (en) * | 2011-03-10 | 2011-08-31 | 浪潮(北京)电子信息产业有限公司 | Cluster storage system and data storage method thereof |
| CN102291450A (en) * | 2011-08-08 | 2011-12-21 | 浪潮电子信息产业股份有限公司 | Data online hierarchical storage method in cluster storage system |
| CN102882928A (en) * | 2012-08-31 | 2013-01-16 | 浪潮电子信息产业股份有限公司 | Hierarchical storage management method of cloud storing system |
| CN103400598A (en) * | 2007-08-14 | 2013-11-20 | 三星电子株式会社 | Solid state memory, computer system including same, and method of operating same |
| CN103713861A (en) * | 2014-01-09 | 2014-04-09 | 浪潮(北京)电子信息产业有限公司 | File processing method and system based on hierarchical division |
| CN203930810U (en) * | 2014-05-26 | 2014-11-05 | 中国能源建设集团广东省电力设计研究院 | A kind of mixing storage system based on multidimensional data similarity |
| CN104346294A (en) * | 2013-07-31 | 2015-02-11 | 华为技术有限公司 | Data reading/writing method and device and computer system on basis of multi-level Cache |
| CN104866434A (en) * | 2015-06-01 | 2015-08-26 | 北京圆通慧达管理软件开发有限公司 | Multi-application-oriented data storage system and data storage and calling method |
| CN105573857A (en) * | 2014-10-10 | 2016-05-11 | 北京计算机技术及应用研究所 | Method and system for buffering mirror image by multi-control disk array |
| CN105653524A (en) * | 2014-11-10 | 2016-06-08 | 阿里巴巴集团控股有限公司 | Data storage method, device and system |
| CN106790422A (en) * | 2016-12-02 | 2017-05-31 | 北京锐安科技有限公司 | A kind of data buffer storage cluster and data retrieval method for WEB application |
-
2017
- 2017-07-14 CN CN201710575755.8A patent/CN107291633A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103400598A (en) * | 2007-08-14 | 2013-11-20 | 三星电子株式会社 | Solid state memory, computer system including same, and method of operating same |
| US20100077154A1 (en) * | 2008-09-24 | 2010-03-25 | Sun Microsystems, Inc. | Method and system for optimizing processor performance by regulating issue of pre-fetches to hot cache sets |
| CN102170460A (en) * | 2011-03-10 | 2011-08-31 | 浪潮(北京)电子信息产业有限公司 | Cluster storage system and data storage method thereof |
| CN102291450A (en) * | 2011-08-08 | 2011-12-21 | 浪潮电子信息产业股份有限公司 | Data online hierarchical storage method in cluster storage system |
| CN102882928A (en) * | 2012-08-31 | 2013-01-16 | 浪潮电子信息产业股份有限公司 | Hierarchical storage management method of cloud storing system |
| CN104346294A (en) * | 2013-07-31 | 2015-02-11 | 华为技术有限公司 | Data reading/writing method and device and computer system on basis of multi-level Cache |
| CN103713861A (en) * | 2014-01-09 | 2014-04-09 | 浪潮(北京)电子信息产业有限公司 | File processing method and system based on hierarchical division |
| CN203930810U (en) * | 2014-05-26 | 2014-11-05 | 中国能源建设集团广东省电力设计研究院 | A kind of mixing storage system based on multidimensional data similarity |
| CN105573857A (en) * | 2014-10-10 | 2016-05-11 | 北京计算机技术及应用研究所 | Method and system for buffering mirror image by multi-control disk array |
| CN105653524A (en) * | 2014-11-10 | 2016-06-08 | 阿里巴巴集团控股有限公司 | Data storage method, device and system |
| CN104866434A (en) * | 2015-06-01 | 2015-08-26 | 北京圆通慧达管理软件开发有限公司 | Multi-application-oriented data storage system and data storage and calling method |
| CN106790422A (en) * | 2016-12-02 | 2017-05-31 | 北京锐安科技有限公司 | A kind of data buffer storage cluster and data retrieval method for WEB application |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109582244A (en) * | 2018-12-05 | 2019-04-05 | 广东浪潮大数据研究有限公司 | A kind of metadata rule method, apparatus, terminal and computer readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102255962B (en) | Distributive storage method, device and system | |
| CN103116661B (en) | A kind of data processing method of database | |
| US20150127691A1 (en) | Efficient implementations for mapreduce systems | |
| US10356150B1 (en) | Automated repartitioning of streaming data | |
| US20160350302A1 (en) | Dynamically splitting a range of a node in a distributed hash table | |
| CN104182508B (en) | A kind of data processing method and data processing equipment | |
| CN104317736B (en) | A kind of distributed file system multi-level buffer implementation method | |
| WO2012126317A1 (en) | File storage method and device thereof | |
| CN106095337A (en) | A kind of cloud disk based on SAN network storage is quickly shared method | |
| CN103176754A (en) | Reading and storing method for massive amounts of small files | |
| CN106534308B (en) | Method and device for solving data block access hot spot in distributed storage system | |
| CN106570113B (en) | Mass vector slice data cloud storage method and system | |
| CN103595799A (en) | Method for achieving distributed shared data bank | |
| CN104054071A (en) | Method for accessing storage device and storage device | |
| Dai et al. | Improving load balance for data-intensive computing on cloud platforms | |
| Islam et al. | Efficient data access strategies for Hadoop and Spark on HPC cluster with heterogeneous storage | |
| CN105516313A (en) | Distributed storage system used for big data | |
| CN107506146A (en) | A kind of data-storage system | |
| CN102970349B (en) | A kind of memory load equalization methods of DHT network | |
| CN107291633A (en) | A kind of new cluster-based storage write buffer grading management method | |
| CN107357532A (en) | A kind of new cache pre-reading implementation method of new cluster-based storage | |
| Qin et al. | Study on Replica Strategy of Big Data Storage based on Cloud Environment | |
| CN107181826A (en) | A kind of data cached synchronization realizing method of new cluster-based storage | |
| Zhuang et al. | Design of a more scalable database system | |
| Bae et al. | Efficient orchestration of host and remote shared memory for memory intensive workloads |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20171024 |
|
| WD01 | Invention patent application deemed withdrawn after publication |