CN101576856A - Buffer data replacement method based on access frequency within long and short cycle - Google Patents

Buffer data replacement method based on access frequency within long and short cycle Download PDF

Info

Publication number
CN101576856A
CN101576856A CNA2009100163633A CN200910016363A CN101576856A CN 101576856 A CN101576856 A CN 101576856A CN A2009100163633 A CNA2009100163633 A CN A2009100163633A CN 200910016363 A CN200910016363 A CN 200910016363A CN 101576856 A CN101576856 A CN 101576856A
Authority
CN
China
Prior art keywords
buffer
data
access frequency
module
frequency value
Prior art date
Application number
CNA2009100163633A
Other languages
Chinese (zh)
Inventor
王文方
Original Assignee
浪潮电子信息产业股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 浪潮电子信息产业股份有限公司 filed Critical 浪潮电子信息产业股份有限公司
Priority to CNA2009100163633A priority Critical patent/CN101576856A/en
Publication of CN101576856A publication Critical patent/CN101576856A/en

Links

Abstract

The invention discloses a buffer data replacement method based on the access frequency within the long and short cycle, and belongs to data buffer management technology for disk arrays. The method is to utilize the access frequency value of buffer data to reflect the access condition of the buffer data within the long and short cycle, sort the buffer data according to the access frequency value of the buffer data, find out buffer data which is not used for the longest time, replace the buffer data out of a buffer area, and replace novel data into the buffer area. The invention structurally comprises the buffer area, a buffer replacement interface module, a buffer access frequency statistic module, a buffer access frequency sorting module, a buffer access frequency value recording module and a buffer replacement processing module. Compared with the prior art, the buffer data replacement method based on the access frequency within the long and short cycle is generally used for data buffer management of the disk arrays, optimizes buffer data replacement, fully utilizes buffer resources of a system, optimizes the utilization rate of buffer space, and improves the buffer hit ratio, thereby improving the overall performance of the disk arrays.

Description

A kind of buffer data replacement method based on access frequency within long and short cycle

Technical field

The present invention relates to a kind of metadata cache administrative skill of disk array, specifically a kind of buffer data replacement method based on access frequency within long and short cycle.

Background technology

Usually in disk array inside,, generally will reserve one section region of memory as metadata cache for improving the access efficiency of system.Metadata cache is divided into two kinds of read data buffer memory and write data buffer memorys, and the read data buffer memory is mainly raising with machine-readable efficient, and the write data buffer memory then is mainly the efficient that raising is write continuously.Metadata cache is the valuable source of disk array, cache hit rate is the important indicator that influences performance of disk arrays, utilizes data cached efficiently replacement technology, can optimize the utilization factor of spatial cache greatly, improve cache hit rate, improve the overall performance of disk array.

Cache hit rate when the buffer memory technology of replacing must be visited with the maximization host side is a starting point, thereby needs to consider the cache data access mode of host side.Host side mainly contains two kinds of access modes to data cached: long-term cycle access and short-term frequent access.Long-term cycle access is meant, data cached in the long at interval time by periodic repeated accesses, typical application comprises daily record, timed backup etc.The short-term frequent access is meant, and is data cached in the short period of time by frequent access.Hit rate for maximization host side access cache, should preserve the frequent accessed data of short-term in the buffer memory as far as possible, if but the frequent accessed data of short-term are not accessed in one long period, they also should be replaced out buffer memory prior to the data of long term periodicities visit so.This is because with regard to access mode, the data of short-term frequent access seldom can be visited once more in the long from now on time, and the data of long-term cycle access will periodically be visited in the long from now on time.

How a kind of data cached replacement technology can be provided, fully take into account the access mode of host side, take into account the situation of long-term cycle access and short-term frequent access, be one of subject matter of facing of magnetic disk array buffer storage administrative institute.

Summary of the invention

Technical assignment of the present invention provides a kind of by taking into account the shot and long term visit of host side, optimizes a kind of buffer data replacement method based on access frequency within long and short cycle of data cached replacement.

Technical assignment of the present invention is realized in the following manner, utilize cache data access frequency value, reflect data cached situation accessed in long and short cycle, and sort according to the size of cache data access frequency value, search be not used at most data cached, should data cached replace out buffer area, and new data was changed to buffer area; Structure comprises buffer area, buffer memory replacement interface module, cache access frequency statistics module, cache access frequency order module, cache access frequency value record module and buffer memory replacement processing module; Buffer area connects buffer memory and replaces interface module and buffer memory visiting frequency statistical module, buffer memory is replaced interface module and is connected the buffer memory replacement processing module, the buffer memory replacement processing module connects cache access frequency order module, and cache access frequency statistics module all is connected with cache access frequency value record module with buffer memory visiting frequency order module.

Cache access frequency value record module is with the form storage data access frequency value of record, provides cache data access frequency value searched, upgrades and replace interface; Cache access frequency statistics module mainly is responsible for calculating data cached visiting frequency value in the buffer area, and periodically upgrades the data cached visiting frequency value in the cache access frequency value record module; Cache access frequency order module can be called the data cached visiting frequency value in the cache access frequency logging modle, and sorts according to the visiting frequency value; The buffer memory replacement processing module is responsible for replacing data cached, calls cache access frequency order module and searches the data cached of visiting frequency value minimum, replaces interface module by buffer memory and should data cached replace out buffer area, and new data is changed to buffer area.

Data cached replacement flow process is two, and wherein one idiographic flow is:

(1), cache access frequency statistics module accesses buffer area data cached, and calculate data cached visiting frequency value in the buffer area;

(2), cache access frequency statistics module is periodically upgraded the data cached visiting frequency value in the cache access frequency value record module;

(3), when data cached in the host side access cache district, cache access frequency statistics module recomputates data cached visiting frequency value;

Another idiographic flow is:

(1), the buffer memory replacement processing module replaces the interface that interface module provides by calling buffer memory, sends data cached replacement request to internal system;

(2), cache access frequency order module is called the data cached visiting frequency value in the cache access frequency logging modle, and is sorted according to the visiting frequency value;

(3), the buffer memory replacement processing module carries out data cached replacement and handles, and calls the ordering of cache access frequency order module to the visiting frequency value, searches the data cached of visiting frequency value minimum;

(4), the buffer memory replacement processing module is replaced data cached replace the out buffer area of interface module with visiting frequency value minimum by buffer memory, and new data is changed to buffer area.

The size of data cached visiting frequency value is 4 bytes, and each byte has from high to low been represented interior data cached accessed situation of interval cycle from short to long; Use B nThe value of the i position of [i] expression n byte from high to low, if in the unit interval, caching data block is accessed, then puts B 0[0] is 1, otherwise puts B 0[0] be 0; Every 8 aThe individual unit interval, adopt following Policy Updates cache data access frequency value:

(1)、B a[i+1]=B a[i](i=0,1,……,6)

(2)、B a+1[i+1]=B a+1[i](i=0,1,……,6)

(3)、B a+1[0]=B a[0]||B a[1]||……||B a[6]||B a[7];

Above-mentioned rule can guarantee, if the host side access mode is long-term cycle access, it is 1 continuously that multidigit will be arranged in the low byte so, if access is the short-term frequent access, it is 1 continuously that multidigit will be arranged in the upper byte so, is 1 situation continuously and multidigit seldom can appear in low byte.

A kind of buffer data replacement method based on access frequency within long and short cycle of the present invention is generally used for the metadata cache management of disk array, has the following advantages:

(1), taken into full account host side to data cached access mode, taken into account long-term cycle access and short-term frequent access; Utilization factor that can the effective optimization spatial cache improves data cached hit rate, improves the overall performance of disk array.

(2), it is applied in the disc array system, optimize data cached replacement policy, can make full use of cache resources, reduce the operation expense of system, satisfy the requirement that performance of disk arrays is improved day by day.

Description of drawings

The present invention is further described below in conjunction with accompanying drawing.

Accompanying drawing 1 is a kind of structured flowchart of the buffer data replacement method based on access frequency within long and short cycle;

Accompanying drawing 2 is a kind of process flow diagram of the buffer data replacement method based on access frequency within long and short cycle;

Accompanying drawing 3 is a kind of data cached visiting frequency value example of the buffer data replacement method based on access frequency within long and short cycle.

Embodiment

Explain below with reference to Figure of description and specific embodiment a kind of buffer data replacement method based on access frequency within long and short cycle of the present invention being done.

Embodiment:

As shown in Figure 1, a kind of buffer data replacement method of the present invention based on access frequency within long and short cycle, its structure comprises utilizes cache data access frequency value, reflect data cached situation accessed in long and short cycle, and sort according to the size of cache data access frequency value, search be not used at most data cached, should data cached replace out buffer area, and new data is changed to buffer area; Structure comprises buffer area, buffer memory replacement interface module, cache access frequency statistics module, cache access frequency order module, cache access frequency value record module and buffer memory replacement processing module; Buffer area connects buffer memory and replaces interface module and buffer memory visiting frequency statistical module, buffer memory is replaced interface module and is connected the buffer memory replacement processing module, the buffer memory replacement processing module connects cache access frequency order module, and cache access frequency statistics module all is connected with cache access frequency value record module with buffer memory visiting frequency order module.

Cache access frequency value record module is with the form storage data access frequency value of record, provides cache data access frequency value searched, upgrades and replace interface; Cache access frequency statistics module mainly is responsible for calculating data cached visiting frequency value in the buffer area, and periodically upgrades the data cached visiting frequency value in the cache access frequency value record module; Cache access frequency order module can be called the data cached visiting frequency value in the cache access frequency logging modle, and sorts according to the visiting frequency value; The buffer memory replacement processing module is responsible for replacing data cached, calls cache access frequency order module and searches the data cached of visiting frequency value minimum, replaces interface module by buffer memory and should data cached replace out buffer area, and new data is changed to buffer area.

As shown in Figure 2, data cached replacement flow process is two, and wherein one idiographic flow is:

(1), cache access frequency statistics module accesses buffer area data cached, and calculate data cached visiting frequency value in the buffer area;

(2), cache access frequency statistics module is periodically upgraded the data cached visiting frequency value in the cache access frequency value record module;

(3), when data cached in the host side access cache district, cache access frequency statistics module recomputates data cached visiting frequency value;

Another idiographic flow is:

(1), the buffer memory replacement processing module replaces the interface that interface module provides by calling buffer memory, sends data cached replacement request to internal system;

(2), cache access frequency order module is called the data cached visiting frequency value in the cache access frequency logging modle, and is sorted according to the visiting frequency value;

(3), the buffer memory replacement processing module carries out data cached replacement and handles, and calls the ordering of cache access frequency order module to the visiting frequency value, searches the data cached of visiting frequency value minimum;

(4), the buffer memory replacement processing module is replaced data cached replace the out buffer area of interface module with visiting frequency value minimum by buffer memory, and new data is changed to buffer area.

As shown in Figure 3, the size of data cached visiting frequency value is 4 bytes, and each byte has from high to low been represented interior data cached accessed situation of interval cycle from short to long; Use B nThe value of the i position of [i] expression n byte from high to low, if in the unit interval, caching data block is accessed, then puts B 0[0] is 1, otherwise puts B 0[0] be 0; Every 8 aThe individual unit interval, adopt following Policy Updates cache data access frequency value:

(1)、B a[i+1]=B a[i](i=0,1,……,6)

(2)、B a+1[i+1]=B a+1[i](i=0,1,……,6)

(3)、B a+1[0]=B a[0]||B a[1]||……||B a[6]||B a[7];

Above-mentioned rule can guarantee, if the host side access mode is long-term cycle access, it is 1 continuously that multidigit will be arranged in the low byte so, if access is the short-term frequent access, it is 1 continuously that multidigit will be arranged in the upper byte so, is 1 situation continuously and multidigit seldom can appear in low byte.

Except that the described technical characterictic of instructions, be the known technology of those skilled in the art.

Claims (4)

1, a kind of buffer data replacement method based on access frequency within long and short cycle, it is characterized in that utilizing cache data access frequency value, reflect data cached situation accessed in long and short cycle, and sort according to the size of cache data access frequency value, search be not used at most data cached, should data cached replace out buffer area, and new data was changed to buffer area; Structure comprises buffer area, buffer memory replacement interface module, cache access frequency statistics module, cache access frequency order module, cache access frequency value record module and buffer memory replacement processing module; Buffer area connects buffer memory and replaces interface module and buffer memory visiting frequency statistical module, buffer memory is replaced interface module and is connected the buffer memory replacement processing module, the buffer memory replacement processing module connects cache access frequency order module, and cache access frequency statistics module all is connected with cache access frequency value record module with buffer memory visiting frequency order module.
2, a kind of buffer data replacement method according to claim 1 based on access frequency within long and short cycle, it is characterized in that the form storage data access frequency value of cache access frequency value record module, provide cache data access frequency value searched, upgrades and replace interface with record; Cache access frequency statistics module mainly is responsible for calculating data cached visiting frequency value in the buffer area, and periodically upgrades the data cached visiting frequency value in the cache access frequency value record module; Cache access frequency order module can be called the data cached visiting frequency value in the cache access frequency logging modle, and sorts according to the visiting frequency value; The buffer memory replacement processing module is responsible for replacing data cached, calls cache access frequency order module and searches the data cached of visiting frequency value minimum, replaces interface module by buffer memory and should data cached replace out buffer area, and new data is changed to buffer area.
3, a kind of buffer data replacement method based on access frequency within long and short cycle according to claim 1 is characterized in that data cached replacement flow process is two, and wherein one idiographic flow is:
(1), cache access frequency statistics module accesses buffer area data cached, and calculate data cached visiting frequency value in the buffer area;
(2), cache access frequency statistics module is periodically upgraded the data cached visiting frequency value in the cache access frequency value record module;
(3), when data cached in the host side access cache district, cache access frequency statistics module recomputates data cached visiting frequency value;
Another idiographic flow is:
(1), the buffer memory replacement processing module replaces the interface that interface module provides by calling buffer memory, sends data cached replacement request to internal system;
(2), cache access frequency order module is called the data cached visiting frequency value in the cache access frequency logging modle, and is sorted according to the visiting frequency value;
(3), the buffer memory replacement processing module carries out data cached replacement and handles, and calls the ordering of cache access frequency order module to the visiting frequency value, searches the data cached of visiting frequency value minimum;
(4), the buffer memory replacement processing module is replaced data cached replace the out buffer area of interface module with visiting frequency value minimum by buffer memory, and new data is changed to buffer area.
4, a kind of buffer data replacement method according to claim 1 based on access frequency within long and short cycle, the size that it is characterized in that data cached visiting frequency value is 4 bytes, each byte has from high to low been represented interior data cached accessed situation of interval cycle from short to long; Use B nThe value of the i position of [i] expression n byte from high to low, if in the unit interval, caching data block is accessed, then puts B 0[0] is 1, otherwise puts B 0[0] be 0; Every 8 aThe individual unit interval, adopt following Policy Updates cache data access frequency value:
(1)、B a[i+1]=B a[i](i=0,1,……,6)
(2)、B a+1[i+1]=B a+1[i](i=0,1,……,6)
(3)、B a+1[0]=B a[0]||B a[1]||……||B a[6]||B a[7];
Above-mentioned rule can guarantee, if the host side access mode is long-term cycle access, it is 1 continuously that multidigit will be arranged in the low byte so, if access is the short-term frequent access, it is 1 continuously that multidigit will be arranged in the upper byte so, is 1 situation continuously and multidigit seldom can appear in low byte.
CNA2009100163633A 2009-06-18 2009-06-18 Buffer data replacement method based on access frequency within long and short cycle CN101576856A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CNA2009100163633A CN101576856A (en) 2009-06-18 2009-06-18 Buffer data replacement method based on access frequency within long and short cycle

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CNA2009100163633A CN101576856A (en) 2009-06-18 2009-06-18 Buffer data replacement method based on access frequency within long and short cycle

Publications (1)

Publication Number Publication Date
CN101576856A true CN101576856A (en) 2009-11-11

Family

ID=41271791

Family Applications (1)

Application Number Title Priority Date Filing Date
CNA2009100163633A CN101576856A (en) 2009-06-18 2009-06-18 Buffer data replacement method based on access frequency within long and short cycle

Country Status (1)

Country Link
CN (1) CN101576856A (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102999443A (en) * 2012-11-16 2013-03-27 广州优倍达信息科技有限公司 Management method of computer cache system
CN103744800A (en) * 2013-12-30 2014-04-23 龙芯中科技术有限公司 Cache operation method and device for replay mechanism
EP2779170A1 (en) * 2013-03-15 2014-09-17 Seagate Technology LLC Staging sorted data in intermediate storage
CN105095107A (en) * 2014-05-04 2015-11-25 腾讯科技(深圳)有限公司 Buffer memory data cleaning method and apparatus
US9384793B2 (en) 2013-03-15 2016-07-05 Seagate Technology Llc Dynamic granule-based intermediate storage
CN106302784A (en) * 2016-08-27 2017-01-04 浪潮(北京)电子信息产业有限公司 A kind of method and apparatus improving caching speed
CN107291635A (en) * 2017-06-16 2017-10-24 郑州云海信息技术有限公司 A kind of buffer replacing method and device
CN110008190A (en) * 2019-03-21 2019-07-12 武汉理工大学 A kind of periodic small documents caching replacement method
CN111221749A (en) * 2019-11-15 2020-06-02 新华三半导体技术有限公司 Data block writing method and device, processor chip and Cache

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102999443A (en) * 2012-11-16 2013-03-27 广州优倍达信息科技有限公司 Management method of computer cache system
CN102999443B (en) * 2012-11-16 2015-09-09 广州优倍达信息科技有限公司 A kind of management method of Computer Cache system
EP2779170A1 (en) * 2013-03-15 2014-09-17 Seagate Technology LLC Staging sorted data in intermediate storage
US9740406B2 (en) 2013-03-15 2017-08-22 Seagate Technology Llc Dynamic granule-based intermediate storage
US9384793B2 (en) 2013-03-15 2016-07-05 Seagate Technology Llc Dynamic granule-based intermediate storage
US9588887B2 (en) 2013-03-15 2017-03-07 Seagate Technology Llc Staging sorted data in intermediate storage
US9588886B2 (en) 2013-03-15 2017-03-07 Seagate Technology Llc Staging sorted data in intermediate storage
CN103744800B (en) * 2013-12-30 2016-09-14 龙芯中科技术有限公司 Caching method and device towards replay mechanism
CN103744800A (en) * 2013-12-30 2014-04-23 龙芯中科技术有限公司 Cache operation method and device for replay mechanism
CN105095107B (en) * 2014-05-04 2019-07-16 腾讯科技(深圳)有限公司 Clear up data cached method and device
CN105095107A (en) * 2014-05-04 2015-11-25 腾讯科技(深圳)有限公司 Buffer memory data cleaning method and apparatus
CN106302784A (en) * 2016-08-27 2017-01-04 浪潮(北京)电子信息产业有限公司 A kind of method and apparatus improving caching speed
CN107291635A (en) * 2017-06-16 2017-10-24 郑州云海信息技术有限公司 A kind of buffer replacing method and device
CN110008190A (en) * 2019-03-21 2019-07-12 武汉理工大学 A kind of periodic small documents caching replacement method
CN111221749A (en) * 2019-11-15 2020-06-02 新华三半导体技术有限公司 Data block writing method and device, processor chip and Cache

Similar Documents

Publication Publication Date Title
Jain et al. Back to the future: leveraging Belady's algorithm for improved cache replacement
CN103019958B (en) Usage data attribute manages the method for the data in solid-state memory
US9449005B2 (en) Metadata storage system and management method for cluster file system
Huang et al. Improving flash-based disk cache with lazy adaptive replacement
Ma et al. LazyFTL: a page-level flash translation layer optimized for NAND flash memory
Do et al. Turbocharging DBMS buffer pool using SSDs
JP6356675B2 (en) Aggregation / grouping operation: Hardware implementation of hash table method
US8700674B2 (en) Database storage architecture
US20160371190A1 (en) Selective compression in data storage systems
KR102005831B1 (en) Managing storage of data for range-based searching
CN1317645C (en) Method and apparatus for multithreaded cache with cache eviction based on thread identifier
US8972690B2 (en) Methods and apparatuses for usage based allocation block size tuning
Xie et al. Locality in search engine queries and its implications for caching
US9355109B2 (en) Multi-tier caching
CN100468400C (en) Method and system for improving information search speed
Park et al. CFLRU: a replacement algorithm for flash memory
US8402205B2 (en) Multi-tiered metadata scheme for a data storage array
CN103294710B (en) A kind of data access method and device
CN102542034B (en) A kind of result set cache method of database interface
CN103488709B (en) A kind of index establishing method and system, search method and system
Lu et al. BloomStore: Bloom-filter based memory-efficient key-value store for indexing of data deduplication on flash
US20180189003A1 (en) File system block-level tiering and co-allocation
EP2895958B1 (en) Address mapping
Zhao et al. SPATL: Honey, I shrunk the coherence directory
Park et al. Hot data identification for flash-based storage systems using multiple bloom filters

Legal Events

Date Code Title Description
PB01 Publication
C06 Publication
SE01 Entry into force of request for substantive examination
C10 Entry into substantive examination
WD01 Invention patent application deemed withdrawn after publication

Open date: 20091111

C02 Deemed withdrawal of patent application after publication (patent law 2001)