CN102521147A - Management method by using rapid non-volatile medium as cache - Google Patents

Management method by using rapid non-volatile medium as cache Download PDF

Info

Publication number
CN102521147A
CN102521147A CN2011103643112A CN201110364311A CN102521147A CN 102521147 A CN102521147 A CN 102521147A CN 2011103643112 A CN2011103643112 A CN 2011103643112A CN 201110364311 A CN201110364311 A CN 201110364311A CN 102521147 A CN102521147 A CN 102521147A
Authority
CN
China
Prior art keywords
buffer memory
state hard
solid state
hard disc
region
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN2011103643112A
Other languages
Chinese (zh)
Other versions
CN102521147B (en
Inventor
袁清波
许建卫
刘新春
邵宗有
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dawning Information Industry Beijing Co Ltd
Dawning Information Industry Co Ltd
Original Assignee
Dawning Information Industry Beijing Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dawning Information Industry Beijing Co Ltd filed Critical Dawning Information Industry Beijing Co Ltd
Priority to CN201110364311.2A priority Critical patent/CN102521147B/en
Publication of CN102521147A publication Critical patent/CN102521147A/en
Application granted granted Critical
Publication of CN102521147B publication Critical patent/CN102521147B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Memory System Of A Hierarchy Structure (AREA)

Abstract

The invention provides a management method by using a rapid non-volatile medium as a cache. According to the method, a solid-state hard disk is used as a caching device, each caching device is shared by multiple disks, and each disk can only use one caching device; the caching device is divided into a region structure, and mapping connection is adopted between the caching devices and the disks; the caching device is divided into two parts, the front part is used for storing the region data structure in a memory, and the back part is a cache. The method provided by the invention adopts a policy that metadata can be written in the solid-state hard disk in real time to solve the problem, and synchronizes the metadata for managing the dirty data in an assigned position of the solid-state hard disk while the dirty data are written in the solid-state hard disk, so that all the data cached in the solid-sate hard disk without being written in the conventional disk can be read only by reading the metadata when the system is restarted, and further all the data can be prevented from being lost when the system breaks down and is in power failure.

Description

A kind of management method of using the high speed non-volatile media to do buffer memory
Technical field
The present invention relates to the disk buffering administrative skill, specifically, relate to a kind of operating strategy of metadata cache in the high speed non-volatile media of disk at a slow speed.
Background technology
Disk buffering is divided into to be read buffer memory and writes buffer memory.Read buffer memory and be meant, operating system is the file data that read, under the idle situation of internal memory, stays in the memory headroom,, next software or user just needn't read from disk again when reading identical file once more, thus raising speed.Write data that buffer memory will write disk in fact exactly earlier the system of being stored in write in the memory headroom that buffer memory distributes, when the data in being saved in memory pool reach a degree, just data are saved in the hard disk.Can reduce actual disk operating like this, effectively protect disk to avoid the read-write operation of repetition and the damage that causes, also can reduce the required time that writes.According to the difference of writing mode, write two kinds of general formula and rewritings.Write general formula when reading hard disc data, system is the inspection request instruction earlier, looks at desired data whether in buffer memory, words just see the data of response off by buffer memory, this process is called hits.The data of system in just needn't access hard disk because the speed ratio magnetic medium of SDRAM is a lot of soon, have therefore also just been accelerated the speed of data transmission like this.Rewriting is also looked in buffer memory when writing hard disc data, just finds to be write in the dish with regard to data by buffer memory, and most hard disks all are the rewriting buffer memorys that adopts, and has so just improved performance greatly.
Yet, adopt the volatile devices as internal memory, after system crash or power down, the data that are buffered on the high-speed equipment will be lost, this to be similar to database such be insufferable to the very high application program of data security requirement.
Summary of the invention
The present invention uses the alternative volatibility high-speed equipment of high speed non-volatile media to come buffer memory data in magnetic disk at a slow speed, even under the situation of system crash or power down, data in buffer can not lost yet, and can data in buffer be read after restarting through the form of agreement.
A kind of management method of using the high speed non-volatile media to do buffer memory uses solid state hard disc as buffer memory equipment, and each buffer memory equipment can be shared by a plurality of disks, and each disk can only use a buffer memory equipment; Said buffer memory equipment is divided the region structure, and the mapping of employing group links to each other between buffer memory equipment and the disk; Buffer memory equipment is divided into two parts, and the region data structure in the previous section stored memory, aft section are buffer memory.
Preferably, said solid state hard disc can be formed raid disk array expansion buffer memory capacity and improve performance.
Preferably, said region size is 64KB~1MB, and is 2 power, and said region can not be modified behind equipment operation.
Preferably, when the mapping of employing group links to each other between said buffer memory equipment and the disk, give tacit consent to every group of 16 region, can be through grand change
Preferably, said buffer memory equipment finds corresponding region data structure according to recorded address information in the bio when read operation, check through the region data structure whether metadata cache is in solid state hard disc then, if then directly read from solid state hard disc; Otherwise, read and be cached to the solid state hard disc from disk.
Preferably, said buffer memory equipment finds corresponding region data structure according to recorded address information in the bio when write operation, if idle solid state hard disc space is arranged, then directly data is write toward solid state hard disc; Otherwise, write toward conventional hard; If the region data structure metadata information of writing past solid state hard disc then upgrading correspondence simultaneously is to solid state hard disc.
Preferably, said buffer memory equipment behind system failure recovery, can scan the metadata information that leaves in the solid state hard disc, and recover all environment before according to these information under the situation of power down after the system crash, and staying alone is buffered in the dirty data in the solid state hard disc.
The strategy that the present invention takes metadata to be write in real time solid state hard disc solves this problem; When dirty data is write into solid state hard disc; The metadata of managing these data also is synchronized to the assigned address of solid state hard disc; Can all be buffered in solid state hard disc and the data that do not write traditional magnetic disk are read as long as read these metadata when restarting, thereby all data can guarantee also under the situation of system crash and power down not lose.
Description of drawings
Fig. 1 is an one-piece construction synoptic diagram of the present invention
Fig. 2 is a mapping relations synoptic diagram of the present invention
Embodiment
The present invention utilizes the Device Mapper mechanism of Linux to manage a plurality of block devices, sets up a two-stage storage system with high-speed equipment wherein as the buffer memory of low-speed device, obtains higher memory property with lower cost, and one-piece construction is as shown in Figure 1.
The present invention is divided into buffer memory equipment and disk unit with the equipment of being managed, the higher solid state hard disc of buffer memory equipment usability wherein, and available polylith solid state hard disc is formed RAID to improve the performance of buffer memory in actual the use.Buffer memory equipment and disk unit all are sightless to the user, but the present invention can provide pseudo-device identical with the disk unit number, that characteristic is identical to the user.The present invention can manage a plurality of buffer memory equipment and disk unit, and buffer memory equipment and disk unit are the relations of one-to-many, and promptly a buffer memory equipment can be shared by a plurality of disk units, but any one disk unit uses a buffer memory equipment only.As for equipment, specifically then corresponding to disk partition, i.e. block device file/dev/sda1 under the Linux etc.
Why selecting solid state hard disc rather than directly use equipment more at a high speed such as internal memory, be because the data on the solid state hard disc can not be lost after system crash or power down, and the capacity of solid state hard disc is bigger, can the buffer memory more data.But, though data in buffer can be kept in the solid state hard disc, must take certain measure to obtain, though we but can't analyze the real meaning of these data otherwise data exist to the management of metadata of these data.
The present invention makes buffer memory to disk, and the thinking of itself is more directly perceived, is similar to processor cache, and buffer memory equipment is divided into many region by fixed size, and usually, the size of region is 64K~1M, and is 2 power, in case operation can not dynamically be changed again.Region is the base unit of buffer memory equipment control, but the cache line in the analogy processor.Each region safeguards information such as corresponding state, necessary lock, tag.Region is bigger with respect to the request of file system; The least unit of file system IO generally is 512 bytes (being referred to as sector); Therefore an IO of file system is more possibly be the subregion that has only covered certain region; Given this, each region also need safeguard some bitmaps, is used for the wherein state of each sector of mark.Set associative is adopted in the mapping of buffer memory equipment and disk unit, gives tacit consent to one group of 16 region, can be to relevant grand the adjustment.
The data structure of management data mapping relations (struct region) is arranged in internal memory, and internal memory is a volatile devices, after system crash or power down, will lose these data structures.Though in fact the data on the solid state hard disc do not lose, because the struct region of these data of management loses, so in fact data have also just been lost.In order to address this problem, the present invention adopts structure as shown in Figure 2.Solid state hard disc is divided into two parts, the struct region data structure in the last part for storage internal memory, and a back part is only the data of real cache.Like this, when after system crash or the power down can through read leave solid state hard disc in before the struct region of a part find and leave latter part of data in, and rebuild whole file system environment, thereby guarantee that data in buffer do not lose.Will any modification to struct region all not be synchronized to solid state hard disc during practical application, expense is bigger like this.Can only when data change, revise struct region again, particularly will write down those and be buffered in solid state hard disc and do not have situation of change in the data of traditional magnetic disk.
Read operation.Find corresponding struct region according to recorded address information in the bio, check through struct region whether metadata cache is in solid state hard disc then, if then directly read from solid state hard disc; Otherwise, read and be cached to the solid state hard disc from disk.
Write operation.Find corresponding struct region according to recorded address information in the bio,, then directly data are write toward solid state hard disc if idle solid state hard disc space is arranged; Otherwise, write toward conventional hard.If the struct region metadata information of writing past solid state hard disc then upgrading correspondence simultaneously is to solid state hard disc.
Recovery operation.Under the situation of power down after the system crash; Behind system failure recovery; The present invention can scan the metadata information that leaves in the solid state hard disc, and recovers all environment before according to these information, and the dirty data that be buffered in the solid state hard disc this moment can intactly be read.

Claims (7)

1. management method of using the high speed non-volatile media to do buffer memory is characterized in that: use solid state hard disc as buffer memory equipment, each buffer memory equipment can be shared by a plurality of disks, and each disk can only use a buffer memory equipment; Said buffer memory equipment is divided the region structure, and the mapping of employing group links to each other between buffer memory equipment and the disk; Buffer memory equipment is divided into two parts, and the region data structure in the previous section stored memory, aft section are buffer memory.
2. the method for claim 1 is characterized in that: said solid state hard disc can be formed the raid disk array and enlarge buffer memory capacity and improve performance.
3. the method for claim 1 is characterized in that: said region size is 64KB~1MB, and is 2 power, and said region can not be modified behind equipment operation.
4. the method for claim 1 is characterized in that: when the mapping of employing group links to each other between said buffer memory equipment and the disk, give tacit consent to every group of 16 region, and can be through grand change.
5. the method for claim 1; It is characterized in that: said buffer memory equipment is when read operation; Find corresponding region data structure according to recorded address information in the bio; Check through the region data structure whether metadata cache is in solid state hard disc then, if then directly read from solid state hard disc; Otherwise, read and be cached to the solid state hard disc from disk.
6. the method for claim 1 is characterized in that: said buffer memory equipment finds corresponding region data structure according to recorded address information in the bio when write operation, if idle solid state hard disc space is arranged, then directly data is write toward solid state hard disc; Otherwise, write toward conventional hard; If the region data structure metadata information of writing past solid state hard disc then upgrading correspondence simultaneously is to solid state hard disc.
7. the method for claim 1; It is characterized in that: said buffer memory equipment is under the situation of power down after the system crash; Behind system failure recovery; Can scanning leave the metadata information in the solid state hard disc in, and recover all environment before according to these information, staying alone is buffered in the dirty data in the solid state hard disc.
CN201110364311.2A 2011-11-17 2011-11-17 Management method by using rapid non-volatile medium as cache Active CN102521147B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201110364311.2A CN102521147B (en) 2011-11-17 2011-11-17 Management method by using rapid non-volatile medium as cache

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201110364311.2A CN102521147B (en) 2011-11-17 2011-11-17 Management method by using rapid non-volatile medium as cache

Publications (2)

Publication Number Publication Date
CN102521147A true CN102521147A (en) 2012-06-27
CN102521147B CN102521147B (en) 2015-03-25

Family

ID=46292077

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201110364311.2A Active CN102521147B (en) 2011-11-17 2011-11-17 Management method by using rapid non-volatile medium as cache

Country Status (1)

Country Link
CN (1) CN102521147B (en)

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102945207A (en) * 2012-10-26 2013-02-27 浪潮(北京)电子信息产业有限公司 Cache management method and system for block-level data
CN103345450A (en) * 2013-06-26 2013-10-09 浙江宇视科技有限公司 Device and method for dynamically regulating redundant RAID array reconstruction
CN103678166A (en) * 2013-08-16 2014-03-26 记忆科技(深圳)有限公司 Method and system for using solid-state disk as cache of computer
CN103744796A (en) * 2013-09-29 2014-04-23 记忆科技(深圳)有限公司 Caching method and system by means of uSSD
CN103744611A (en) * 2013-12-17 2014-04-23 记忆科技(深圳)有限公司 Computer system based on solid state disc as cache and cache accelerating method
CN103827837A (en) * 2012-08-22 2014-05-28 华为技术有限公司 Method, device and system for accessing solid state disk space
CN104050014A (en) * 2014-05-23 2014-09-17 上海爱数软件有限公司 Efficient storage management method based on virtualization platform
CN104793892A (en) * 2014-01-20 2015-07-22 上海优刻得信息科技有限公司 Method for accelerating random in-out (IO) read-write of disk
CN104866438A (en) * 2014-02-20 2015-08-26 联想(北京)有限公司 Storage device, storage device control method and electronic equipment
CN105094711A (en) * 2015-09-22 2015-11-25 浪潮(北京)电子信息产业有限公司 Method and device for achieving copy-on-write file system
CN105302656A (en) * 2015-10-21 2016-02-03 浪潮(北京)电子信息产业有限公司 Data test method and apparatus
CN106443740A (en) * 2016-09-13 2017-02-22 上海顺砾智能科技有限公司 Rapid positioning system and method applied to unmanned aerial vehicle
CN106528001A (en) * 2016-12-05 2017-03-22 北京航空航天大学 Cache system based on nonvolatile memory and software RAID
CN107229573A (en) * 2017-05-22 2017-10-03 上海天玑数据技术有限公司 A kind of elastic high-availability cache method based on solid state hard disc
CN107402724A (en) * 2017-07-31 2017-11-28 郑州云海信息技术有限公司 The store method and system of Journal metadata in a kind of SSD
CN107402819A (en) * 2017-08-04 2017-11-28 郑州云海信息技术有限公司 The management method and system of a kind of client-cache
CN107480074A (en) * 2017-08-31 2017-12-15 郑州云海信息技术有限公司 A kind of caching method, device and electronic equipment
CN107704400A (en) * 2017-10-12 2018-02-16 郑州云海信息技术有限公司 A kind of caching of page method towards non-volatile memory
CN108829835A (en) * 2018-06-19 2018-11-16 深圳市瑞驰信息技术有限公司 A kind of novel high speed cache file system architecture
CN113312300A (en) * 2021-06-17 2021-08-27 上海天玑科技股份有限公司 Nonvolatile memory caching method integrating data transmission and storage
CN115437853A (en) * 2022-11-07 2022-12-06 四川省华存智谷科技有限责任公司 Cache data protection method and system under system abnormal restart

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5860083A (en) * 1996-11-26 1999-01-12 Kabushiki Kaisha Toshiba Data storage system having flash memory and disk drive
CN1465009A (en) * 2000-06-23 2003-12-31 英特尔公司 Non-volatile cache integrated with mass storage device
CN1527973A (en) * 2000-06-23 2004-09-08 英特尔公司 Non-volatile cache
CN201698255U (en) * 2010-05-14 2011-01-05 上海浙大网新易得科技发展有限公司 Server capable of accessing disc at high speed

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5860083A (en) * 1996-11-26 1999-01-12 Kabushiki Kaisha Toshiba Data storage system having flash memory and disk drive
CN1465009A (en) * 2000-06-23 2003-12-31 英特尔公司 Non-volatile cache integrated with mass storage device
CN1527973A (en) * 2000-06-23 2004-09-08 英特尔公司 Non-volatile cache
CN201698255U (en) * 2010-05-14 2011-01-05 上海浙大网新易得科技发展有限公司 Server capable of accessing disc at high speed

Cited By (33)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103827837B (en) * 2012-08-22 2016-08-03 华为技术有限公司 Access the method in solid state hard disc space, Apparatus and system
CN103827837A (en) * 2012-08-22 2014-05-28 华为技术有限公司 Method, device and system for accessing solid state disk space
CN102945207A (en) * 2012-10-26 2013-02-27 浪潮(北京)电子信息产业有限公司 Cache management method and system for block-level data
CN103345450A (en) * 2013-06-26 2013-10-09 浙江宇视科技有限公司 Device and method for dynamically regulating redundant RAID array reconstruction
CN103345450B (en) * 2013-06-26 2016-11-02 浙江宇视科技有限公司 The apparatus and method that a kind of dynamic adjustment redundancy RAID array is rebuild
CN103678166A (en) * 2013-08-16 2014-03-26 记忆科技(深圳)有限公司 Method and system for using solid-state disk as cache of computer
CN103744796A (en) * 2013-09-29 2014-04-23 记忆科技(深圳)有限公司 Caching method and system by means of uSSD
CN103744611A (en) * 2013-12-17 2014-04-23 记忆科技(深圳)有限公司 Computer system based on solid state disc as cache and cache accelerating method
CN104793892B (en) * 2014-01-20 2019-04-19 优刻得科技股份有限公司 A method of accelerating disk stochastic inputs output (IO) read-write
CN104793892A (en) * 2014-01-20 2015-07-22 上海优刻得信息科技有限公司 Method for accelerating random in-out (IO) read-write of disk
CN104866438A (en) * 2014-02-20 2015-08-26 联想(北京)有限公司 Storage device, storage device control method and electronic equipment
CN104050014B (en) * 2014-05-23 2017-12-29 上海爱数信息技术股份有限公司 Efficient storage management method based on virtual platform
CN104050014A (en) * 2014-05-23 2014-09-17 上海爱数软件有限公司 Efficient storage management method based on virtualization platform
CN105094711A (en) * 2015-09-22 2015-11-25 浪潮(北京)电子信息产业有限公司 Method and device for achieving copy-on-write file system
CN105094711B (en) * 2015-09-22 2018-05-18 浪潮(北京)电子信息产业有限公司 A kind of method and device for realizing copy-on-write file system
CN105302656B (en) * 2015-10-21 2018-09-04 浪潮(北京)电子信息产业有限公司 A kind of data test method and device
CN105302656A (en) * 2015-10-21 2016-02-03 浪潮(北京)电子信息产业有限公司 Data test method and apparatus
CN106443740A (en) * 2016-09-13 2017-02-22 上海顺砾智能科技有限公司 Rapid positioning system and method applied to unmanned aerial vehicle
CN106528001A (en) * 2016-12-05 2017-03-22 北京航空航天大学 Cache system based on nonvolatile memory and software RAID
CN106528001B (en) * 2016-12-05 2019-08-23 北京航空航天大学 A kind of caching system based on nonvolatile memory and software RAID
CN107229573A (en) * 2017-05-22 2017-10-03 上海天玑数据技术有限公司 A kind of elastic high-availability cache method based on solid state hard disc
CN107229573B (en) * 2017-05-22 2020-04-28 上海天玑数据技术有限公司 Elastic high-availability caching method based on solid state disk
CN107402724A (en) * 2017-07-31 2017-11-28 郑州云海信息技术有限公司 The store method and system of Journal metadata in a kind of SSD
CN107402724B (en) * 2017-07-31 2020-07-24 苏州浪潮智能科技有限公司 Method and system for storing Journal metadata in SSD
CN107402819A (en) * 2017-08-04 2017-11-28 郑州云海信息技术有限公司 The management method and system of a kind of client-cache
CN107480074A (en) * 2017-08-31 2017-12-15 郑州云海信息技术有限公司 A kind of caching method, device and electronic equipment
CN107480074B (en) * 2017-08-31 2020-02-07 郑州云海信息技术有限公司 Caching method and device and electronic equipment
CN107704400A (en) * 2017-10-12 2018-02-16 郑州云海信息技术有限公司 A kind of caching of page method towards non-volatile memory
CN108829835A (en) * 2018-06-19 2018-11-16 深圳市瑞驰信息技术有限公司 A kind of novel high speed cache file system architecture
CN113312300A (en) * 2021-06-17 2021-08-27 上海天玑科技股份有限公司 Nonvolatile memory caching method integrating data transmission and storage
CN113312300B (en) * 2021-06-17 2024-05-03 上海天玑科技股份有限公司 Nonvolatile memory caching method integrating data transmission and storage
CN115437853A (en) * 2022-11-07 2022-12-06 四川省华存智谷科技有限责任公司 Cache data protection method and system under system abnormal restart
CN115437853B (en) * 2022-11-07 2023-03-10 四川省华存智谷科技有限责任公司 Cache data protection method and system under system abnormal restart

Also Published As

Publication number Publication date
CN102521147B (en) 2015-03-25

Similar Documents

Publication Publication Date Title
CN102521147B (en) Management method by using rapid non-volatile medium as cache
US9940261B2 (en) Zoning of logical to physical data address translation tables with parallelized log list replay
US10289556B2 (en) Techniques to perform power fail-safe caching without atomic metadata
US10521131B2 (en) Storage apparatus and storage control apparatus
CN106528001B (en) A kind of caching system based on nonvolatile memory and software RAID
US10061704B2 (en) Systems and methods for managing cache of a data storage device
US9798623B2 (en) Using cache to manage errors in primary storage
KR20100021868A (en) Buffer cache management method for flash memory device
CN109739696B (en) Double-control storage array solid state disk caching acceleration method
WO2018090255A1 (en) Memory access technique
US11144224B2 (en) Locality-aware, memory-efficient, time-efficient hot data identification using count-min-sketch for flash or streaming applications
US20140237190A1 (en) Memory system and management method therof
CN110502188A (en) A kind of date storage method and device based on data base read-write performance
JP7318367B2 (en) Storage control device and storage control program
CN105786721A (en) Memory address mapping management method and processor
US20210081321A1 (en) Method and apparatus for performing pipeline-based accessing management in a storage server
CN101377788A (en) Method and system of caching management in cluster file system
CN101807212B (en) Caching method for embedded file system and embedded file system
JP4734432B2 (en) Data storage system
CN102362263A (en) SSD controller, and method for operating an SSD controller
CN102160038A (en) Method and an apparatus to manage non-volatile disl cache
Yang et al. A shingle-aware persistent cache management scheme for DM-SMR disks
US8108605B2 (en) Data storage system and cache data—consistency assurance method
CN105608014B (en) A kind of storage device using MRAM
JP5953245B2 (en) Information processing system

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20220725

Address after: 100193 No. 36 Building, No. 8 Hospital, Wangxi Road, Haidian District, Beijing

Patentee after: Dawning Information Industry (Beijing) Co.,Ltd.

Patentee after: DAWNING INFORMATION INDUSTRY Co.,Ltd.

Address before: 100084 Beijing Haidian District City Mill Street No. 64

Patentee before: Dawning Information Industry (Beijing) Co.,Ltd.