CN102637147A - Storage system using solid state disk as computer write cache and corresponding management scheduling method - Google Patents
Storage system using solid state disk as computer write cache and corresponding management scheduling method Download PDFInfo
- Publication number
- CN102637147A CN102637147A CN2011103583535A CN201110358353A CN102637147A CN 102637147 A CN102637147 A CN 102637147A CN 2011103583535 A CN2011103583535 A CN 2011103583535A CN 201110358353 A CN201110358353 A CN 201110358353A CN 102637147 A CN102637147 A CN 102637147A
- Authority
- CN
- China
- Prior art keywords
- solid state
- state hard
- hard disc
- page
- write
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a storage system using a solid state disk as a computer write cache and a corresponding management scheduling method. The storage system comprises one or more solid state disk devices and one or more traditional disk devices, wherein the solid state disk devices are low in capacity and high in access speed, the traditional disk devices are high in capacity and low in access speed, and the devices are connected with a bus of a computer through one interface of standard PCI-E (peripheral component interconnect express), SAS (serial attached SCSI) or SCSI (small computer system interface), and are visible in a computer system.
Description
Technical field
The present invention relates to processing data information, particularly relate to a kind of storage system and corresponding management and dispatching method that utilizes solid state hard disc to write buffer memory as computing machine.
Background technology
Disk seek time has slowly become the bottleneck place in the large data processing application.Compare other processing speed of CPU nanosecond, the seek time of disk Millisecond has had a strong impact on the response time and the throughput of total system.Solid state hard disc has high speed of random access as a kind of novel electricity storage medium, is considered to follow-on mainstream storage device.
Yet we notice; Current solid state hard disc will replace disk fully and remain unpractical; Following 2 points of tracing it to its cause: at first the capacity of solid state hard disc is less; The capacity of the solid state hard disc that equal price can be bought has only more than percent even a few per mille of disk, and the cost that uses solid state hard disc to store all data fully is extremely high; Secondly the storage chip flash memory of solid state hard disc has the read-write asymmetry; The random write of flash memory being carried out small data quantity often need be to carrying out once very slow erase operation in a big way; Therefore although current solid state hard disc has extremely powerful with performance machine-readable and the order read-write; But the relative disk of the performance of its random write does not have remarkable advantages, is the performance bottleneck place of solid state hard disc.
Under this background, how to make good use of solid state hard disc, making it in whole storage architecture, to give full play to its advantage, avoiding its defective is a significant technical matters, is the key point that improves the performance of computer system in large-scale data is used.
Summary of the invention
The object of the invention is to provide a kind of storage system of utilizing solid state hard disc to write buffer memory as computing machine; And base dispatching method on it; Give full play to the solid state hard disc high bandwidth and at a high speed with machine-readable advantage, avoid the inferior position of its low speed random write, increase substantially the response performance of system.
For reaching above-mentioned target, the invention discloses a kind of solid state hard disc that utilizes and write the storage system of buffer memory as computing machine, comprising:
One or more low capacity, access speed faster solid state hard disc equipment and
One or more high capacity, the slower traditional magnetic disk equipment of access speed,
The said equipment links to each other with the bus of said computing machine through one of Standard PC I-E, SAS or scsi interface, and visible to said computer system.
Above-mentioned storage system; Its characteristic also is: the primary data of said computing machine all is stored on the above-mentioned disk; Wherein said solid state hard disc is in system's dynamic buffering computing machine in service recent renewal data; Under original state, the amount of capacity of said solid state hard disc is to the not influence of correctness of system simultaneously.
Above-mentioned storage system, its characteristic also is: in operational process, can irregularly dynamically increase the number of solid state hard disc at said system for computer, improve the performance of system; But existing said solid state hard disc can not dynamically be removed.
In addition, the invention also discloses a kind of method that is used for above-mentioned storage system is carried out management and dispatching, said method comprises:
All solid state hard discs are formatted as paging structure; The length of each page is identical with the interior external memory exchange of the said computer system page; The page on whole said solid state hard discs is taken as the formation of an integral body and carries out the sequential loop use; When the dirty page is discharged from internal memory, at first it is write back last page of said formation, rather than directly write back said solid magnetic disc;
In internal memory, safeguard the address mapping table of a solid state hard disc; After the page writes back said solid state hard disc; Its page number, solid state hard disc skew are used as a doublet index entry; And maintain in the said address mapping table, and, then have only last index entry effective if a page is write back repeatedly;
When the outer deposit data of needs visits, check in said address mapping table at first whether the index entry that comprises this page number exists, if the skew of the solid state hard disc in the respective index item is then visited in existence,, then visit the address of disk if do not exist.
Above-mentioned management and dispatching method also comprises step: because equipment is visible to computer system, therefore said dispatching method can be realized in different levels such as operating system, file system, database and concrete application programs by requirement in practical systems.
Above-mentioned management and dispatching method also comprises: when said The whole calculations machine system collapse takes place and restarts, need at first scan said solid state hard disc, and rebuild address mapping table according to the said page of all buffer memorys wherein, carry out the normal operation of system again.
The invention solves the less but performance of the capacity that how to make full use of faster solid state hard disc come the problem of optimizing computer storage system responding ability, the beneficial effect that has is:
1) eliminated in the original computer system random write operation, reduced the tracking pressure of disk effectively, converted random write into sequential write, improved system performance disk.
2) utilize solid state hard disc to carry out buffer memory to the data of recent renewal, the high speed of having given full play to solid state hard disc is with machine-readable ability, reduced in the original computer system the read operation at random of disk, further improved system performance.
3) operation of solid state hard disc is had only the order read-write and with machine-readable, avoided the more weak performance bottleneck that brings of solid state hard disc random write.
4) basis of the system and method among the present invention hypothesis be standard based on the computer organization of interior external memory and the foundation characteristic of solid state hard disc; Do not rely on any concrete solid state hard disc model, computer instruction system, operating system etc.; Also can have sufficient portability in flexible realizations at all levels such as operating system, file system, database and concrete application programs.
Description of drawings
Fig. 1 is the entire system frame diagram.
Fig. 2 is the address mapping table structural representation.
Fig. 3 is the workflow diagram of the dirty page write-back of system.
Fig. 4 is the workflow diagram of system's accession page.
Fig. 5 is the workflow diagram of system's backstage write-back thread.
Embodiment
Below in conjunction with accompanying drawing and embodiment the present invention is done further description:
As shown in Figure 1, provided the structural representation of whole storage system.Its memory device comprises one or more solid state hard disc equipment, and one or more traditional magnetic disk equipment, and all equipment links to each other with computer bus through standard interfaces such as PCI-E, SATA or SCSI, and the concrete quantity of equipment is looked the extensibility decision of bus.
Storage system is externally transparent fully; All memory devices and the device type that belongs to thereof all are formatted as paging structure; And can be by computer Recognition; As the basic cross-over unit between internal memory and the external memory, the length of the page is 2 power number of times kilobyte to computing machine with the page of regular length, generally at 1KB between the 32KB.The content of pages that computing machine can begin to a certain skew of arbitrary storage device requests reads the page in the internal memory, and visit data wherein.
The page on all solid state hard discs is organized as a round-robin queue structure, and the head and the tail of formation equate under the original state, writes fashionablely as the page, and rear of queue moves backward, when crossing the border, returns the formation head.When rear of queue during, explain that then the capacity of all solid state hard discs soon is used up near queue heads.
Storage system is when initialization, and its memory device can include only disk, and raw data is loaded in the middle of the disk.After system brings into operation; Can in system, increase one or more solid state hard disc equipment aperiodically; These solid state hard disc equipment can increase under computer operation condition, also can stop the operation of computing machine earlier, increase solid state hard disc equipment and start computing machine once more; Its process is looked the particular type of equipment and is decided, and the solid state hard disc of increase inserts in the formation after being slit into sheets too again.All existing solid state hard disc equipment can not be removed after operation again.
Simultaneity factor need be opened up certain zone in internal memory, safeguard an address mapping table, supplies management and dispatching to use.
As shown in Figure 2, provided the structural representation of address mapping table.Address mapping table is the structure of Hash table, and its hash index key assignments is a page number, and its node is the doublet index entry of (page number, solid state hard disc skew).In the index entry, page number is meant the physical label of the page, and solid state hard disc skew is meant and the skew of this page in solid state hard disc can navigates to the position of the corresponding page in solid state hard disc rapidly through this side-play amount.All index entries calculate cryptographic hash with page number, and the index entry that belongs to same hash is maintained in together with the form of individual event chained list.
As shown in Figure 3, this figure is the workflow of the single dirty page write-back of the present invention's proposition, and this flow process is carried out by single worker thread, and it is described in detail as follows:
301 write back to the page in the page of solid state hard disc rear of queue, and this moves sequential write solid state hard disc all the time, the high bandwidth of utilizing solid state hard disc that can be good.Under the working environment of multi-thread concurrent, this step needs the critical section protection for the visit and the modification of rear of queue position.
Check that whether distance between rear of queue and the queue heads is less than the write-back threshold value 302 this moments.The write-back threshold value generally was greater than in the time of write-back thread one action, the number of pages of other worker thread institute write-back, otherwise before the write-back thread work was accomplished, the write-back request of other thread need get clogged.
If the distance of being checked in 303 302 is less than the write-back threshold value, this moment, the write-back thread was waken up, and the dirty page that is stored in the formation is write back hard disk.It should be noted that in the process of write-back thread write-back, do not hinder worker thread and continue in solid state hard disc, to write back data.
Whether 304 these pages of inspection have had index entry in address mapping table, this process at first calculates cryptographic hash through page number, carry out searching of single-track link table by corresponding hash index inlet again.Because page number is a round values very clocklike, therefore good hash function can be searched cost with this and drops near 0 (1) complexity.
If 305 have found corresponding index entry, then upgrade the respective offsets value in the index entry.
If 306 do not find corresponding index entry, then generate an index entry of inciting somebody to action, and (page number, off-set value) write.
As shown in Figure 4, this figure is the workflow that reads the page that the present invention proposes, and this flow process is carried out by single worker thread, and it is described in detail as follows:
401 check at first whether this page has index entry in address mapping table, and this process also is to calculate cryptographic hash through page number earlier, carry out searching of single-track link table by corresponding hash index inlet again.
If 402 index entries exist, the latest edition that this page then is described is in solid state hard disc, and read data in solid state hard disc according to the skew of index entry this moment.
If 403 index entries do not exist, explain that then this page is not written back to mistake in the solid state hard disc, this moment is according to page number degaussing dish reading of data.
As shown in Figure 5, this figure is the workflow diagram of write-back thread proposed by the invention, and this flow process realizes that by backstage write-back thread it is described in detail as follows:
501 obtain the position of current solid state hard disc rear of queue, and this action need be accomplished under the protection of critical section.
502 sort index entries all in the mapping table of current address by page number, because the length of index entry is very little, so this action can be carried out in internal memory.
Index entry after the 503 inspection orderings, if the page number of two index entries is continuous, promptly the position of their raw data in disk is continuous, then merges into a longer page to them.
504 index entries by ordering carry out write-back, and its concrete steps are for reading an index entry, and from solid state hard disc, read the data that deviation post belongs to, and are written back on the respective page of disk again.Can see that here the access sequence to solid state hard disc is with machine-readable, and be sequential write, all meet their performance characteristic the access characteristic of disk.
The index entry that write-back is crossed picked-off from address mapping table.
505 last formation head with current queue are updated to the former rear of queue that in 501 steps, is obtained, and this action is also carried out under the protection of critical section.Owing in the write-back process, still have worker thread can in formation, write the page of some up-to-date write-backs, so the index entry information of these pages still need be retained.
Though accompanying drawing and above stated specification have provided embodiments of the invention.But it is understandable that, it will be appreciated by those skilled in the art that and can the one or more assemblies in this assembly be combined into the individual feature assembly well.In alternative, specific assembly can be divided into a plurality of functional modules, otherwise or.Simultaneously, scope of the present invention does not receive the restriction of these particular instances.Multiple variation all is possible, the difference on structure etc. for example, and no matter whether it is clearly provided in instructions.Scope of the present invention is the same wide with the scope that accompanying claims provides at least.
Claims (6)
1. one kind is utilized solid state hard disc to write the storage system of buffer memory as computing machine, comprising:
One or more low capacity, access speed faster solid state hard disc equipment and
One or more high capacity, the slower traditional magnetic disk equipment of access speed,
The said equipment links to each other with the bus of said computing machine through one of Standard PC I-E, SAS or scsi interface, and visible to said computer system.
2. storage system according to claim 1 is characterized in that:
The primary data of said computing machine all is stored on the above-mentioned disk, and wherein said solid state hard disc is in system's dynamic buffering computing machine in service recent renewal data, and under original state, the amount of capacity of said solid state hard disc is to the not influence of correctness of system simultaneously.
3. storage system according to claim 1 is characterized in that:
In operational process, can irregularly dynamically increase the number of solid state hard disc at said system for computer, improve the performance of system; But existing said solid state hard disc can not dynamically be removed.
4. one kind is carried out the method for management and dispatching among the claim 1-3 any one, it is characterized in that:
All solid state hard discs are formatted as paging structure; The length of each page is identical with the interior external memory exchange of the said computer system page; The page on whole said solid state hard discs is taken as the formation of an integral body and carries out the sequential loop use; When the dirty page is discharged from internal memory, at first it is write back last page of said formation, rather than directly write back said solid magnetic disc;
In internal memory, safeguard the address mapping table of a solid state hard disc; After the page writes back said solid state hard disc; Its page number, solid state hard disc skew are used as a doublet index entry; And maintain in the said address mapping table, and, then have only last index entry effective if a page is write back repeatedly;
When the outer deposit data of needs visits, check in said address mapping table at first whether the index entry that comprises this page number exists, if the skew of the solid state hard disc in the respective index item is then visited in existence,, then visit the address of disk if do not exist;
And when the active volume of all solid state hard disc equipment all is lower than preset threshold value, by the write-back thread on backstage the index entry in the said address mapping table is reset by page number, the adjacent page is merged, and they are write back in the disk in order.
5. management and dispatching method according to claim 4 is characterized in that:
Because any related equipment is visible to computer system among the claim 1-3, therefore said dispatching method can be realized in different levels such as operating system, file system, database and concrete application programs by requirement in practical systems.
6. according to 4 or 5 management and dispatching method, it is characterized in that:
When collapse takes place and restarts in said The whole calculations machine system, need at first scan said solid state hard disc, and rebuild address mapping table according to the said page of all buffer memorys wherein, carry out the normal operation of system again.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011103583535A CN102637147A (en) | 2011-11-14 | 2011-11-14 | Storage system using solid state disk as computer write cache and corresponding management scheduling method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011103583535A CN102637147A (en) | 2011-11-14 | 2011-11-14 | Storage system using solid state disk as computer write cache and corresponding management scheduling method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102637147A true CN102637147A (en) | 2012-08-15 |
Family
ID=46621548
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011103583535A Pending CN102637147A (en) | 2011-11-14 | 2011-11-14 | Storage system using solid state disk as computer write cache and corresponding management scheduling method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102637147A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103257935A (en) * | 2013-04-19 | 2013-08-21 | 华中科技大学 | Cache management method and application thereof |
| CN103279561A (en) * | 2013-06-13 | 2013-09-04 | 三珠数码软件开发（上海）有限公司 | Method for increasing random database data read-write speed |
| CN103678166A (en) * | 2013-08-16 | 2014-03-26 | 记忆科技(深圳)有限公司 | Method and system for using solid-state disk as cache of computer |
| CN104298620A (en) * | 2014-10-10 | 2015-01-21 | 张维加 | Erasable-resistant low-energy consumption external computer accelerating equipment |
| CN107145449A (en) * | 2016-03-01 | 2017-09-08 | 日本电气株式会社 | Storage device and storage method |
| WO2018028529A1 (en) * | 2016-08-08 | 2018-02-15 | 北京忆恒创源科技有限公司 | Lock-free io processing method and apparatus therefor |
| CN107832013A (en) * | 2017-11-03 | 2018-03-23 | 中国科学技术大学 | A kind of method for managing solid-state hard disc mapping table |
| CN108228482A (en) * | 2016-12-21 | 2018-06-29 | 伊姆西Ip控股有限责任公司 | For managing the method and system of the buffer memory device in storage system |
| CN108664211A (en) * | 2017-03-31 | 2018-10-16 | 深圳市中兴微电子技术有限公司 | A kind of method and device for realizing reading and writing data |
| CN109213420A (en) * | 2017-06-29 | 2019-01-15 | 杭州海康威视数字技术股份有限公司 | Date storage method, apparatus and system |
| CN109446113A (en) * | 2018-11-10 | 2019-03-08 | 苏州韦科韬信息技术有限公司 | A kind of optimization solid state hard disk buffer memory management method |
| CN109918234A (en) * | 2019-03-06 | 2019-06-21 | 苏州浪潮智能科技有限公司 | A kind of metadata restoration methods, device, equipment and medium based on SSD |
| CN111143236A (en) * | 2019-12-07 | 2020-05-12 | 杭州安恒信息技术股份有限公司 | Memory mapping implementation queue and data reading and writing method thereof |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101266573A (en) * | 2008-04-29 | 2008-09-17 | 中国船舶重工集团公司第七〇九研究所 | Covering allowable flash memory even wearing circulating queue technology |
-
2011
- 2011-11-14 CN CN2011103583535A patent/CN102637147A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101266573A (en) * | 2008-04-29 | 2008-09-17 | 中国船舶重工集团公司第七〇九研究所 | Covering allowable flash memory even wearing circulating queue technology |
Non-Patent Citations (2)
| Title |
|---|
| 傅家祥: "《操作系统》", 31 July 2004, 重庆大学出版社 * |
| 徐昶: "基于闪存的数据库存储引擎技术研究", 《万方学位论文数据库》 * |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103257935A (en) * | 2013-04-19 | 2013-08-21 | 华中科技大学 | Cache management method and application thereof |
| CN103257935B (en) * | 2013-04-19 | 2016-07-13 | 华中科技大学 | A kind of buffer memory management method and application thereof |
| CN103279561A (en) * | 2013-06-13 | 2013-09-04 | 三珠数码软件开发（上海）有限公司 | Method for increasing random database data read-write speed |
| CN103678166A (en) * | 2013-08-16 | 2014-03-26 | 记忆科技(深圳)有限公司 | Method and system for using solid-state disk as cache of computer |
| CN104298620A (en) * | 2014-10-10 | 2015-01-21 | 张维加 | Erasable-resistant low-energy consumption external computer accelerating equipment |
| CN107145449A (en) * | 2016-03-01 | 2017-09-08 | 日本电气株式会社 | Storage device and storage method |
| WO2018028529A1 (en) * | 2016-08-08 | 2018-02-15 | 北京忆恒创源科技有限公司 | Lock-free io processing method and apparatus therefor |
| CN107704194A (en) * | 2016-08-08 | 2018-02-16 | 北京忆恒创源科技有限公司 | Without lock I O process method and its device |
| CN108228482B (en) * | 2016-12-21 | 2021-11-05 | 伊姆西Ip控股有限责任公司 | Method and system for managing cache devices in a storage system |
| CN108228482A (en) * | 2016-12-21 | 2018-06-29 | 伊姆西Ip控股有限责任公司 | For managing the method and system of the buffer memory device in storage system |
| US11403224B2 (en) | 2016-12-21 | 2022-08-02 | EMC IP Holding Company, LLC | Method and system for managing buffer device in storage system |
| CN108664211A (en) * | 2017-03-31 | 2018-10-16 | 深圳市中兴微电子技术有限公司 | A kind of method and device for realizing reading and writing data |
| CN109213420A (en) * | 2017-06-29 | 2019-01-15 | 杭州海康威视数字技术股份有限公司 | Date storage method, apparatus and system |
| CN107832013A (en) * | 2017-11-03 | 2018-03-23 | 中国科学技术大学 | A kind of method for managing solid-state hard disc mapping table |
| CN107832013B (en) * | 2017-11-03 | 2019-10-25 | 中国科学技术大学 | A method of management solid-state hard disc mapping table |
| CN109446113A (en) * | 2018-11-10 | 2019-03-08 | 苏州韦科韬信息技术有限公司 | A kind of optimization solid state hard disk buffer memory management method |
| CN109918234B (en) * | 2019-03-06 | 2020-07-07 | 苏州浪潮智能科技有限公司 | Metadata recovery method, device, equipment and medium based on SSD |
| CN109918234A (en) * | 2019-03-06 | 2019-06-21 | 苏州浪潮智能科技有限公司 | A kind of metadata restoration methods, device, equipment and medium based on SSD |
| CN111143236A (en) * | 2019-12-07 | 2020-05-12 | 杭州安恒信息技术股份有限公司 | Memory mapping implementation queue and data reading and writing method thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102637147A (en) | Storage system using solid state disk as computer write cache and corresponding management scheduling method | |
| CN102012791B (en) | Flash based PCIE (peripheral component interface express) board for data storage | |
| CN101236530B (en) | High speed cache replacement policy dynamic selection method | |
| Caulfield et al. | Gordon: using flash memory to build fast, power-efficient clusters for data-intensive applications | |
| CN101907978B (en) | Mixed storage system and storage method based on solid state disk and magnetic hard disk | |
| CN106066890B (en) | Distributed high-performance database all-in-one machine system | |
| CN106445405B (en) | Data access method and device for flash memory storage | |
| US20100235568A1 (en) | Storage device using non-volatile memory | |
| CN105630700B (en) | A kind of storage system and reading/writing method with secondary cache structure | |
| WO2010066098A1 (en) | Method and device for constructing high speed solid state storage disk with larger capacity dram involved in management of flash media | |
| CN103838676B (en) | Data-storage system, date storage method and PCM bridges | |
| US20180107601A1 (en) | Cache architecture and algorithms for hybrid object storage devices | |
| JP2013156977A (en) | Elastic cache of redundant cache data | |
| CN103530237A (en) | Solid-state disc array garbage collecting method | |
| CN103076993A (en) | Storage system and method for concentration type system | |
| US9507534B2 (en) | Home agent multi-level NVM memory architecture | |
| CN106469123A (en) | A kind of write buffer distribution based on NVDIMM, method for releasing and its device | |
| CN101414244A (en) | A kind of methods, devices and systems of processing data under network environment | |
| Huang et al. | SSDs Striking Back: The Storage Jungle and Its Implications to Persistent Indexes. | |
| JPH11288387A (en) | Disk cache device | |
| Xie et al. | MICRO: A multilevel caching-based reconstruction optimization for mobile storage systems | |
| CN202795333U (en) | Magnetic disk redundancy array high-speed read-write control circuit structure in server | |
| CN102262511B (en) | Cache management system and method for RAID (Redundant Array of Independent Disks) | |
| Zhizhuo et al. | An energy-efficient storage for video surveillance | |
| CN105608014B (en) | A kind of storage device using MRAM |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| DD01 | Delivery of document by public notice |
Addressee: Tianjin Shenzhou General Data Co., Ltd. Document name: Notification that Application Deemed to be Withdrawn |
|
| C05 | Deemed withdrawal (patent law before 1993) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20120815 |