CN101907978B - Mixed storage system and storage method based on solid state disk and magnetic hard disk - Google Patents
Mixed storage system and storage method based on solid state disk and magnetic hard disk Download PDFInfo
- Publication number
- CN101907978B CN101907978B CN2010102379368A CN201010237936A CN101907978B CN 101907978 B CN101907978 B CN 101907978B CN 2010102379368 A CN2010102379368 A CN 2010102379368A CN 201010237936 A CN201010237936 A CN 201010237936A CN 101907978 B CN101907978 B CN 101907978B
- Authority
- CN
- China
- Prior art keywords
- solid state
- page
- disk
- hard disc
- magnetic hard
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Abstract
The invention discloses a mixed storage system and a storage method based on solid state disks and magnetic hard disks. The mixed storage system is characterized by comprising an array consisting of one or a plurality of magnetic hard disks, an array consisting of one or a plurality of solid hard disks and a storage controller, wherein the magnetic hard disk array and the solid state disk array are connected with a computer bus through the storage controller. The storage controller maintains a mapping table from logic pages to physical pages, and is in charge of counting access modes of the logic pages at the same time. According to the performance characteristics of the solid state disks and the magnetic hard disks, the pages are optimally distributed so that hotspot pages which is fewer and is random accessed more are distributed to the solid state disk array, and most common pages with fewer access or mainly based on sequential access are distributed in the magnetic hard disk array, thereby fully exerting the performance superiority of high speed random access of the solid state disks, improving the access performance of the whole system and shortening the time of searching and responding.
Description
Technical field
The present invention relates to processing data information, particularly a kind of mixing storage system and storage means based on solid state hard disc and magnetic hard-disk.
Background technology
Along with developing rapidly of flash memory technology, solid state hard disc has progressed into market, becomes the permanence storage equipment of new generation that may replace the traditional magnetic hard disk.On performance, there is not seek time in solid state hard disc owing to be electrical devices, compares the random access performance of traditional magnetic disk high tens to hundred times thereby can provide.Another key property of solid state hard disc is the read-write asymmetry, compares and reads, and it is expensive many that the write operation cost of solid state hard disc is wanted, and this not only is embodied on the performance, is also embodied in to write number of times flash chip is scrapped, and shortens on the life-span of solid state hard disc.
Notice that simultaneously although the manufacturing process progress is rapid, solid state hard disc still can't be competed with traditional magnetic disk on capacity and price in a short time, in the mass data storage process field, replacing disk fully with solid state hard disc still has very big difficulty.Therefore, relatively rational solution is the mixing storage system that makes up a kind of solid state hard disc and magnetic hard-disk, with most of deposit data on the disk and the small amount of thermal point data is left on the solid state hard disc, thereby improve the overall performance of system, shorten the average response time of inquiry.
Yet, simply solid state hard disc and magnetic hard-disk are disposed on computers, make it to be difficult to satisfy protean practical application to using visible solution.This be because: 1) Xian Dai database software is bigger to the distribution granularity of data, often tables with one or a subregion is a unit.And in actual applications, be not that all row or the row of a table all have identical visiting frequency.2) logical implication of Ying Yonging not is what fix, along with continuous renewal, and the hot spot data no longer focus that may become, and some new data may become hot spot data.Because the capacity of solid state hard disc is less and cost an arm and a leg, therefore how can the most frequent Data Dynamic of visit can be deployed on the solid state hard disc on ground to the greatest extent, be the key issue of elevator system performance.
Summary of the invention
The object of the invention is to provide the mixing storage system of a kind of solid state hard disc and magnetic hard-disk, and dynamically adjusts the method for page physical distribution according to using ruuning situation in this system.
For achieving the above object, the technical solution used in the present invention is as follows:
One, a kind of mixing storage system based on solid state hard disc and magnetic hard-disk:
Comprise the solid state hard disc array of one or more solid state hard disc composition, magnetic hard-disk array and memory controller that one or more magnetic hard-disk is formed; The solid state hard disc array links to each other with computer system bus by memory controller with the magnetic hard-disk array; Described mixing storage system is presented as a logical device to operating system, and its capacity is described two array capacity sums; The visit that mixes storage system is undertaken by the location of getting to logical page (LPAGE), and the size of logical page (LPAGE) is confirmed as the integral multiple of the sector of single 512 bytes when system initialization.
Described solid state hard disc array and magnetic hard-disk array freely customize the redundant array type that becomes from RAID0 to RAID6 as required, thereby self guarantees data fault-tolerant ability and reading performance.
Two, a kind of mixing storage means based on solid state hard disc and magnetic hard-disk:
In memory controller, safeguard the mapping table from the logical page (LPAGE) to the physical page, single logical page (LPAGE) is mapped on the physical page of solid state hard disc or magnetic hard-disk; The request of access of logical page (LPAGE) is converted into request of access to physical page by mapping table;
Memory controller is eigenstate of each logical page (LPAGE) record, when logical page (LPAGE) is accessed, memory controller is revised the eigenstate of the page according to the access module of current accessed request, and transfer on the solid state hard disc array according to the statistical information page that random access is more, the page that sequential access is more is transferred on the magnetic hard-disk array, to improve the response efficiency of system.
Described access module comprises four kinds: sequential read, sequential write, with machine-readable and write at random, the eigenstate of the page is determined that by four kinds of access module sums on behalf of this page, the eigenstate of the page leave in and compared the performance boost of leaving the magnetic hard-disk upper erengy band on the solid state hard disc.
Described statistical information is maintained in the memory controller, and dynamically revise two threshold values according to statistical information, represent the storage characteristic threshold value of magnetic hard-disk and solid state hard disc respectively, in system's operational process, when the eigenstate of a certain page surpasses the storage characteristic threshold value of solid state hard disc, this page is replaced in the solid state hard disc, and memory controller upgrades the map information of this page in the mapping table.
The value of described four kinds of access modules is determined according to performance characteristics, price and three factors of capacity of solid state hard disc array that storage system was equipped with and magnetic hard-disk array.
The invention solves in the commingled system of solid state hard disc and magnetic hard-disk and optimize DATA DISTRIBUTION,, have following beneficial effect to adapt to the problem of complicated applications:
1) mixing storage system adopts capacity bigger, the magnetic hard-disk that price is more cheap is stored most of data, adopt the solid state hard disc of random access performance excellence to store the focus visit data simultaneously, thereby bring into play the performance advantage of solid state hard disc as much as possible, under the prerequisite of lower cost, improve the performance of magnanimity scale data query.
2) mixing that storage system adopted with the page is the self-organization method of thread, thus avoided the granularity that is difficult to avoid by the administrator hand distributed data excessive, be difficult to difficulty such as adjusting.Storage system externally shows as a logical device, has simplified the design of database, has alleviated system designer and keeper's burden.
3) statistical information of system's dynamic adjustments self in operational process, the eigenstate of each page, and the physical distribution of the dynamic optimization page are to improve the access performance of system.Can satisfy various types of application demands, and when application changes, adapt to automatically.
Description of drawings
Fig. 1 is the entire system frame diagram.
Fig. 2 is the memory controller structural representation.
Fig. 3 is system's memory access process flow diagram.
Embodiment
The present invention is described in further detail below in conjunction with accompanying drawing:
As shown in Figure 1, the present invention provides the general frame figure that mixes storage system.Comprise the solid state hard disc array of one or more solid state hard disc composition, magnetic hard-disk array and memory controller that one or more magnetic hard-disk is formed; The solid state hard disc array links to each other with computer system bus by memory controller with the magnetic hard-disk array; Described mixing storage system is presented as a logical device to operating system, and its capacity is described two array capacity sums; The visit that mixes storage system is undertaken by the location of getting to logical page (LPAGE), and the size of logical page (LPAGE) is confirmed as the integral multiple of the sector of single 512 bytes when system initialization, generally at 2KB between the 32KB.
Described solid state hard disc array and magnetic hard-disk array freely customize the redundant array type that becomes from RAID0 to RAID6 as required, thereby self guarantees data fault-tolerant ability and reading performance.
As shown in Figure 2, this figure is the memory controller structural representation of most critical in the storage system of the present invention.Nucleus module in the memory controller is a page-map table, has safeguarded the mapping relations from the logical page (LPAGE) to the physical page.The hash index that also comprises logical page (LPAGE), and the statistical information of total system.
Mapping table is physically according to solid state hard disc and magnetic hard-disk connective tissue, and write down current the logical page number on the physical location in each cell in the table.
The hash index of logical page (LPAGE) is used to quicken searching from the logical page (LPAGE) to the physical page.Each Hash inlet is represented a value of hash function, and the cell that belongs to the logical page (LPAGE) place of this cryptographic hash is gone here and there together with single-track link table.
The statistical information of system mainly comprises the histogram of logical page (LPAGE) eigenstate.Be conserve storage, the page of a histogram some that the statistical nature state is the highest, this number is determined by the capacity of solid state hard disc.Suppose that solid state hard disc has N physical page, then the value of the logical page (LPAGE) number of statistics with histogram is 2 * N, and the page in the histogram is divided into a plurality of wide group.
The statistical information of system also comprises: the solid state hard disc characteristic threshold value of system and magnetic hard-disk characteristic threshold value.Wherein, the magnetic hard-disk characteristic threshold value is histogrammic eigenstate lower bound, and the solid state hard disc characteristic threshold value is the histogram feature state value of the most preceding k * N, and the k here is the characteristic weighing coefficient from 0 to 1, is customized when system initialization.When the eigenstate of a logical page (LPAGE) rises to the magnetic hard-disk threshold value when above, it need be added in the histogram; When the eigenstate of a logical page (LPAGE) rose on the solid state hard disc characteristic threshold value, it must be stored on the solid state hard disc.A certain size difference is arranged between two threshold values, play the effect of buffering.
As shown in Figure 3, this figure is the self-optimizing storage means based on above-mentioned mixing storage system that the present invention proposes, and may further comprise the steps:
301 when accessing page request arrives, and at first carries out Hash calculation according to logical page number, searches from corresponding index entry, navigates to the physical location of this page, according to this content of pages of request of access read/write.
302 according to access module correction page eigenstate.Access module is divided into 4 kinds, is described in detail as follows:
Sequential read: the current logical page (LPAGE) that reads is immediately following the logical page (LPAGE) that read last time;
With machine-readable: the current logical page (LPAGE) that reads is not immediately following the logical page (LPAGE) that read last time;
The eigenstate of the page is a page access pattern sum.
The value of each access module is solid state hard disc cost poor of the magnetic hard-disk cost of this pattern and this pattern.
Above-mentioned cost considers that by the average response time of visiting respective stored separately, memory capacity and three aspects of storage price cost is directly proportional with the average response time of this storage of visit, is inversely proportional to memory capacity, is directly proportional with the storage price.For different solid state hard disc products, it is bigger that the cost result calculated may differ, but the common feature random read-write of solid state hard disc is fast more than magnetic hard-disk, and order read-write is very little even do not have to the advantage of magnetic hard-disk.To some expensive solid state hard disc, also need to consider the influence of write operation to the solid state hard disc life-span.As can be seen, the value of access module has reflected with the advantage of this mode access solid state hard disc to magnetic hard-disk.
Along with the long-play of system, the eigenstate of the page has reflected which kind of medium this page is fit to be stored in, judges by safeguarding statistical information.
303 judge whether to need to upgrade the statistical information histogram.When the change of the eigenstate of the page has been crossed over the group border in the histogram or has been crossed over the characteristic threshold value of magnetic hard-disk:
304 upgrade the information of this page in histogram.
305 judge whether to need to adjust the memory location of the page.Have only when original eigenstate of the page solid state hard disc characteristic threshold value, and current eigenstate is during greater than the solid state hard disc characteristic threshold value of system, thinks that just needs transfer to this page in the solid state hard disc less than system.
306 transfer to this page in the solid state hard disc, according to histogram information, substitute the page that the original characteristics state is less.Upgrade the information in the mapping table, keep the consistance of index information.
The threshold value of 307 update system.In order to reduce the expense of renewal, after histogram upgrades the certain number of times of accumulation, just carry out the renewal of threshold value.
Claims (6)
1. the mixing storage system based on solid state hard disc and magnetic hard-disk is characterized in that: comprise the solid state hard disc array of one or more solid state hard disc composition, magnetic hard-disk array and memory controller that one or more magnetic hard-disk is formed; The solid state hard disc array links to each other with computer system bus by memory controller with the magnetic hard-disk array; Described mixing storage system is presented as a logical device to operating system, and its capacity is solid state hard disc array and magnetic hard-disk array capacity sum; The visit that mixes storage system is undertaken by the location of getting to logical page (LPAGE), and the size of logical page (LPAGE) is confirmed as the integral multiple of the sector of single 512 bytes when system initialization.
2. a kind of mixing storage system according to claim 1 based on solid state hard disc and magnetic hard-disk, it is characterized in that: described solid state hard disc array and magnetic hard-disk array, freely customize the redundant array type that becomes from RAID0 to RAID6 as required, thereby self guarantees data fault-tolerant ability and reading performance.
3. according to a kind of mixing storage means of the described mixing storage system of claim 1, it is characterized in that based on solid state hard disc and magnetic hard-disk:
In memory controller, safeguard the mapping table from the logical page (LPAGE) to the physical page, single logical page (LPAGE) is mapped on the physical page of solid state hard disc or magnetic hard-disk; The request of access of logical page (LPAGE) is converted into request of access to physical page by mapping table;
Memory controller is eigenstate of each logical page (LPAGE) record, when logical page (LPAGE) is accessed, memory controller is revised the eigenstate of the page according to the access module of current accessed request, and transfer on the solid state hard disc array according to the statistical information page that random access is more, the page that sequential access is more is transferred on the magnetic hard-disk array, to improve the response efficiency of system.
4. a kind of mixing storage means according to claim 3 based on solid state hard disc and magnetic hard-disk, it is characterized in that: described access module comprises four kinds: sequential read, sequential write, with machine-readable and write at random, the eigenstate of the page is determined that by four kinds of access module sums on behalf of this page, the eigenstate of the page leave in and compared the performance boost of leaving the magnetic hard-disk upper erengy band on the solid state hard disc.
5. a kind of mixing storage means according to claim 3 based on solid state hard disc and magnetic hard-disk, it is characterized in that: described statistical information is maintained in the memory controller, and dynamically revise two threshold values according to statistical information, represent the storage characteristic threshold value of magnetic hard-disk and solid state hard disc respectively, in system's operational process, when the eigenstate of a certain page surpasses the storage characteristic threshold value of solid state hard disc, this page is replaced in the solid state hard disc, and memory controller upgrades the map information of this page in the mapping table.
6. according to claim 3 or 4 described a kind of mixing storage meanss based on solid state hard disc and magnetic hard-disk, it is characterized in that: the value of described four kinds of access modules is determined according to performance characteristics, price and three factors of capacity of solid state hard disc array that storage system was equipped with and magnetic hard-disk array.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010102379368A CN101907978B (en) | 2010-07-27 | 2010-07-27 | Mixed storage system and storage method based on solid state disk and magnetic hard disk |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010102379368A CN101907978B (en) | 2010-07-27 | 2010-07-27 | Mixed storage system and storage method based on solid state disk and magnetic hard disk |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101907978A CN101907978A (en) | 2010-12-08 |
| CN101907978B true CN101907978B (en) | 2011-12-07 |
Family
ID=43263448
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010102379368A Expired - Fee Related CN101907978B (en) | 2010-07-27 | 2010-07-27 | Mixed storage system and storage method based on solid state disk and magnetic hard disk |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101907978B (en) |
Families Citing this family (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102043689B (en) * | 2010-12-28 | 2012-11-07 | 武汉固捷联讯科技有限公司 | Fault tolerance design method for solid-state memory device |
| WO2011147187A1 (en) * | 2010-12-31 | 2011-12-01 | 华为技术有限公司 | Writing method for solid-state drive in multi-level cache hierarchy and device thereof |
| US9189405B2 (en) * | 2011-08-03 | 2015-11-17 | International Business Machines Corporation | Placement of data in shards on a storage device |
| WO2012119449A1 (en) | 2011-09-30 | 2012-09-13 | 华为技术有限公司 | Method and system for configuring storage devices under hybrid storage environment |
| CN102541466A (en) * | 2011-10-27 | 2012-07-04 | 忆正存储技术（武汉）有限公司 | Hybrid storage control system and method |
| CN102520885B (en) * | 2011-12-15 | 2015-04-15 | 清华大学 | Data management system for hybrid hard disk |
| CN102609360B (en) * | 2012-01-12 | 2015-03-25 | 华为技术有限公司 | Data processing method, data processing device and data processing system |
| KR102072449B1 (en) * | 2012-06-01 | 2020-02-04 | 삼성전자주식회사 | Storage device including non-volatile memory device and repair method thereof |
| CN102831088A (en) * | 2012-07-27 | 2012-12-19 | 国家超级计算深圳中心(深圳云计算中心) | Data migration method and device based on mixing memory |
| CN103049349B (en) * | 2012-11-29 | 2016-02-24 | 记忆科技(深圳)有限公司 | Based on Snapshot Method and the system of Bedding storage |
| TWI505090B (en) * | 2013-03-12 | 2015-10-21 | Macronix Int Co Ltd | Difference l2p method |
| CN104516471B (en) * | 2013-09-27 | 2017-04-12 | 国际商业机器公司 | Method and device for managing power supply of storage system |
| CN103530236A (en) * | 2013-10-21 | 2014-01-22 | 华为技术有限公司 | Method and device for obtaining hybrid hard disk |
| CN110825324B (en) | 2013-11-27 | 2023-05-30 | 北京奥星贝斯科技有限公司 | Hybrid storage control method and hybrid storage system |
| CN113836084A (en) * | 2014-11-10 | 2021-12-24 | 创新先进技术有限公司 | Data storage method, device and system |
| CN105608085A (en) * | 2014-11-17 | 2016-05-25 | 北京明略软件系统有限公司 | Data query method and device of hybrid storage media |
| US9891831B2 (en) * | 2014-11-25 | 2018-02-13 | Sap Se | Dual data storage using an in-memory array and an on-disk page structure |
| CN106547476B (en) | 2015-09-22 | 2021-11-09 | 伊姆西Ip控股有限责任公司 | Method and apparatus for data storage system |
| CN106557264B (en) * | 2015-09-25 | 2019-11-08 | 伊姆西公司 | For the storage method and equipment of solid state hard disk |
| WO2019037093A1 (en) * | 2017-08-25 | 2019-02-28 | 深圳大学 | Spark distributed computing data processing method and system |
| CN107589918A (en) * | 2017-10-12 | 2018-01-16 | 苏州韦科韬信息技术有限公司 | The high storage cost performance of method carried using storage system is mixed |
| CN108089820A (en) * | 2017-12-19 | 2018-05-29 | 上海磁宇信息科技有限公司 | A kind of storage device for being used in mixed way MRAM and DRAM |
| CN109144411A (en) * | 2018-07-24 | 2019-01-04 | 中国电子科技集团公司第三十八研究所 | Data center's hybrid magnetic disc array and its data dynamic migration strategy |
| CN117472795A (en) * | 2020-05-29 | 2024-01-30 | 超聚变数字技术有限公司 | Storage medium management method and server |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5872955A (en) * | 1994-11-29 | 1999-02-16 | International Business Machines Corporation | Method and apparatus for reading and writing data in a disk drive system |
| CN2335192Y (en) * | 1998-02-25 | 1999-08-25 | 穆春虎 | Computer hard disc switcher |
-
2010
- 2010-07-27 CN CN2010102379368A patent/CN101907978B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5872955A (en) * | 1994-11-29 | 1999-02-16 | International Business Machines Corporation | Method and apparatus for reading and writing data in a disk drive system |
| CN2335192Y (en) * | 1998-02-25 | 1999-08-25 | 穆春虎 | Computer hard disc switcher |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101907978A (en) | 2010-12-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101907978B (en) | Mixed storage system and storage method based on solid state disk and magnetic hard disk | |
| US9098395B2 (en) | Logical block management method for a flash memory and control circuit storage system using the same | |
| CN102012791B (en) | Flash based PCIE (peripheral component interface express) board for data storage | |
| US8725946B2 (en) | Mass storage system and method of using hard disk, solid-state media, PCIe edge connector, and raid controller | |
| Andersen et al. | Rethinking flash in the data center | |
| CN102122235B (en) | RAID4 (redundant array of independent disks) system and data reading and writing method thereof | |
| CN106066890B (en) | Distributed high-performance database all-in-one machine system | |
| US20110246716A1 (en) | Concatenating a first raid with a second raid | |
| CN102637147A (en) | Storage system using solid state disk as computer write cache and corresponding management scheduling method | |
| CN103838676B (en) | Data-storage system, date storage method and PCM bridges | |
| US11809723B2 (en) | Unbalanced plane management method, associated data storage device and controller thereof | |
| US10872622B1 (en) | Method and system for deploying mixed storage products on a uniform storage infrastructure | |
| CN107422989A (en) | A kind of more copy read methods of Server SAN systems and storage architecture | |
| US11922019B2 (en) | Storage device read-disturb-based block read temperature utilization system | |
| CN111651126B (en) | Memory writing operation method, system, electronic device and medium | |
| US11379326B2 (en) | Data access method, apparatus and computer program product | |
| US20190057121A1 (en) | Predictable Allocation Latency in Fragmented Log Structured File Systems | |
| CN102262511B (en) | Cache management system and method for RAID (Redundant Array of Independent Disks) | |
| CN107589918A (en) | The high storage cost performance of method carried using storage system is mixed | |
| Gu et al. | Dynamic file cache optimization for hybrid SSDs with high-density and low-cost flash memory | |
| CN101859232A (en) | Variable length data memory interface | |
| US11922035B2 (en) | Read-disturb-based read temperature adjustment system | |
| US11928354B2 (en) | Read-disturb-based read temperature determination system | |
| Pan et al. | Chameleon: a data organization transformation scheme for big data systems | |
| US20230229342A1 (en) | Read-disturb-based read temperature identification system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20111207 Termination date: 20210727 |