CN114153384A - Storage method and device of hybrid cache - Google Patents
Storage method and device of hybrid cache Download PDFInfo
- Publication number
- CN114153384A CN114153384A CN202111320322.0A CN202111320322A CN114153384A CN 114153384 A CN114153384 A CN 114153384A CN 202111320322 A CN202111320322 A CN 202111320322A CN 114153384 A CN114153384 A CN 114153384A
- Authority
- CN
- China
- Prior art keywords
- cache
- data
- command
- storage
- storage method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000013507 mapping Methods 0.000 claims description 19
- 239000002245 particle Substances 0.000 claims description 9
- 230000007717 exclusion Effects 0.000 claims description 2
- 238000000926 separation method Methods 0.000 claims description 2
- 239000012634 fragment Substances 0.000 abstract description 2
- 230000001788 irregular Effects 0.000 abstract description 2
- 230000009466 transformation Effects 0.000 abstract description 2
- 230000005540 biological transmission Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 238000013506 data mapping Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 239000008187 granular material Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/062—Securing storage systems
- G06F3/0622—Securing storage systems in relation to access
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Software Systems (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
A storage method of a mixed cache belongs to the technical field of storage, and is characterized by comprising the following steps: dividing data into data blocks with preset sizes; associating and writing the data blocks into a queue buffer in a data linked list form; based on complex and irregular data commands of an operating system, the storage method can dynamically cache and store the system write commands, and can give consideration to IOPS and sequential write performance of the storage device; meanwhile, since the device caches are subjected to classified management, the cache space can be utilized to the maximum extent, and the generation of memory fragments can be reduced to the maximum extent; through reconstruction transformation of continuous small data commands, the command efficiency is improved, table item operation performed by a CPU of the device when the FTL is established is reduced, and the CPU utilization rate is improved; the high concurrency number of random writing and the large data throughput of sequential writing are considered, and the overall performance of the storage device can be remarkably improved.
Description
Technical Field
The invention belongs to the technical field of storage, and particularly relates to a storage method and a storage device for hybrid cache.
Background
With the increasing performance requirements on data transmission and storage, higher requirements are put forward on the randomness of small data and the throughput performance of large data.
Currently, storage devices such as solid state disks mainly use Nand Flash as a main storage medium, the medium has the function of erasing and then writing, the erasing is performed in Block units, the Block units have various sizes from 4k × 128 to 16k × 2048, the medium also has the characteristic of using Page as the minimum writing unit, the size of the Page also has size difference from 4k to 16k, and the reading and writing of the current common operating system use 512Byte as a logical unit, so a set of FTL mapping layer of logical address-physical address is required in the working process of the storage device.
The performance of a storage device often includes random concurrency and data throughput, but it is currently common to map a block with a uniform size, which is generally consistent with a data block provided by an operating system, such as 4K, 8K, 16K, 32K to 128K.
Disclosure of Invention
The present invention aims to solve the above problems, and provides a storage method and apparatus for a hybrid cache in which a large data map and a small data map coexist.
In a first aspect, the present invention provides a storage method for a hybrid cache, including the following processes:
dividing data into data blocks with preset sizes;
associating and writing the data blocks into a queue cache in a data linked list form, and starting to read and analyze the command cache after the command interrupt is received by the equipment;
after the storage equipment acquires the command, command reconstruction is carried out on the data linked list based on the command length;
the command reconstruction includes: when the data addresses are continuous, the logic circuit merges the small data blocks into the supportable maximum data block, and the maximum data block is adopted for mapping, so that the command transmission efficiency can be improved in multiples; when the data address is discrete or is a discrete command, adopting a cache block mapping with the same size as the data block of the operating system;
setting a buffer type flag bit in the command reconstruction to mark the buffer type used by the current command linked list; the storage device distributes physical blocks with corresponding sizes for the type of data based on the cache type flag bit, generates a write command, writes the write command into the non-lost storage particles, and fills in a corresponding mapping table;
after the write command is actually transmitted and written into the non-lost storage particles, the corresponding state table is searched based on the cache class mark, and the cache is released.
Further, in the hybrid cache storage method of the present invention, the cache manner of the write queue cache includes a nested manner and a separated manner;
the nested formula is: the data cache area is not divided, a large data cache and a small data cache are multiplexed, the data cache takes the preset size as a minimum unit, and each unit data block independently uses a state table; the independent small data block and the multiplexing large data block independently use a state table for managing each unit data;
the separation formula is as follows: the data cache region comprises a plurality of cache regions with different sizes, each cache region independently uses an address space, and the state tables are independent.
Furthermore, in the storage method of the hybrid cache, the cache is acquired by adopting a prefetching mechanism, and each unit cache corresponds to a prefetching table; the logic circuit searches the corresponding state table when the prefetch table is not full, and stops searching when the prefetch table is full.
Further, in the hybrid cache storage method of the present invention, when the cache manner is a split search, each type of cache state table is independent; when the cache mode is nested search, various cache state tables meet the mutual exclusion relationship.
Further, in the hybrid cache storage method according to the present invention, the unit cache state table size = cache size ÷ (data block unit × 8).
Further, according to the storage method of the hybrid cache disclosed by the invention, the preset size range comprises 512b-128 kb.
In a second aspect, the present invention provides a storage device for a hybrid cache, where the storage device may operate the storage method for the hybrid cache according to any one of the first aspect when storing data.
The storage method and the device of the hybrid cache are based on complex and irregular data commands of an operating system, can dynamically cache and store system write commands by adopting the storage method, and can give consideration to IOPS and sequential write performance of storage equipment; meanwhile, since the device caches are subjected to classified management, the cache space can be utilized to the maximum extent, and the generation of memory fragments can be reduced to the maximum extent; through reconstruction transformation of continuous small data commands, the command efficiency is improved, table item operation performed by a CPU of the device when the FTL is established is reduced, and the CPU utilization rate is improved; the high concurrency number of random writing and the large data throughput of sequential writing are considered, and the overall performance of the storage device can be remarkably improved.
Drawings
FIG. 1 is a diagram illustrating a nested data cache according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating a split data cache according to an embodiment of the present invention;
fig. 3 is a schematic diagram illustrating data cache state management according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a command parsing and reconstructing module according to an embodiment of the present invention;
fig. 5 is a schematic diagram of a nested cache eviction module according to an embodiment of the present invention.
Detailed Description
The storage method and apparatus of the hybrid cache according to the present invention are described in detail with reference to the accompanying drawings and embodiments.
Example one
The embodiment of the disclosure discloses a storage method of a hybrid cache, which comprises the following processes:
dividing data into data blocks with preset sizes;
associating and writing the data blocks into a queue cache in a data linked list form, and starting to read and analyze the command cache after the command interrupt is received by the equipment;
after the storage equipment acquires the command, command reconstruction is carried out on the data linked list based on the command length;
the command reconstruction includes: when the data addresses are continuous, the logic circuit merges the small data blocks into the supportable maximum data block, and the maximum data block is adopted for mapping, so that the command transmission efficiency can be improved in multiples; when the data address is discrete or is a discrete command, adopting a cache block mapping with the same size as the data block of the operating system;
setting a buffer type flag bit in the command reconstruction to mark the buffer type used by the current command linked list; the storage device distributes physical blocks with corresponding sizes for the type of data based on the cache type flag bit, generates a write command, writes the write command into the non-lost storage particles, and fills in a corresponding mapping table;
after the write command is actually transmitted and written into the non-lost storage particles, the corresponding state table is searched based on the cache class mark, and the cache is released.
In the embodiment of the present disclosure, taking SSD as an example, the front end transmission protocol is NVME; the hybrid cache storage process comprises the following steps:
the operating system divides data into data blocks with the size of 4k in the data command transmission process, the size of the data blocks is different from 4k to 128k, but the data blocks are consistent in size, and the specific size can be determined according to actual use conditions. After the operating system executes the write command, the data is divided into data blocks with the size of 4k, each data block is associated in a data linked list mode and written into a queue cache, and the device starts to read and analyze the command cache after receiving the command interrupt.
After the device side obtains the command, the data linked list is reconstructed based on the command length, and in specific application, based on different configurations of the device, the caching mode can be divided into a nested mode and a separated mode, wherein the nested mode shown in fig. 1 is as follows: the data buffer area of the equipment end is not divided, the large data buffer and the small data buffer are multiplexed, wherein the data buffer takes 4k as the minimum unit, and 8 minimum data blocks form a 32k data block. Correspondingly, if the minimum unit in use is 512 bytes, 8 minimum data caches form a 4k data block, and 64 minimum data caches form a 32k data block. As shown in fig. 3, each type of unit data block independently uses a state table for managing each unit data.
As shown in fig. 2, the separate device data buffer areas are divided into 4k and 32k, the two data buffer areas use the address space independently, and the state tables are independent; the 8k/16k data area can be increased based on the requirement in specific application; where unit buffer status table size = buffer size ÷ (data block unit × 8).
In command reconstruction, when data addresses of operating system commands are continuous, the logic circuit merges small data blocks into the maximum supportable data block, in the embodiment of the present disclosure, as shown in fig. 4, continuous 8 4k address continuous commands can be reconstructed into one 32k command, and the FTL mapping block of the command is mapped by 32k, so that command transmission efficiency can be improved by times. Such as command data address scatter or scatter commands, will use cache blocks of the same data block size as the operating system, with the FTL mapping table using a 4k mapping.
In the embodiment of the present disclosure, a prefetch mechanism is adopted for obtaining the device-side cache, each unit cache corresponds to one prefetch table, the size of each type of cache prefetch table in the embodiment of the present disclosure is 32 double words, and each double word is used for storing one cache pointer; the prefetch table adopts a FIFO (first input first output) mechanism, when the prefetch table is not full, the logic circuit searches the corresponding state table, when the prefetch table is full, the search is stopped, and after each cache pointer is written into the FIFO, the cache state table is invalid. During separated searching, all types of cache state tables are independent; as shown in FIG. 5, when searching in a nested manner, the various state tables satisfy mutually exclusive relationships.
Providing a buffer class flag bit in the command reconstruction process, wherein the flag bit indicates the buffer class used by the current command linked list, and in application, the equipment firmware allocates physical blocks with corresponding sizes for the class data based on the flag bit, generates a command for writing in non-lost storage particles and fills in a corresponding FTL mapping table; when the write command is actually transmitted and written into the non-lost storage granule, the logic circuit finds the corresponding state table and releases the cache based on the cache class mark, and at this moment, the corresponding cache class state mark is valid.
Example two
The embodiment of the present disclosure discloses a storage device adopting a hybrid cache storage method, wherein a computer program is stored on the storage device, and the storage device is executed by a processor when performing data storage operation, so that the hybrid cache storage method of the first embodiment can be implemented. Firstly, dividing data into data blocks with preset sizes; and associating the data blocks in a data linked list form and writing the data blocks into a queue cache, and starting to read and analyze the command cache after the device receives the command interrupt.
After the storage equipment acquires the command, command reconstruction is carried out on the data linked list based on the command length; the command reconstruction includes: when the data addresses are continuous, the logic circuit merges the small data blocks into the supportable maximum data block, and the maximum data block is adopted for mapping, so that the command transmission efficiency can be improved in multiples; when the data address is discrete or is a discrete command, a cache block mapping with the same size as the data block of the operating system is adopted.
Setting a buffer type flag bit in the command reconstruction to mark the buffer type used by the current command linked list; the storage device distributes physical blocks with corresponding sizes for the type of data based on the cache type flag bit, generates a write command, writes the write command into the non-lost storage particles, and fills in a corresponding mapping table;
after the write command is actually transmitted and written into the non-lost storage particles, searching a corresponding state table based on the cache class mark and releasing the cache; the specific storage operation steps are the same as the storage method of the hybrid cache in the first embodiment, and are not described herein again.
According to the storage method and the storage device of the hybrid cache, the mapping layer is constructed by adopting a mode of coexistence of the big data mapping and the small data mapping, the command transmission efficiency is improved, the hybrid mapping mode can give consideration to both high concurrent number of random writing and big data throughput of sequential writing, and the overall performance of equipment can be remarkably improved.
Claims (7)
1. A storage method of a hybrid cache is characterized by comprising the following steps:
dividing data into data blocks with preset sizes;
associating and writing the data blocks into a queue buffer in a data linked list form;
after the storage equipment acquires the command, command reconstruction is carried out on the data linked list based on the command length;
the command reconstruction includes: when the data addresses are continuous, combining the small data blocks into a supportable maximum data block, and adopting maximum data block mapping; when the data address is discrete or is a discrete command, adopting a cache block mapping with the same size as the data block of the operating system;
setting a buffer type flag bit in the command reconstruction to mark the buffer type used by the current command linked list; the storage device distributes physical blocks with corresponding sizes for the type of data based on the cache type flag bit, generates a write command, writes the write command into the non-lost storage particles, and fills in a corresponding mapping table;
after the write command is actually transmitted and written into the non-lost storage particles, the corresponding state table is searched based on the cache class mark, and the cache is released.
2. The storage method of the hybrid cache according to claim 1, wherein: the caching mode of the write queue cache comprises a nested mode and a separated mode;
the nested formula is: the data cache area is not divided, the data cache takes the preset size as the minimum unit, and each unit data block independently uses a state table;
the separation formula is as follows: the data cache region comprises a plurality of cache regions with different sizes, each cache region independently uses an address space, and the state tables are independent.
3. The storage method of the hybrid cache according to claim 2, wherein: the cache is obtained by adopting a prefetching mechanism, and each unit cache corresponds to a prefetching table; the logic circuit searches the corresponding state table when the prefetch table is not full, and stops searching when the prefetch table is full.
4. The storage method of the hybrid cache according to claim 3, wherein: when the cache mode is the separated search, all types of cache state tables are independent; when the cache mode is nested search, various cache state tables meet the mutual exclusion relationship.
5. The storage method of the hybrid cache according to claim 4, wherein: the unit buffer status table size = buffer size ÷ (data block unit × 8).
6. The storage method of the hybrid cache according to claim 5, wherein: the preset size range includes 512b-128 k.
7. A hybrid cache storage device, comprising: the storage device can operate the storage method of the hybrid cache of any one of the preceding claims 1 to 7 when storing data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111320322.0A CN114153384A (en) | 2021-11-09 | 2021-11-09 | Storage method and device of hybrid cache |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111320322.0A CN114153384A (en) | 2021-11-09 | 2021-11-09 | Storage method and device of hybrid cache |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114153384A true CN114153384A (en) | 2022-03-08 |
Family
ID=80459749
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111320322.0A Withdrawn CN114153384A (en) | 2021-11-09 | 2021-11-09 | Storage method and device of hybrid cache |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114153384A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101739353A (en) * | 2008-11-06 | 2010-06-16 | 瑞昱半导体股份有限公司 | Adaptive buffer device and method thereof |
| CN104503710A (en) * | 2015-01-23 | 2015-04-08 | 福州瑞芯微电子有限公司 | Method and device for increasing writing speed of nand flash |
| CN105739919A (en) * | 2016-01-21 | 2016-07-06 | 捷鼎国际股份有限公司 | Data access system and method |
| CN107066393A (en) * | 2017-01-12 | 2017-08-18 | 安徽大学 | The method for improving map information density in address mapping table |
| CN111352865A (en) * | 2018-12-24 | 2020-06-30 | 北京忆芯科技有限公司 | Write caching for memory controllers |
| CN112380148A (en) * | 2020-11-30 | 2021-02-19 | 海光信息技术股份有限公司 | Data transmission method and data transmission device |
-
2021
- 2021-11-09 CN CN202111320322.0A patent/CN114153384A/en not_active Withdrawn
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101739353A (en) * | 2008-11-06 | 2010-06-16 | 瑞昱半导体股份有限公司 | Adaptive buffer device and method thereof |
| CN104503710A (en) * | 2015-01-23 | 2015-04-08 | 福州瑞芯微电子有限公司 | Method and device for increasing writing speed of nand flash |
| CN105739919A (en) * | 2016-01-21 | 2016-07-06 | 捷鼎国际股份有限公司 | Data access system and method |
| CN107066393A (en) * | 2017-01-12 | 2017-08-18 | 安徽大学 | The method for improving map information density in address mapping table |
| CN111352865A (en) * | 2018-12-24 | 2020-06-30 | 北京忆芯科技有限公司 | Write caching for memory controllers |
| CN112380148A (en) * | 2020-11-30 | 2021-02-19 | 海光信息技术股份有限公司 | Data transmission method and data transmission device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11681473B2 (en) | Memory system and control method | |
| CN105917319B (en) | Memory cell and method | |
| US20210042235A1 (en) | Invalidation data area for cache | |
| KR101480659B1 (en) | Two-level system main memory | |
| US8954656B2 (en) | Method and system for reducing mapping table size in a storage device | |
| KR20170098187A (en) | Associative and atomic write-back caching system and method for storage subsystem | |
| US10310984B2 (en) | Storage apparatus and storage control method | |
| US9317312B2 (en) | Computer and memory management method | |
| US20210103518A1 (en) | Deallocating command processing method and storage device thereof | |
| US20170004069A1 (en) | Dynamic memory expansion by data compression | |
| Lee et al. | iLSM-SSD: An intelligent LSM-tree based key-value SSD for data analytics | |
| US20240160579A1 (en) | Accessing compressed computer memory | |
| CN113039531B (en) | Method, system and storage medium for allocating cache resources | |
| US11630779B2 (en) | Hybrid storage device with three-level memory mapping | |
| US20200250087A1 (en) | Dynamic allocation of memory between containers | |
| Benveniste et al. | Cache-memory interfaces in compressed memory systems | |
| CN114153384A (en) | Storage method and device of hybrid cache | |
| US20230205449A1 (en) | Storage device, operation method of storage device, and storage system using the same | |
| JP2018136970A (en) | Information processing device | |
| CN117806570B (en) | Online memory expansion method, device, equipment and storage medium | |
| WO2022021337A1 (en) | Flash memory control method and device | |
| US20220229552A1 (en) | Computer system including main memory device having heterogeneous memories, and data management method thereof | |
| CN114398297A (en) | Storage method and storage device based on mixed address mapping | |
| CN116643999A (en) | L2P accelerator | |
| No12 et al. | Fully Harnessing the Performance Potential of DRAM-less Mobile Flash Storage |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20220308 |