WO2019047612A1 - Procédé de stockage et de gestion de valeurs de clés pour une optimisation de chemins de stockage basés sur flash - Google Patents

Procédé de stockage et de gestion de valeurs de clés pour une optimisation de chemins de stockage basés sur flash Download PDF

Info

Publication number
WO2019047612A1
WO2019047612A1 PCT/CN2018/094909 CN2018094909W WO2019047612A1 WO 2019047612 A1 WO2019047612 A1 WO 2019047612A1 CN 2018094909 W CN2018094909 W CN 2018094909W WO 2019047612 A1 WO2019047612 A1 WO 2019047612A1
Authority
WO
WIPO (PCT)
Prior art keywords
key value
flash
value storage
storage management
management system
Prior art date
Application number
PCT/CN2018/094909
Other languages
English (en)
Chinese (zh)
Inventor
陆游游
舒继武
张佳程
Original Assignee
清华大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 清华大学 filed Critical 清华大学
Publication of WO2019047612A1 publication Critical patent/WO2019047612A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614—Improving the reliability of storage systems
    • G06F3/0616—Improving the reliability of storage systems in relation to life time, e.g. increasing Mean Time Between Failures [MTBF]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629—Configuration or reconfiguration of storage systems
    • G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks

Definitions

  • the present invention relates to the field of flash memory storage technologies, and in particular, to a key value storage management method based on flash memory storage path optimization.
  • Flash memory is an electronic erasable programming memory. Compared with traditional disk media, flash memory has the characteristics of high read/write bandwidth, low access latency, low power consumption, and strong stability. It has begun to spread in data centers, personal computers, and mobile devices. The flash memory is read and written in units of pages. Before the flash memory rewrites a page, it needs to be erased first. Flash memory is erased in blocks, and a flash block contains hundreds of flash pages. The cells of the flash have a limited number of erase operations, ie each flash cell has a limited lifetime. The internal structure of the flash drive and the disk are also significantly different. Inside the flash drive, the flash chip is connected to the flash controller through different channels.
  • a flash chip In a flash chip, a plurality of flash granules are packaged, and each granule can execute instructions independently. Each particle contains multiple flash slices, each with separate registers that provide pipelined instruction execution between multiple flash slices. With different levels of concurrency, SSDs provide sufficient access bandwidth. This feature is known as the internal concurrency of flash devices.
  • the current general-purpose SSD is built on a flash drive. It uses the flash translation layer to manage the read and write erase of the flash and internal concurrency, and provides the same read and write interface to the software system as the traditional disk.
  • the flash translation layer consists of three main functions: address mapping, garbage collection, and wear leveling.
  • the address mapping is used to record the mapping relationship between the logical address and the physical address of the flash during the remote update of the flash memory; garbage collection is to select and erase the invalid flash page to restore the idle state and keep the new data written; wear leveling is to ensure Reliable storage of data, evenly writing data to each flash page.
  • the flash translation layer also includes ECC (Error Correction Code), bad block management and other functions.
  • the flash translation layer manages the unique properties of flash and is transparent to the upper layers of software. This allows existing software programs to run on SSDs without any modifications, saving the cost of secondary software development and deployment.
  • existing software such as key-value storage management software
  • Existing key value storage management software runs on top of the file system.
  • the file system is functionally redundant with the flash translation layer. For example, the file system and the flash translation layer have the functions of space allocation, garbage collection, and addressing. These redundant features bring additional software overhead to the performance of the software.
  • the flash translation layer cannot be specifically optimized for the upper-level key-value storage management software, and the file system cannot sense the characteristics of the underlying flash memory and exert its internal concurrency.
  • Another outstanding problem is that all software middleware is designed and developed based on the characteristics of the previous disk in the storage path of the current key storage system. It is not optimized according to the characteristics of the flash and key storage system. Take advantage of the performance of both. There is currently little research on how to optimize the storage path for flash-based key-value storage.
  • the present invention aims to solve at least one of the above technical problems.
  • the object of the present invention is to provide a flash value-based storage path optimization key value storage management method, which improves the performance of the key value storage system, reduces the writing amount to the flash memory device, and prolongs the use of the device. life.
  • an embodiment of the present invention provides a flash value-based storage path optimization key value storage management method, including the following steps: S1: directly managing a bare flash memory device through a key value storage management system, bypassing files System and flash translation layer; S2: when the key value storage management system performs physical space allocation, the concurrent data layout method is adopted, and the key file is distributed to different flash channels of the flash device in units of flash blocks, and at the same time, the key The value storage management system stores the key value storage file as an integer multiple of the flash block; S3: when the key value storage management system performs data compression, adopts a dynamic compression method, and dynamically adopts a corresponding quantity according to the access characteristics of the foreground user.
  • the flash channel writes the compressed data; S4: when the key value storage management system performs data caching, the compressed sensing cache algorithm is used, and the compressed data is not cached, and the saved space is used for caching the user's reading.
  • flash value-based storage path optimization key value storage management method may further have the following additional technical features:
  • the bare flash device directly exports the internal structure information of the device to the user state, and through a specific interface, enables the key value storage management system to be in the user mode, directly to the bare flash memory.
  • the device is managed to bypass the file system and the flash translation layer in the traditional key storage management system structure, wherein the internal structure information of the device includes at least the number of flash channels, the size of the flash block, and the read/write and erase control of the flash memory. .
  • the concurrent data layout method specifically includes: the number of flash channels and the flash block size that are transmitted by the key value storage management system through the bare flash device, and the length of the key file is set;
  • the key value storage management system divides the key file in units of flash block lengths when storing the key value file, and distributes different flash blocks to different flash channels; the data in the key file is in the flash block. , distributed in a polling manner.
  • the dynamic compression method specifically includes: when the key value storage management system performs background compression on the key value data, first determining the read/write ratio requested by the foreground user, when the user When the write ratio is greater than the first preset ratio, the key value storage management system writes the compressed key file using all the flash channels, and when the read ratio of the user is higher than the second preset ratio, The key value storage management system writes the compressed file using half of the flash channel; the key value storage management system performs the judgment of the read/write ratio by recording the type of the user request between two consecutive compression operations; Compressing the request, the key value storage management system uses all flash channels for data writing.
  • the method for compressing data specifically includes: the key value storage management system first reads a key value file that needs to be compressed when performing data compression, and the key value storage management system reads multiple times at the same time.
  • the fixed length in the file is in the cache; after the key value data in the cache is compressed, the subsequent data in the plurality of files is read, and so on, all the files to be compressed are read; the key value storage The management system writes the compressed data to the flash device and the compression process ends.
  • the compression-aware caching algorithm specifically includes: the key value storage management system reads the first part of the key file to be compressed into the cache after the compression process is started. And compressing the data in the cache; when the key value storage management system compresses, the cache of the key value storage management system starts a prefetch process, and the latter part of the key value file to be compressed is required, Preloading into the cache; after the first part of the data is compressed, the key value storage management system compresses the pre-fetched part of the data, and at this time, the cache replaces the data that has been used in the first part, and replaces the cache; The user reads and writes the request, and the cache of the key value storage management system adopts a caching algorithm optimized for the foreground request.
  • the cache algorithm optimized for the foreground request includes: read and write requests to the foreground user, use the length of the flash page as the cache granularity, perform cache management, and read requests to the foreground user. , no prefetch processing; when the cache space is insufficient, the cache data is replaced according to the principle of least recently used.
  • the priority-based scheduling policy specifically includes: for a read/write request of a foreground user, the key value storage management system gives a high priority to scheduling when scheduling For the read and write request generated by the background data compression operation, the key value storage management system gives its low priority and schedules when scheduling; in the same priority level, the read request takes precedence over the write request for scheduling; In addition to the request, the key value storage management system dynamically adjusts its priority according to the usage of the current flash device when scheduling.
  • the key value storage management system dynamically adjusts the priority of the wipe request, specifically: the key value storage management system records the available space of the current flash device; when the available space is less than the third storage of the total storage space When the ratio is set, the key value storage management system gives the highest priority to the erasure request, and the erasure request is first scheduled; when the available space is greater than the third preset ratio of the total storage space, the key value storage management system gives The erase request has the lowest priority and the erase request is scheduled at the latest.
  • the third predetermined ratio is 40%.
  • the flash value-based storage path optimization key value storage management method removes the file system and the flash translation layer in the traditional key value storage management system structure on the storage architecture, and directly pairs the key value data in the user state.
  • Management of storage on flash memory eliminates the problem of semantic isolation and redundancy management caused by the file system and flash translation layer, avoiding the overhead of additional garbage collection and write amplification; the physicality of key-value data
  • the concurrent data layout method by designing the size of the key file to an integral multiple of the flash block, and distributing the flash blocks belonging to the same key file to different flash channels, the internal concurrency advantage of the flash is utilized. Improve the effective bandwidth of data reading and writing, and reduce the read and write delay.
  • When compressing key-value data use dynamic compression method to dynamically reduce the amount of compressed data written according to the read/write ratio of the foreground user.
  • the number of channels can reduce the interference of the front-end user's reading and writing when the data is compressed in the background; cache the key-value data
  • Using the compression-aware caching algorithm prioritizes the elimination of compressed data, and uses more space to store the user's read and write data, which can effectively improve the cache hit rate; when scheduling read and write requests to the flash, use priority-based
  • the scheduling policy which preferentially schedules user and foreground compression requests, can reduce the delays visible to the user. Therefore, the method improves the performance of the key value storage system, reduces the writing amount to the flash memory device, and prolongs the service life of the device.
  • FIG. 1 is a flow chart of a method for managing a key value storage based on a flash memory based storage path optimization according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram showing an implementation principle of a flash value-based storage path optimization key value storage management method according to an embodiment of the present invention
  • FIG. 3 is a schematic diagram of a concurrent data layout in accordance with one embodiment of the present invention.
  • FIG. 4 is a schematic diagram of dynamic compression in accordance with one embodiment of the present invention.
  • FIG. 5 is a schematic diagram of a buffering algorithm for compressed sensing according to an embodiment of the present invention.
  • FIG. 6 is a schematic diagram of a priority based scheduling policy in accordance with one embodiment of the present invention.
  • connection In the description of the present invention, it should be noted that the terms “installation”, “connected”, and “connected” are to be understood broadly, and may be fixed or detachable, for example, unless otherwise explicitly defined and defined. Connected, or integrally connected; can be mechanical or electrical; can be directly connected, or indirectly connected through an intermediate medium, can be the internal communication of the two components.
  • Connected, or integrally connected can be mechanical or electrical; can be directly connected, or indirectly connected through an intermediate medium, can be the internal communication of the two components.
  • the specific meaning of the above terms in the present invention can be understood in a specific case by those skilled in the art.
  • FIG. 1 is a flow chart of a flash memory based storage path optimized key value storage management method in accordance with one embodiment of the present invention. As shown in Figure 1, the method includes the following steps:
  • Step S1 directly manage the bare flash device through the key value storage management system, bypassing the file system and the flash translation layer.
  • the bare flash memory device refers to a flash memory storage device in which the flash memory storage device removes the flash memory conversion layer of the conventional flash memory.
  • the bare flash device directly exports the internal structure information of the device to the user mode, and through a specific interface, enables the key value storage management system to directly manage the bare flash device in the user state to bypass the traditional key value storage management system.
  • the internal structure information of the device includes at least the number of flash channels, the size of the flash block, and the read/write and erase control of the flash memory.
  • the bare flash device includes, for example, a flash disk device that removes the flash translation layer, and transfers the flash device internal information and control commands to the kernel mode device driver of the user state key value storage management system.
  • the key value storage management system directly manages the bare flash device, including: the key value storage management system allocates a physical flash page for the key file to be written; the key value storage management system uses the erase request to reclaim the used physical flash memory. Space, this process is also called garbage collection; the key value storage management system caches the read and written data; the key value storage management system schedules read requests, write requests, and erase requests to the flash device.
  • Step S2 When the key value storage management system performs physical space allocation, the concurrent data layout method is adopted, and the key value file is distributed to different flash channels of the flash device in units of flash blocks, and at the same time, the key value storage management system sets the key value.
  • the storage file is stored as an integer multiple of the flash block.
  • the concurrent data layout method specifically includes: the number of flash channels and the flash block size transmitted by the key value storage management system through the bare flash device, and the length of the key file, wherein the length of the key file is set. Is the product of the number of flash channels and the length of the flash block; when storing the key file, the key value storage system divides the key file into units of flash block length, and distributes different flash blocks to different flash channels; The data in the value file is distributed in the flash block in a polling manner.
  • Step S3 When the key value storage management system performs data compression, the dynamic compression method is adopted, and the compressed data is dynamically written by using a corresponding number of flash channels according to the access characteristics of the foreground user.
  • the dynamic compression method specifically includes: when the key value storage management system performs background compression on the key value data, first determines the read/write ratio requested by the foreground user, and when the user write ratio is greater than the first preset ratio (ie, indicating a high write ratio), the key value storage management system uses all the flash channels to write the compressed key file.
  • the user requests the read ratio is higher than the second preset ratio (ie, the read ratio is higher).
  • the key value storage management system uses half of the flash channel to write compressed files; the key value storage management system records the user request type between two consecutive compression operations to determine the read and write ratio; The front-end compression request, the key value storage management system will use all the flash channels for data writing.
  • the method for determining the ratio of the read and write requests by the foreground user includes: the key value storage management system records the number of read and write requests of the foreground user between the two compressions; and by comparing the ratio of the number of times the user reads the request and the number of write requests, Whether the current user has a higher read ratio or a higher write ratio.
  • the method for compressing data specifically includes: when the data storage is performed, the key value storage management system first reads the key value file that needs to be compressed, and the key value storage management system reads multiple files at the same time. The fixed length is in the cache; after the key value data in the cache is compressed, the subsequent data in the multiple files is read, and so on, all the files to be compressed are read; the key value storage management system will The compressed data is written to the flash device and the compression process ends.
  • Step S4 When the key value storage management system performs data caching, the compressed sensing cache algorithm is used to cache the compressed data, and the saved space is used to cache the user's read and write data.
  • the compressed sensing cache algorithm specifically includes: after the compression process is started, the key value storage management system reads the first part of the key file to be compressed into the cache, and The data is compressed; when the key value storage management system is compressed, the cache of the key value storage management system starts the prefetch process, and the latter part of the data of the key file to be compressed is preloaded into the cache; in the first part of the data compression After the completion, the key value storage management system compresses the data of the pre-fetched part. At this time, the cache replaces the data that has been used in the first part, and replaces the cache; for the read and write requests of the user in the foreground, the cache of the key value storage management system is adopted.
  • the cache algorithm optimized for the foreground request includes: a read and write request to the foreground user, a length of the flash page is used for the cache granularity, and the cache management is performed; and the read request to the foreground user is not performed. Prefetch processing; when the cache space is insufficient, the cache data is replaced according to the least recently used principle.
  • Step S5 When the key value storage management system performs the request scheduling, the priority-based scheduling policy is adopted, and the compression request of the user and the foreground is preferentially scheduled, and the priority of the erasure request scheduling is determined according to the available space of the current flash storage device.
  • the priority-based scheduling policy specifically includes: for the read/write request of the foreground user, the key value storage management system gives the high priority to perform scheduling during scheduling; and generates the background data compression operation.
  • Read and write request the key value storage management system gives its low priority to schedule when scheduling; in the same priority level, the read request is prioritized over the write request for scheduling; for the erase request, the key value storage management system is When scheduling, its priority is dynamically adjusted based on the current flash device usage.
  • the key value storage management system dynamically adjusts the priority of the wipe request, and specifically includes: the key value storage management system records the available space of the current flash device; when the available space is less than the third preset ratio of the total storage space, the key The value storage management system gives the highest priority to the erase request, and the erase request is first scheduled; when the available space is greater than the third preset ratio of the total storage space, the key storage management system gives the lowest priority to the erase request, In addition to requesting the latest schedule.
  • the third preset ratio is, for example, 40%.
  • the key value storage management system records the available space of the current flash memory device; when the available space is less than 40% of the total storage space, the key value storage management system gives the highest priority to the erase request, and the erase request is first scheduled; When the available space is greater than 40% of the total storage space, the key value storage management system gives the lowest priority to the erase request, and the erase request is scheduled at the latest.
  • the key value storage management system is directly deployed on the bare flash memory device, and the bare flash memory device uses the specific interface to read the internal structure information and read of the flash memory.
  • the write and erase control interfaces are passed to the key value storage management system; the key value storage management system distributes the key value files on the physical unit of the flash memory through the concurrent data layout method; the key value storage management system adopts a dynamic compression method according to the foreground The user's read/write ratio dynamically selects the number of flash channels used for compression; the key value storage management system uses a compression-aware caching algorithm to adopt a strategy of prefetching and prioritizing the compressed key-value data, and data requested by the foreground user.
  • Adopting a strategy of non-prefetching, the least recently used policy to replace; the key value storage management system uses a priority-based scheduling policy to schedule read, write, and erase requests of the flash device, wherein the user's read and write requests are prioritized Level is higher than the priority of the background compression request, the priority of the erase request is current Memory device is inversely proportional to the available space.
  • the bare flash device is a SSD device that removes the flash translation layer and passes the internal information and control commands of the flash device to the user-mode key value storage management software through a specific interface.
  • the internal information of the device includes: the number of channels of the flash device, the length of the flash block, the length of the flash page, and the capacity of the flash chip; the control commands of the device include: a read flash page, a write flash page, and an erase flash block command.
  • the key value storage management system can directly control the space management and garbage collection of the underlying flash memory device, without the participation of the file system and the flash translation layer, eliminating the original
  • the problem of functional redundancy and semantic isolation caused by the existence of these two layers reduces the overhead on the system software and improves the overall performance of the system.
  • the middleware function of the method in the entire storage path has been specifically optimized, including: optimization of concurrent data layout for data layout distribution; dynamic compression method for data compression; data cache
  • a compression-aware caching algorithm is designed.
  • a priority-based scheduling strategy is designed for request scheduling.
  • the method uses the key value management system to directly manage the underlying flash memory device, bypassing the redundancy overhead of the file system and the flash translation layer, and optimizing the storage path, thereby reducing the software overhead of the key value storage management system during data storage and improving The performance of the system; at the same time, reduce the amount of writing to the flash device, which is conducive to the improvement of the life of the device.
  • FIG. 2 is a schematic diagram showing an implementation principle of a flash value based storage path optimization key value storage management method according to an embodiment of the present invention.
  • the functional structure of the flash-based storage path optimized key value storage management method is mainly divided into three parts, namely user-level key value storage management software and kernel-mode bare flash memory.
  • Drive bare flash device.
  • the bare flash device consists of a flash chip, flash channel and flash control firmware that removes the flash translation layer of the traditional SSD.
  • the bare flash drive runs in the kernel state of the operating system, which is responsible for the internal information of the bare flash device, including: number of flash channels, flash page size, flash block size, flash chip capacity, and read, write, and erase command interfaces.
  • the user-mode key value storage management software at the same time, the bare flash drive is also responsible for forwarding read, write, and erase requests from the user state key value storage management software to the bare flash device.
  • User-mode key-value storage management software which manages the internal information of the flash memory and the read, write, and erase control interfaces transmitted by the bare flash drive, and directly manages the underlying flash memory hardware in the user state, through concurrent data layout, dynamic compression, and compressed sensing.
  • the cache algorithm and the priority-based scheduling policy optimize the storage path.
  • Figure 3 is a functional diagram of the concurrent data layout.
  • the key value storage management software sets the length of the key file to a length of four flash blocks distributed in four flash co-channels, called a block group.
  • the key value storage management system writes the key value file
  • the flash blocks corresponding to the key value file are distributed on different flash channels. Therefore, the key file can be simultaneously and concurrently written into the blocks of the four channels of the flash memory, and the internal concurrency characteristics of the flash device are exerted in the write operation of the file.
  • the key data is distributed in four flash blocks in a polling manner.
  • the consecutive key values are distributed in four flash channels in a polling manner, that is, the data will be simultaneously read out from the flash device, and the file is read.
  • the internal concurrency features of the flash device are utilized.
  • the key value storage management system manages the flash device in units of block groups, which reduces the management of the physical space of the flash memory by the key value storage management system. Overhead.
  • FIG. 1 a functional schematic diagram of dynamic compression is shown in FIG.
  • the key value storage management system records the number of read and write requests of the foreground user between the two compressions; by comparing the ratio of the number of times the user reads the request and the number of write requests, it can be determined whether the current user has a higher read ratio or a higher write ratio. When the user's write ratio is high, the key value storage management system will use all the flash channels to write the compressed key file to relieve the pressure of the front user data writing.
  • the system uses Full concurrency writes to the compressed file; when the user's request has a high read ratio, the key value storage management system uses half of the flash channel to write the compressed file to reduce the interference of the background data compression on the foreground user read request. Reduce the delay of the user's read request, as shown by the dotted line in Figure 4, semi-concurrency compression; for the user's foreground compression request, the key value storage management system uses all the flash channels to write data to reduce the user's waiting time. Delay.
  • the key value file to be compressed is first read, and the key value storage management system simultaneously reads the fixed files in the file to be compressed.
  • the length is in the cache; after the key value data in the cache is compressed, the subsequent data in the multiple files is read, and so on, until all the files to be compressed are read; the key value storage management system will compress The post data is written to the flash device and the compression process ends.
  • the functional structure diagram of the compressed sensing cache algorithm is shown in Figure 5.
  • the key value storage management system After the compression process is started, the key value storage management system reads the first part of the plurality of key value files that need to be compressed into the cache, and The data is compressed; when the key value storage management system is compressed, the cache of the key value storage management system starts the prefetch process, and preloads the latter part of the data of the plurality of key value files that need to be compressed into the cache; After the data compression is completed, the key value storage management system compresses the pre-fetched part of the data, and at this time, the cache replaces the data that has been used in the first part, and replaces the cache; for the front-end user read and write request, the key value storage management system
  • the cache uses a caching algorithm optimized for foreground requests.
  • the length of the flash page is used as the cache granularity for cache management; the read request to the foreground user is not prefetched; when the cache space is insufficient, the cached data is used according to the least recently used principle. Replace it.
  • the key value storage management system uses a priority-based scheduling policy to schedule a request, such as a priority-based scheduling policy diagram as shown in FIG. 6.
  • a priority-based scheduling policy diagram as shown in FIG. 6.
  • the key value storage management system gives high priority to the scheduling when scheduling, and for the read and write requests generated by the background data compression operation, the key value storage management system gives the low priority to the scheduling.
  • Level, scheduling in the same priority level, the read request is prioritized over the write request; for the erase request, the key value storage management system dynamically adjusts its priority according to the current flash device usage during scheduling.
  • the key value storage management system records the available space of the current flash device; when the available space is less than 40% of the total storage space, the key value storage management system gives the highest priority to the erase request, and the erase request is first scheduled; when the available space is larger than At 40% of the total storage space, the key value storage management system gives the lowest priority to the erase request, and the erase request is scheduled at the latest.
  • the flash value-based storage path optimization key value storage management method removes the file system and the flash translation layer in the traditional key value storage management system structure in the storage architecture, and directly in the user state.
  • Key-value data is managed on the storage of flash memory, eliminating the problem of semantic isolation and redundant management caused by the file system and flash translation layer, avoiding the overhead of additional garbage collection and write amplification;
  • On the physical storage of data using the concurrent data layout method, by designing the size of the key file to an integral multiple of the flash block, and distributing the flash blocks belonging to the same key file to different flash channels, the flash memory is utilized.

Abstract

L'invention concerne un procédé de stockage et de gestion de valeurs de clés pour une optimisation de chemins de stockage basés sur Flash, le procédé comportant les étapes consistant à: faire gérer directement un dispositif Flash nu par un système de stockage et de gestion de valeurs de clés en contournant un système de fichiers et une couche de conversion Flash; lors de l'attribution d'un espace physique, distribuer un fichier de valeurs de clés à différents passages Flash du dispositif Flash par unités de blocs Flash, et stocker un fichier de stockage de valeurs de clés comportant un multiple entier des blocs Flash; lors de la compression de données, selon une caractéristique d'accès d'un utilisateur de premier plan, utiliser dynamiquement une quantité correspondante de passages Flash pour inscrire les données compressées; lors de la mise en antémémoire de données, utiliser un algorithme de mise en antémémoire à perception de compression, et ne pas mettre en antémémoire les données compressées; et lors de la réalisation d'une programmation de demandes, programmer préférentiellement une demande de compression d'un utilisateur et du premier plan, et déterminer la priorité d'une programmation de demandes d'effacement en fonction de l'espace disponible du dispositif de stockage Flash actuel. La présente invention améliore les performances d'un système de stockage de valeurs de clés, réduit la quantité d'inscription d'un dispositif Flash, et prolonge la durée de vie en service du dispositif.
PCT/CN2018/094909 2017-09-11 2018-07-06 Procédé de stockage et de gestion de valeurs de clés pour une optimisation de chemins de stockage basés sur flash WO2019047612A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710812821.9A CN107678685B (zh) 2017-09-11 2017-09-11 基于闪存的存储路径优化的键值存储管理方法
CN201710812821.9 2017-09-11

Publications (1)

Publication Number Publication Date
WO2019047612A1 true WO2019047612A1 (fr) 2019-03-14

Family

ID=61135865

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/094909 WO2019047612A1 (fr) 2017-09-11 2018-07-06 Procédé de stockage et de gestion de valeurs de clés pour une optimisation de chemins de stockage basés sur flash

Country Status (2)

Country Link
CN (1) CN107678685B (fr)
WO (1) WO2019047612A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113742127A (zh) * 2021-09-16 2021-12-03 重庆大学 一种裸闪存文件系统的故障恢复方法

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107678685B (zh) * 2017-09-11 2020-01-17 清华大学 基于闪存的存储路径优化的键值存储管理方法
CN108509353A (zh) * 2018-03-14 2018-09-07 清华大学 基于裸闪存的对象存储构建方法及装置
CN113742304B (zh) * 2021-11-08 2022-02-15 杭州雅观科技有限公司 一种混合云的数据存储方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102436420A (zh) * 2010-10-20 2012-05-02 微软公司 使用辅助存储器的低ram空间、高吞吐量的持久键值存储
CN102929793A (zh) * 2011-08-08 2013-02-13 株式会社东芝 包括键-值存储的存储器系统
US20170139594A1 (en) * 2015-11-17 2017-05-18 Samsung Electronics Co., Ltd. Key-value integrated translation layer
CN107066498A (zh) * 2016-12-30 2017-08-18 成都华为技术有限公司 键值kv存储方法和装置
CN107678685A (zh) * 2017-09-11 2018-02-09 清华大学 基于闪存的存储路径优化的键值存储管理方法

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10127150B2 (en) * 2012-11-09 2018-11-13 Sandisk Technologies Llc Key value addressed storage drive using NAND flash based content addressable memory
CN106469198B (zh) * 2016-08-31 2019-10-15 华为技术有限公司 键值存储方法、装置及系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102436420A (zh) * 2010-10-20 2012-05-02 微软公司 使用辅助存储器的低ram空间、高吞吐量的持久键值存储
CN102929793A (zh) * 2011-08-08 2013-02-13 株式会社东芝 包括键-值存储的存储器系统
US20170139594A1 (en) * 2015-11-17 2017-05-18 Samsung Electronics Co., Ltd. Key-value integrated translation layer
CN107066498A (zh) * 2016-12-30 2017-08-18 成都华为技术有限公司 键值kv存储方法和装置
CN107678685A (zh) * 2017-09-11 2018-02-09 清华大学 基于闪存的存储路径优化的键值存储管理方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113742127A (zh) * 2021-09-16 2021-12-03 重庆大学 一种裸闪存文件系统的故障恢复方法

Also Published As

Publication number Publication date
CN107678685A (zh) 2018-02-09
CN107678685B (zh) 2020-01-17

Similar Documents

Publication Publication Date Title
US10013177B2 (en) Low write amplification in solid state drive
US10489295B2 (en) Systems and methods for managing cache pre-fetch
US10095613B2 (en) Storage device and data processing method thereof
US9256527B2 (en) Logical to physical address mapping in storage systems comprising solid state memory devices
US8914600B2 (en) Selective data storage in LSB and MSB pages
Wu et al. GCaR: Garbage collection aware cache management with improved performance for flash-based SSDs
US10572379B2 (en) Data accessing method and data accessing apparatus
US10203876B2 (en) Storage medium apparatus, method, and program for storing non-contiguous regions
WO2019047612A1 (fr) Procédé de stockage et de gestion de valeurs de clés pour une optimisation de chemins de stockage basés sur flash
US20100287333A1 (en) Data storage device and related method of operation
US20140082323A1 (en) Address mapping
KR20210130829A (ko) 캐시 이동을 비휘발성 대량 메모리 시스템에 제공하기 위한 장치 및 방법
CN110413537B (zh) 一种面向混合固态硬盘的闪存转换层及转换方法
US20170228191A1 (en) Systems and methods for suppressing latency in non-volatile solid state devices
WO2016056104A1 (fr) Dispositif de stockage et procédé de commande de mémoire
US11907129B2 (en) Information processing device, access controller, information processing method, and computer program for issuing access requests from a processor to a sub-processor
US20190303019A1 (en) Memory device and computer system for improving read performance and reliability
Mativenga et al. RFTL: Improving performance of selective caching-based page-level FTL through replication
KR101180288B1 (ko) 하이브리드 메모리와 ｓｓｄ 로 구성된 시스템에서의 읽기 캐시 및 쓰기 캐시 관리 방법
US20240020014A1 (en) Method for Writing Data to Solid-State Drive
CN110908595B (zh) 存储装置及信息处理系统
JP6254986B2 (ja) 情報処理装置、アクセスコントローラ、および情報処理方法
JP6243884B2 (ja) 情報処理装置、プロセッサ、および情報処理方法
KR102088945B1 (ko) 메모리 컨트롤러 및 이를 포함하는 스토리지 디바이스
US11036414B2 (en) Data storage device and control method for non-volatile memory with high-efficiency garbage collection

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18852815

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18852815

Country of ref document: EP

Kind code of ref document: A1