WO2022052147A1 - 基于三维闪存存储结构可预测闪存块使用寿命方法及系统 - Google Patents

基于三维闪存存储结构可预测闪存块使用寿命方法及系统 Download PDF

Info

Publication number
WO2022052147A1
WO2022052147A1 PCT/CN2020/115599 CN2020115599W WO2022052147A1 WO 2022052147 A1 WO2022052147 A1 WO 2022052147A1 CN 2020115599 W CN2020115599 W CN 2020115599W WO 2022052147 A1 WO2022052147 A1 WO 2022052147A1
Authority
WO
WIPO (PCT)
Prior art keywords
flash memory
service life
training model
predicting
memory block
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/115599
Other languages
English (en)
French (fr)
Inventor
黄敏
杜雅芝
肖仲喆
顾济华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou University
Original Assignee
Suzhou University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou University filed Critical Suzhou University
Publication of WO2022052147A1 publication Critical patent/WO2022052147A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614—Improving the reliability of storage systems
    • G06F3/0616—Improving the reliability of storage systems in relation to life time, e.g. increasing Mean Time Between Failures [MTBF]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs

Definitions

  • the invention relates to the technical field of predicting the service life of flash memory blocks, in particular to a method and system for predicting the service life of flash memory blocks based on a three-dimensional flash memory storage structure.
  • NAND Flash is widely used in storage systems and computer systems due to its high storage density, high throughput and low power consumption.
  • the feature size under the process technology has reached the limit, and it becomes extremely difficult to further increase the storage capacity on the traditional two-dimensional planar structure.
  • three-dimensional flash memory storage structure is mentioned, which breaks the bottleneck of storage capacity but its reliability is difficult to guarantee.
  • the flash block can only maintain limited Program/Erase (P/E) cycles, that is, the durability problem, which is defined as the maximum P/E cycles, which means that the flash block can be Its raw bit error rate (Raw Bit Error Rate, referred to as RBER) exceeds the error correction capability of the error correction code (Error Corrector Code, referred to as ECC) to maintain the service life.
  • P/E Program/Erase
  • RBER Raw bit error rate
  • ECC Error Corrector Code
  • the two main problems are: first, in order to ensure data Integrity, too much ECC will be used, resulting in the problem of excessive delay in the decoding process of data; secondly, the traditional endurance definition method uses conservative P/E cycles as the maximum endurance of flash blocks, resulting in many flash memory Blocks are not fully used, which not only causes the problem of wasting resources, but also seriously wastes storage space.
  • the technical problem to be solved by the present invention is to overcome the problem that many flash memory blocks in the prior art are not fully used and seriously waste storage space, so as to provide a three-dimensional flash memory based three-dimensional flash memory that can be fully used and avoid wasting storage space.
  • a method for predicting the service life of a flash memory block based on a three-dimensional flash memory storage structure of the present invention includes: step S1: inputting request data into a data queue; step S2: judging whether the data queue is full, and if so, Enter step S3, if not, return to step S1; step S3: load the training model for predicting the service life of the flash memory block into the host system; step S4: judge whether the accuracy of the training model is less than the set threshold, if so, update the training model , and store the updated training model in the storage device as a new training model for predicting the service life of the next flash block, if not, return to step S3; step S5: allocate in the reserved space of the storage device A new flash block is used as a replacement block, into which data from a flash block near the maximum useful life is migrated before failure occurs.
  • the request data is received through a file system, and a request instruction is sent through the file system.
  • the file system after the file system sends the request instruction, it records the number of different wear operations and the retention time between two wear operations, and sends and collects the original bit error rate data and uncorrectable data in a fixed period. Bit error rate data, and input the above data and historical data as the training model.
  • the training model in the training model, is updated according to the characteristics of each flash memory block, and the updated training model is saved in the storage device.
  • the training model when the training model is updated according to the characteristics of each flash memory block, the following training models are used based on machine learning algorithms: Bayesian classifier, random forest, gradient boosting decision tree, long and short-term memory network, volume Productive Neural Networks - Long Short-Term Memory Networks.
  • the method for loading the training model for predicting the lifetime of a flash memory block into the host system is: loading the training model of the flash memory block from a storage device and loading it onto the host system.
  • the updated training model is stored in a storage device, and is used as the next training model, and the original model is replaced.
  • the new training model for predicting the service life of the next flash memory block is loaded into the host system, and the accuracy of the new training model is judged Whether it is less than the set threshold, if so, update the training model and store the updated training model in the storage device as a new training model for predicting the service life of the next flash block; if not, predict the next flash A new trained model for the block lifetime is loaded into the host system.
  • the mapping table is updated after migrating data in flash blocks that are close to the maximum useful life into new replacement blocks before failure occurs.
  • the present invention also provides a system for predicting the service life of flash memory blocks based on a three-dimensional flash memory storage structure, comprising: a data request module for inputting request data into a data queue; a first judgment module for judging whether the data queue has been If it is full, if it is, enter the training model module, if not, return to the data request module; the training model module is used to load the training model for predicting the service life of the flash memory block into the host system; the second judgment module is used to judge Whether the accuracy of the training model is less than the set threshold, if so, update the training model and store the updated training model in the storage device as a new training model for predicting the service life of the next flash block, if not, return to A loading module; a flash memory block replacement module, used for allocating a new flash memory block as a replacement block in the reserved space of the storage device, and migrating the data in the flash memory block close to the maximum service life to the replacement block before failure occurs in the block.
  • the method and system for predicting the service life of a flash memory block based on the three-dimensional flash memory storage structure of the present invention can effectively predict the service life of the flash memory block, make full use of the underutilized flash memory block, and prolong the service life of the storage system; in addition, The application works efficiently to preserve the integrity of the stored data even when the flash block is nearing its maximum useful life.
  • FIG. 1 is a flow chart of a method for predicting the service life of a flash memory block based on a three-dimensional flash memory storage structure of the present invention
  • FIG. 2 is a structural diagram of a method for predicting the service life of a flash memory block based on a three-dimensional flash memory storage structure of the present invention
  • this embodiment provides a method for predicting the service life of a flash memory block based on a three-dimensional flash memory storage structure, including the following steps: Step S1 : input request data into a data queue; Step S2 : determine whether the length of the data queue is not Within the specified range, if yes, go to step S3, if not, go back to step S1; step S3: load the training model for predicting the service life of the flash memory block into the host system; step S4: determine whether the accuracy of the training model is less than the set threshold , if so, update the training model, and store the updated training model in the storage device as a new training model for predicting the service life of the next flash block; if not, return to step S3; step S5: in the storage A new flash block is allocated as a replacement block within the reserved space of the device, and the data in the flash block near the maximum useful life is migrated to the replacement block before failure occurs.
  • step S1 the request data is input into the data queue, and a life cycle prediction model is established on the basis of the collected data, which is conducive to predicting the flash memory The service life of the block; in the step S2, judging whether the data queue is full is to judge whether the length of the data queue is within the specified range, if so, go to step S3, if not, return to step S1, which is conducive to loading the training model; In the step S3, the training model for predicting the service life of the flash memory block is loaded into the host system, which is conducive to judging the accuracy of the training model; in the step S4, it is judged whether the accuracy of the training model is less than the set threshold, and if so, update the model, and store the updated model in the storage device as a new training model for predicting the service life of the next flash block.
  • step S3 By updating the training model of the flash block, the problem of memory overhead can not only be reduced , it can also solve the problem that the prediction accuracy of the prediction model decreases over time; in addition, because the service life of the flash memory block is effectively predicted and the underutilized flash memory block is fully utilized, it is beneficial to prolong the service life of the storage system;
  • step S5 a new flash memory block is allocated as a replacement block in the reserved space of the storage device, and the data in the flash memory block close to the maximum service life is migrated to the replacement block before the failure occurs, and the data in the flash memory block is moved to the replacement block before the failure occurs. Before approaching the maximum service life, the data on the flash memory block is protected and backed up in time to ensure the integrity of the stored data.
  • the request data is received through a file system (file system), and a request instruction is sent through the file system.
  • file system file system
  • the client sends the request data, and sends the request command through the file system, and performs corresponding operations such as reading and writing.
  • the file system After the file system sends the request command, it is transmitted to the storage device side of the open channel solid state disk, and the original bit error rate data is sent and collected in a fixed period by recording the number of different wear operations and the retention time between two wear operations. and uncorrectable bit error rate data, and use the above data as well as the historical database as input to the flash block lifetime prediction model.
  • the flash block lifetime prediction model can predict when a flash block will fail in advance. It helps to ensure the reliability and integrity of the data.
  • the life prediction system updates the training model, and saves the updated training model as a file in the open-channel solid-state disk. Because the data on the flash memory block is protected and backed up in time before the flash memory block approaches the maximum service life, so The integrity of the stored data can be guaranteed.
  • the training model is updated according to the characteristics of each flash memory block, and the updated training model is saved in the storage device.
  • the following training models are used based on the machine learning algorithm: Bayes classifier (Bayes for short), Random Forest (RF for short), gradient boosting decision tree Gradient Boosted decision trees (GBDT for short), Long short-term memory networks (LSTM for short), Convolutional neural network with long short-term memory (CNNLSTM for short).
  • the updated training model is stored in the storage device and used as the next training model, and the original model is replaced
  • the updated training model is stored in the storage device and used as the next training model, and the original model is replaced.
  • step S5 a new flash memory block is allocated as a replacement block in the storage space, the data in the flash memory block close to the maximum service life is migrated to the replacement block before failure occurs, and the mapping table is updated, so that a new flash memory block can be obtained. physical address.
  • the flash block marked as a dashed box in the figure is the flash block with the maximum useful life, which is called a Failure Block; find a new flash block in the reserved space of the storage device, which is marked as The flash memory of the solid line frame is called the Replacement Block. Migrate the data in the Failure Block to the Replacement Block in a timely manner, and then update the mapping table, so that a new physical address can be obtained.
  • the present application sends and collects the original bit error rate and uncorrectable bit error rate data and historical database in a fixed period by recording the number of different wear operations and the retention time between two wear operations. Input to a predictable model of flash block lifetime.
  • LUN0 and LUN1 LUNs
  • Planes LUNs
  • Block1 Block1, Block2 and Block3
  • the upper file system sends the operation command to the device side of the open channel SSD
  • the flash memory-based device controller records the number of wear times and calculates the retention time between two times, and sends and collects all data in two cycles.
  • the data is sent to a lifespan prediction system consisting of a historical database and a training model, which is used to pre-train the lifespan prediction model and update the model over time, in addition, the model can be saved as a file Saved on an open-channel SSD, each flash block has a corresponding model file.
  • this embodiment provides a system for predicting the service life of a flash memory block based on a three-dimensional flash memory storage structure.
  • the principle of solving the problem is similar to the method for predicting the service life of a flash memory block based on a three-dimensional flash memory storage structure. No longer.
  • This embodiment provides a system for predicting the service life of flash memory blocks based on a three-dimensional flash memory storage structure, including:
  • the data request module is used to input the request data into the data queue
  • the first judgment module is used to judge whether the length of the data queue is within the specified range, if so, enter the training model module, if not, return to the data request module;
  • a training model module for loading a training model for predicting the lifetime of flash blocks into the host system
  • the second judgment module is used to judge whether the accuracy of the training model is less than the set threshold, and if so, update the training model, and store the updated training model in the storage device as a new training method for predicting the service life of the next flash block. model, if not, return to load module;
  • a flash memory block replacement module configured to allocate a new flash memory block in the reserved space of the storage device as a replacement block, and migrate the data in the flash memory block close to the maximum service life to the replacement block before failure occurs .
  • the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
  • computer-usable storage media including, but not limited to, disk storage, CD-ROM, optical storage, etc.
  • These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions
  • the apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Human Computer Interaction (AREA)
  • Techniques For Improving Reliability Of Storages (AREA)

Abstract

一种基于三维闪存存储结构可预测闪存块使用寿命方法及系统，该方法包括：将请求数据输入至数据队列中；判断数据队列是否已满；若否，则继续执行将请求数据输入至数据队列中的步骤，若是，将预测闪存块使用寿命的训练模型加载到主机系统中，对数据进行训练；判断训练模型的精度是否小于设定阈值，若是，则更新模型，并将更新后的模型存储在存储设备中，作为预测下一个闪存块使用寿命的新的训练模型，若否，则返回执行将预测闪存块使用寿命的训练模型加载到主机系统中，对数据进行训练的步骤；在所述存储设备的预留空间内分配一个新的闪存块作为替换块，将接近最大使用寿命的闪存块中的数据在发生故障之前迁移至所述替换块中，更新映射表。该方法有利于保证存储数据的完整性，使得所有闪存块尽可能的均匀磨损，延长存储系统的使用寿命以增强其可靠性。

Description

基于三维闪存存储结构可预测闪存块使用寿命方法及系统 技术领域
本发明涉及预测闪存块使用寿命的技术领域，尤其是指一种基于三维闪存存储结构可预测闪存块使用寿命方法及系统。
背景技术
NAND Flash因其高存储密度、高吞吐量以及低功耗等优势被广泛地应用在存储系统和计算机系统中。然而，在工艺技术下的特征尺寸已经达到极限，若要在传统的二维平面结构上进一步增加存储容量变得极其困难。为了解决这个问题，三维闪存存储结构被提及，打破了存储容量的瓶颈但其可靠性难以得到保证。
无论是在二维还是三维存储结构下，闪存块只能维持有限的Program/Erase(简称P/E)cycles，即耐久性问题，被定义为最大P/E cycles，也就是说闪存块能够在其原始误码率(Raw Bit Error Rate，简称RBER)超过纠错码(Error Corrector Code，简称ECC)的纠错能力之前维持使用寿命。在三维闪存存储系统中，闪存块与闪存块之间，物理页与物理页之间的RBER存在很大差异，带来了很多可靠性问题，最主要的两个问题：其一，为了保证数据完整性，会使用过多的ECC，导致数据在解码过程中会引起延迟过大的问题；其次，传统的耐久性定义方法将保守的P/E cycles作为闪存块的最大耐久性，导致很多闪存块未被充分使用，不但造成浪费资源的问题，而且严重浪费存储空间。
发明内容
为此，本发明所要解决的技术问题在于克服现有技术中很多闪存块未被充分使用，严重浪费存储空间的问题，从而提供一种闪存块可以被充分使用， 避免浪费存储空间的基于三维闪存存储结构可预测闪存块使用寿命方法及系统。
为解决上述技术问题，本发明的一种基于三维闪存存储结构可预测闪存块使用寿命方法，包括：步骤S1：将请求数据输入至数据队列中；步骤S2：判断数据队列是否已满，若是，进入步骤S3，若否，则返回步骤S1；步骤S3：将预测闪存块使用寿命的训练模型加载到主机系统中；步骤S4：判断训练模型的精度是否小于设定阈值，若是，则更新训练模型，并将更新后的训练模型存储在存储设备中，作为预测下一个闪存块使用寿命的新的训练模型，若否，则返回步骤S3；步骤S5：在所述存储设备的预留空间内分配一个新的闪存块作为替换块，将接近最大使用寿命的闪存块中的数据在发生故障之前迁移至所述替换块中。
在本发明的一个实施例中，所述请求数据通过文件系统接收，并通过文件系统发送请求指令。
在本发明的一个实施例中，所述文件系统发送请求指令后，记录不同磨损操作次数以及两次磨损操作之间的保留时间，在固定的周期内发送并收集原始误码率数据和不可纠正误码率数据，并将上述数据以及历史数据作为训练模型输入。
在本发明的一个实施例中，所述训练模型中，根据每个闪存块的特点更新训练模型，并将更新好的训练模型保存到存储设备中。
在本发明的一个实施例中，根据每个闪存块的特点更新训练模型时，基于机器学习算法利用以下训练模型：贝叶斯分类器、随机森林、梯度提升决策树、长短时记忆网络、卷积神经网络-长短时记忆网络。
在本发明的一个实施例中，将预测闪存块使用寿命的训练模型加载到主机系统中的方法为：从存储设备中加载所述闪存块的训练模型并将其加载到所述主机系统上。
在本发明的一个实施例中，更新后的训练模型存储在存储设备，作为下一次训练模型使用，原始模型被替换。
在本发明的一个实施例中，作为预测下一个闪存块使用寿命的新的训练模型时，将预测下一个闪存块使用寿命的新的训练模型加载到主机系统中，判断新的训练模型的精度是否小于设定阈值，若是，则更新训练模型，并将更新后的训练模型存储在存储设备中，作为预测再下一个闪存块使用寿命的新的训练模型，若否，则将预测下一个闪存块使用寿命的新的训练模型加载到主机系统中。
在本发明的一个实施例中，将接近最大使用寿命的闪存块中的数据在发生故障之前迁移至新的替换块中后，更新映射表。
本发明还提供了一种基于三维闪存存储结构可预测闪存块使用寿命系统，包括：数据请求模块，用于将请求数据输入至数据队列中；第一判断模块，用于判断数据队列的是否已满，若是，进入训练模型模块，若否，则返回至所述数据请求模块；训练模型模块，用于将预测闪存块使用寿命的训练模型加载到主机系统中；第二判断模块，用于判断训练模型的精度是否小于设定阈值，若是，则更新训练模型，并将更新后的训练模型存储在存储设备中，作为预测下一个闪存块使用寿命的新的训练模型，若否，则返回至加载模块；闪存块替换模块，用于在所述存储设备的预留空间内分配一个新的闪存块作为替换块，将接近最大使用寿命的闪存块中的数据在发生故障之前迁移至所述替换块中。
本发明的上述技术方案相比现有技术具有以下优点：
本发明所述的基于三维闪存存储结构可预测闪存块使用寿命方法及系统，能有效地预测出闪存块的使用寿命，充分利用未被完全利用的闪存块，延长存储系统的使用寿命；另外，即使闪存块接近最大使用寿命时，本申请也能有效工作来保存存储数据的完整性。
附图说明
为了使本发明的内容更容易被清楚的理解，下面根据本发明的具体实施例并结合附图，对本发明作进一步详细的说明，其中
图1是本发明基于三维闪存存储结构可预测闪存块使用寿命方法流程 图；
图2是本发明基于三维闪存存储结构的可预测闪存块使用寿命方法结构图；
图3是本发明闪存块替换方法。
具体实施方式
实施例一
如图1所示，本实施例提供一种基于三维闪存存储结构可预测闪存块使用寿命方法，包括如下步骤：步骤S1：将请求数据输入至数据队列中；步骤S2：判断数据队列的长度是否在指定范围内，若是，进入步骤S3，若否，则返回步骤S1；步骤S3：将预测闪存块使用寿命的训练模型加载到主机系统中；步骤S4：判断训练模型的精度是否小于设定阈值，若是，则更新训练模型，并将更新后的训练模型存储在存储设备中，作为预测下一个闪存块使用寿命的新的训练模型，若否，则返回步骤S3；步骤S5：在所述存储设备的预留空间内分配一个新的闪存块作为替换块，将接近最大使用寿命的闪存块中的数据在发生故障之前迁移至替换块中。
本实施例所述基于三维闪存存储结构可预测闪存块使用寿命方法，所述步骤S1中，将请求数据输入至数据队列中，在收集到的数据基础上建立生命周期预测模型，有利于预测闪存块的使用寿命；所述步骤S2中，判断数据队列是否已满，是判断数据队列的长度是否在指定范围内，若是，进入步骤S3，若否，则返回步骤S1，有利于加载训练模型；所述步骤S3中，将预测闪存块使用寿命的训练模型加载到主机系统中，有利于判断训练模型的精度；所述步骤S4中，判断训练模型的精度是否小于设定阈值，若是，则更新模型，并将更新后的模型存储在存储设备中，作为预测下一个闪存块使用寿命的新的训练模型，若否，则返回步骤S3，通过更新闪存块的训练模型，不但可以降低内存开销问题，也可以解决随着时间推移预测模型的预测精度降低的问题；另外，由于有效地预测出闪存块的使用寿命，充分利用未被完全利用的闪存块，因此有利于延长存储系统的使用寿命；所述步骤S5 中，在所述存储设备的预留空间内分配一个新的闪存块作为替换块，将接近最大使用寿命的闪存块中的数据在发生故障之前迁移至替换块中，在闪存块接近最大使用寿命之前对所述闪存块上的数据进行及时保护备份，以保证存储数据的完整性。
如图2和图3所示，所述请求数据通过文件系统(file system)接收，并通过文件系统发送请求指令。具体地，用户端发出请求数据，并通过文件系统发送请求指令，执行对应的读写等操作。
所述文件系统发送请求指令后，传输至开放通道固态盘的存储设备端，通过记录不同磨损操作次数以及两次磨损操作之间的保留时间，在固定的周期内发送并收集原始误码率数据和不可纠正误码率数据，并将上述数据以及历史数据库作为闪存块使用寿命预测模型的输入。在三维闪存存储系统中，闪存块会随着时间的推移逐渐被磨损，但是每个闪存块磨损的程度差异很大，因此借助闪存块使用寿命可预测模型可以提前预测闪存块何时失效，有利于保证数据的可靠性和完整性。
所述寿命预测系统更新训练模型，并将更新后的训练模型另存为文件保存在开放通道固态盘中，由于在闪存块接近最大使用寿命之前对所述闪存块上的数据进行及时保护备份，因此可以保证存储数据的完整性。
所述步骤S3中，所述训练模型中，根据每个闪存块的特点更新训练模型，并将更新好的训练模型保存到存储设备中。具体地，根据每个闪存块的特点更新训练模型时，基于机器学习算法利用以下训练模型：贝叶斯分类器Bayes classifier(简称Bayes)、随机森林Random Forest(简称RF)、梯度提升决策树Gradient boosted decision trees(简称GBDT)、长短时记忆网络Long short-term memory networks(简称LSTM)、卷积神经网络-长短时记忆网络Convolutional neural network with long short-term memory(简称CNNLSTM)。
更新后的训练模型存储在存储设备，作为下一次训练模型使用，原始模型被替换
所述步骤S4中，更新后的训练模型存储在存储设备，作为下一次训练模型使用，原始模型被替换。
作为预测下一个闪存块使用寿命的新的训练模型时，将预测下一个闪存块使用寿命的新的训练模型加载到主机系统中，判断新的训练模型的精度是否小于设定阈值，若是，则更新模型，并将更新后的模型存储在存储设备中，作为预测再下一个闪存块使用寿命的新的训练模型，若否，则将预测下一个闪存块使用寿命的新的训练模型加载到主机系统中。由于闪存块与闪存块的变化导致没有一个单独的模型能够适合所有块的问题，若训练每个闪存块的寿命预测模型，会造成很大的开销，因此，通过使用历史数据库来更新闪存块的训练模型，不但可以降低内存开销问题，也可以解决随着时间推移预测模型的预测精度降低的问题。
所述步骤S5中，在存储空间上分配一个新的闪存块作为替换块，将接近最大使用寿命的闪存块中的数据在发生故障之前迁移至替换块中，更新映射表，从而可以得到新的物理地址。
具体地，如图3所示，假设底层设备结构中每个Plane里有4个物理块，两个物理块作为预留空间，每个物理块有4个物理页，通过闪存块使用寿命预测模型预测闪存块的寿命，在图中标记成虚框的闪存块就是接近最大使用寿命的闪存块，称之为Failure Block；在存储设备的预留空间中找到新的闪存块，在图中标记成实线框的闪存，称之Replacement Block。将Failure Block中的数据进行及时的数据迁移到Replacement Block中，随之更新映射表，从而可以得到新的物理地址。
如图2所示，本申请通过记录不同磨损操作次数以及两次磨损操作之间的保留时间，在固定的周期内发送并收集所述原始误码率和不可纠正误码率数据以及历史数据库作为闪存块使用寿命可预测模型的输入。假设三维闪存存储系统中的每个闪存芯片里两个LUN(LUN0和LUN1)，每个LUN里有两个Plane，每个Plane里有4个物理块(Block0、Block1、Block2以及Block3)，两个物理块作为预留空间，每个Block里有n个物理页(Page0、 Page1...Page n)。具体流程如下：首先，上层文件系统将操作命令发送到开放通道固态盘的设备端，基于闪存的设备控制器记录磨损次数并计算两次之间的保留时间，在两个周期内发送并收集所述数据，将所述数据发送到由历史数据库和训练模型组成的寿命预测系统，所述寿命预测系统用于预训练寿命预测模型并随着时间的推移更新模型，另外，可以将模型另存为文件保存在开放通道固态盘中，每个闪存块都有一个对应的模型文件。当需要预测任何闪存块寿命时，将相应的模型文件加载到训练模型中，然后，将预测出的即将达到使用寿命的闪存块标记出Failure Block，在预留空间里面分配一个新块即Replacement Block用来迁移Failure Block中的数据，从而可以保证存储数据的完整性，充分利用未完全使用的闪存块使得所有闪存块尽可能均匀磨损，达到延长存储系统的使用寿命增强其可靠性的目的。
实施例二
基于同一发明构思，本实施例提供了一种基于三维闪存存储结构可预测闪存块使用寿命系统，其解决问题的原理与所述基于三维闪存存储结构可预测闪存块使用寿命方法类似，重复之处不再赘述。
本实施例提供一种基于三维闪存存储结构可预测闪存块使用寿命系统，包括：
数据请求模块，用于将请求数据输入至数据队列中；
第一判断模块，用于判断数据队列的长度是否在指定范围内，若是，进入训练模型模块，若否，则返回至所述数据请求模块；
训练模型模块，用于将预测闪存块使用寿命的训练模型加载到主机系统中；
第二判断模块，用于判断训练模型的精度是否小于设定阈值，若是，则更新训练模型，并将更新后的训练模型存储在存储设备中，作为预测下一个闪存块使用寿命的新的训练模型，若否，则返回至加载模块；
闪存块替换模块，用于在所述存储设备的预留空间内分配一个新的闪存 块，作为替换块，将接近最大使用寿命的闪存块中的数据在发生故障之前迁移至所述替换块中。
本领域内的技术人员应明白，本申请的实施例可提供为方法、系统、或计算机程序产品。因此，本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本申请是参照根据本申请实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
显然，上述实施例仅仅是为清楚地说明所作的举例，并非对实施方式的限定。对于所属领域的普通技术人员来说，在上述说明的基础上还可以做出其它不同形式变化或变动。这里无需也无法对所有的实施方式予以穷举。而 由此所引伸出的显而易见的变化或变动仍处于本发明创造的保护范围之中。

Claims (10)

  1. 一种基于三维闪存存储结构可预测闪存块使用寿命方法，其特征在于，包括如下步骤：
    步骤S1：将请求数据输入至数据队列中；
    步骤S2：判断数据队列是否已满，若是，进入步骤S3，若否，则返回步骤S1；
    步骤S3：将预测闪存块使用寿命的训练模型加载到主机系统中；
    步骤S4：判断训练模型的精度是否小于设定阈值，若是，则更新训练模型，并将更新后的训练模型存储在存储设备中，作为预测下一个闪存块使用寿命的新的训练模型，若否，则返回步骤S3；
    步骤S5：在所述存储设备的预留空间内分配一个新的闪存块，作为替换块，将接近最大使用寿命的闪存块中的数据在发生故障之前迁移至所述替换块中。
  2. 根据权利要求1所述的基于三维闪存存储结构可预测闪存块使用寿命方法，其特征在于：所述请求数据通过文件系统接收，并通过文件系统发送请求指令。
  3. 根据权利要求2所述的基于三维闪存存储结构可预测闪存块使用寿命方法，其特征在于：所述文件系统发送请求指令后，记录不同磨损操作次数以及两次磨损操作之间的保留时间，在固定的周期内发送并收集原始误码率数据和不可纠正误码率数据，并将上述数据以及历史数据作为训练模型输入。
  4. 根据权利要求1所述的基于三维闪存存储结构可预测闪存块使用寿命方法，其特征在于：所述训练模型中，根据每个闪存块的特点更新训练模型，并将更新好的训练模型保存到存储设备中。
  5. 根据权利要求4所述的基于三维闪存存储结构可预测闪存块使用寿命 方法，其特征在于：根据每个闪存块的特点更新训练模型时，基于机器学习算法利用以下训练模型：贝叶斯分类器、随机森林、梯度提升决策树、长短时记忆网络、卷积神经网络-长短时记忆网络。
  6. 根据权利要求1所述的基于三维闪存存储结构可预测闪存块使用寿命方法，其特征在于：将预测闪存块使用寿命的训练模型加载到主机系统中的方法为：从存储设备中加载所述闪存块的训练模型并将其加载到所述主机系统上。
  7. 根据权利要求1所述的基于三维闪存存储结构可预测闪存块使用寿命方法，其特征在于：更新后的训练模型存储在存储设备，作为下一次训练模型使用，原始模型被替换。
  8. 根据权利要求1所述的基于三维闪存存储结构可预测闪存块使用寿命方法，其特征在于：作为预测下一个闪存块使用寿命的新的训练模型时，将预测下一个闪存块使用寿命的新的训练模型加载到主机系统中，判断新的训练模型的精度是否小于设定阈值，若是，则更新训练模型，并将更新后的训练模型存储在存储设备中，作为预测再下一个闪存块使用寿命的新的训练模型，若否，则将预测下一个闪存块使用寿命的新的训练模型加载到主机系统中。
  9. 根据权利要求1所述的基于三维闪存存储结构可预测闪存块使用寿命方法，其特征在于：将接近最大使用寿命的闪存块中的数据在发生故障之前迁移至新的替换块中后，更新映射表。
  10. 一种基于三维闪存存储结构可预测闪存块使用寿命系统，其特征在于，包括：
    数据请求模块，用于将请求数据输入至数据队列中；
    第一判断模块，用于判断数据队列是否已满，若是，进入训练模型模块，若否，则返回至所述数据请求模块；
    训练模型模块，用于将预测闪存块使用寿命的训练模型加载到主机系统 中；第二判断模块，用于判断训练模型的精度是否小于设定阈值，若是，则更新训练模型，并将更新后的训练模型存储在存储设备中，作为预测下一个闪存块使用寿命的新的训练模型，若否，则返回至加载模块；
    闪存块替换模块，用于在所述存储设备的预留空间内分配一个新的闪存块，作为替换块，将接近最大使用寿命的闪存块中的数据在发生故障之前迁移至所述替换块中。
PCT/CN2020/115599 2020-09-10 2020-09-16 基于三维闪存存储结构可预测闪存块使用寿命方法及系统 Ceased WO2022052147A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010949191.1A CN112085107A (zh) 2020-09-10 2020-09-10 基于三维闪存存储结构可预测闪存块使用寿命方法及系统
CN202010949191.1 2020-09-10

Publications (1)

Publication Number Publication Date
WO2022052147A1 true WO2022052147A1 (zh) 2022-03-17

Family

ID=73736904

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/115599 Ceased WO2022052147A1 (zh) 2020-09-10 2020-09-16 基于三维闪存存储结构可预测闪存块使用寿命方法及系统

Country Status (2)

Country Link
CN (1) CN112085107A (zh)
WO (1) WO2022052147A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120406834A (zh) * 2025-04-09 2025-08-01 成都芯盛集成电路有限公司 一种基于人工智能的NAND Flash坏块管理方法及系统

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118673780A (zh) * 2023-03-16 2024-09-20 中国科学院微电子研究所 一种阻变存储器芯片的寿命预测及修复方法
CN120526834A (zh) * 2025-04-22 2025-08-22 联和存储科技(江苏)有限公司 基于控制器的nand flash寿命预测方法、系统、装置和控制器

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101226504A (zh) * 2008-01-25 2008-07-23 炬力集成电路设计有限公司 一种防止存储器读损的方法及装置
CN108766496A (zh) * 2018-05-23 2018-11-06 武汉忆数存储技术有限公司 一种在线动态预测闪存芯片寿命的方法及装置
CN109634527A (zh) * 2018-12-12 2019-04-16 华中科技大学 一种ssd内实现的闪存寿命预测方法
US20190155520A1 (en) * 2017-11-21 2019-05-23 Distech Controls Inc. Computing device and method for inferring a predicted number of physical blocks erased from a flash memory
CN109817267A (zh) * 2018-12-17 2019-05-28 武汉忆数存储技术有限公司 一种基于深度学习的闪存寿命预测方法、系统及计算机可读存取介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101226504A (zh) * 2008-01-25 2008-07-23 炬力集成电路设计有限公司 一种防止存储器读损的方法及装置
US20190155520A1 (en) * 2017-11-21 2019-05-23 Distech Controls Inc. Computing device and method for inferring a predicted number of physical blocks erased from a flash memory
CN108766496A (zh) * 2018-05-23 2018-11-06 武汉忆数存储技术有限公司 一种在线动态预测闪存芯片寿命的方法及装置
CN109634527A (zh) * 2018-12-12 2019-04-16 华中科技大学 一种ssd内实现的闪存寿命预测方法
CN109817267A (zh) * 2018-12-17 2019-05-28 武汉忆数存储技术有限公司 一种基于深度学习的闪存寿命预测方法、系统及计算机可读存取介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120406834A (zh) * 2025-04-09 2025-08-01 成都芯盛集成电路有限公司 一种基于人工智能的NAND Flash坏块管理方法及系统

Also Published As

Publication number Publication date
CN112085107A (zh) 2020-12-15

Similar Documents

Publication Publication Date Title
CN103150263B (zh) 分级存储方法
US9946642B2 (en) Distributed multimode storage management
US9824092B2 (en) File storage system including tiers
US11029853B2 (en) Dynamic segment allocation for write requests by a storage system
KR102401596B1 (ko) 비휘발성 장치에서 데이터 속성 기반 데이터 배치를 활용하기 위해 컴퓨터를 구동하는 경험적 인터페이스
KR102541458B1 (ko) 불휘발성 메모리 장치에서의 데이터 특성 기반 데이터 배치
US10649890B2 (en) Information processing system, storage control apparatus, storage control method, and storage control program
KR102275094B1 (ko) 저장된 데이터를 플래시 메모리에 기초한 저장 매체에 기입하기 위한 방법 및 디바이스
EP4012547B1 (en) Storage method and apparatus for key value (kv) and storage device
JP2018041245A (ja) ストレージ装置、及びストレージ制御装置
WO2019000950A1 (zh) 分片管理方法和分片管理装置
US11474724B2 (en) System and method for speed up data rebuild in a distributed storage system with local deduplication
WO2022052147A1 (zh) 基于三维闪存存储结构可预测闪存块使用寿命方法及系统
US20210382825A1 (en) Efficient accessing methods for bypassing second layer mapping of data blocks in file systems of distributed data systems
CN118760623B (zh) 一种用于半导体存储的数据管理方法
KR20170046769A (ko) 인코딩 형식을 바꿈으로써 저장 디바이스의 공표되는 용량을 감소시키기 위한 프로세스 및 장치
US20190026191A1 (en) Managing Data in a Storage System
CN107203341A (zh) 基于闪存的数据存储方法、装置以及闪存芯片
CN105955672B (zh) 用于灵活控制损耗均衡的固态存储系统及方法
US10872015B2 (en) Data storage system with strategic contention avoidance
CN117149473A (zh) 一种硬盘故障预测的方法、系统、设备和介质
US11281392B2 (en) Garbage collection in a memory component using an adjusted parameter
WO2020215223A1 (zh) 分布式存储系统和分布式存储系统中垃圾回收方法
US12422999B2 (en) Solid-state disks wear leveling with predictive migration of data devices in a raid container system
CN112799590A (zh) 一种针对在线主存储重删的差异化缓存方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20952935

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20952935

Country of ref document: EP

Kind code of ref document: A1