CN116820351B - Cold and hot data calibration method and device, storage medium and electronic equipment - Google Patents

Cold and hot data calibration method and device, storage medium and electronic equipment Download PDF

Info

Publication number
CN116820351B
CN116820351B CN202310898643.1A CN202310898643A CN116820351B CN 116820351 B CN116820351 B CN 116820351B CN 202310898643 A CN202310898643 A CN 202310898643A CN 116820351 B CN116820351 B CN 116820351B
Authority
CN
China
Prior art keywords
data
cold
ssd
hot
hot data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310898643.1A
Other languages
Chinese (zh)
Other versions
CN116820351A (en
Inventor
秦东润
刘晓健
王嵩
康雷
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Dera Technology Co Ltd
Original Assignee
Beijing Dera Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Dera Technology Co Ltd filed Critical Beijing Dera Technology Co Ltd
Priority to CN202310898643.1A priority Critical patent/CN116820351B/en
Publication of CN116820351A publication Critical patent/CN116820351A/en
Application granted granted Critical
Publication of CN116820351B publication Critical patent/CN116820351B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608—Saving storage space on storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614—Improving the reliability of storage systems
    • G06F3/0616—Improving the reliability of storage systems in relation to life time, e.g. increasing Mean Time Between Failures [MTBF]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647—Migration mechanisms
    • G06F3/0649—Lifecycle management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0653—Monitoring storage devices or systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0673—Single storage device
    • G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention relates to the technical field of data storage, and provides a cold and hot data calibration method, a device, a storage medium and electronic equipment, wherein the method comprises the following steps: acquiring cold and hot data pre-judging thresholds counted in the last counting period as data heat indexes of SSD when different values are taken in a preset threshold value set; calculating corresponding write amplification coefficients of the SSD when different data heat indexes are obtained, and selecting a target data heat index corresponding to the time when the value of the write amplification coefficient is minimum; acquiring a cold and hot data pre-judging threshold corresponding to the target data heat index to obtain an optimal cold and hot data pre-judging threshold; and calibrating the cold and hot data of the logic block in the SSD according to the optimal cold and hot data prejudging threshold value. The invention can accurately realize the identification and marking of cold and hot data, and effectively reduce the write amplification factor in the SSD use process, thereby improving the SSD performance and prolonging the SSD service life.

Description

Cold and hot data calibration method and device, storage medium and electronic equipment
Technical Field
The present invention relates to the field of data storage technologies, and in particular, to a method and apparatus for calibrating cold and hot data, a storage medium, and an electronic device.
Background
With the increasing level of the global internet, there is an increasing global demand for data storage. The main stream memory devices of current computer servers are mainly divided into two types: mechanical Hard Disk (HD) and solid state Disk (Solid State Drive, SSD). Solid state drives and mechanical drives are essentially hardware for data storage, which essentially differ in their storage media. The traditional mechanical hard disk uses a mechanical magnetic disk as a storage medium, and data storage and reading are carried out through a mechanical structure between a magnetic arm and a magnetic head as well as between the magnetic disk; the solid state disk uses a NAND flash memory (nonvolatile memory) as a storage medium, and reads and writes data by charge number in the memory, i.e. on/off of a cell, so as to realize data storage.
The primary storage medium used in solid state drives is NAND flash memory grain. Due to the physical characteristics of flash memory, each time data is written, the original data needs to be erased first, and then the writing operation is carried out. While an erase operation may result in all of the data in the flash block being deleted, rather than only a portion thereof. When the erasing operation is frequently performed during the data writing process, a large amount of invalid data is generated in the flash memory block, so that the utilization rate of the flash memory space is reduced. To avoid this, it is necessary to garbage collect (Garbage Collection, GC) NAND particles, clear invalid data and free space. When garbage collection is performed, if cold and hot identification of data cannot be performed effectively, a large number of invalid garbage collection operations are caused.
Therefore, the prior art often adopts a threshold comparison mode to perform identification judgment on cold and hot data, but the threshold adopted in the prior art is often an artificial experience value, and cannot be adaptively adjusted according to the statistical information of the current data, so that the applicability of the existing cold and hot data identification and calibration mode is poor, and the actual scene requirement cannot be met.
Disclosure of Invention
The present invention has been made in view of the above problems, and provides a cold and hot data calibration method, apparatus, storage medium, and electronic device that overcome or at least partially solve the above problems.
In one aspect of the invention, a method for calibrating cold and hot data is provided, the method comprising:
acquiring cold and hot data pre-judging thresholds counted in the last counting period as data heat indexes of the SSD when different values are taken in a preset threshold value set, wherein the data heat indexes comprise the block proportion of a target logic block in the SSD to a total logic block and the data writing frequency of the target block, and the target block is a logic block with writing access times larger than or equal to the current cold and hot data pre-judging threshold value in the logic block of the SSD;
calculating corresponding write amplification coefficients of the SSD when different data heat indexes are obtained, and selecting a target data heat index corresponding to the time when the value of the write amplification coefficient is minimum;
Acquiring a cold and hot data pre-judging threshold corresponding to the target data heat index to obtain an optimal cold and hot data pre-judging threshold;
and calibrating the cold and hot data of the logic block in the SSD according to the optimal cold and hot data prejudging threshold value.
In a second aspect, the invention also provides a cold and hot data calibration device, which comprises a functional module for realizing the cold and hot data calibration method.
In a third aspect, the present invention also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the above cold and hot data calibration method.
In a fourth aspect, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the steps of the above cold and hot data calibration method are implemented when the processor executes the computer program.
According to the cold and hot data calibration method, the device, the storage medium and the electronic equipment, based on the fact that cold and hot data pre-judging thresholds counted in the previous statistic period are data heat indexes of the SSD when different values are taken in the preset threshold value set, according to the corresponding write amplification coefficients of the SSD when the different data heat indexes are taken, the target data heat index corresponding to the time when the write amplification coefficients are the smallest is selected, the cold and hot data pre-judging threshold corresponding to the time when the target data heat index is obtained is used as the optimal cold and hot data pre-judging threshold, cold and hot data calibration is carried out on a logic block in the SSD according to the obtained optimal cold and hot data pre-judging threshold.
The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present invention more readily apparent.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 is a flow chart of a method for calibrating cold and hot data according to an embodiment of the present invention;
FIG. 2 is a system architecture diagram suitable for the cold and hot data calibration method proposed by the present invention;
FIG. 3 is a flowchart of a historical access information statistics flow provided by an embodiment of the present invention;
FIG. 4 is a flowchart of calculating the write data frequency of the current statistical period according to the historical cold and hot data decision ratio according to the embodiment of the invention;
FIG. 5 is a block diagram of a device for calibrating cold and hot data according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It will be understood by those skilled in the art that all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs unless defined otherwise. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
FIG. 1 schematically shows a flow chart of a cold and hot data calibration method according to an embodiment of the invention. Referring to fig. 1, the method for calibrating cold and hot data according to the embodiment of the invention specifically includes the following steps:
s11, acquiring cold and hot data pre-judging thresholds counted in the last counting period as data heat indexes of SSD when different values are taken in a preset threshold value set.
In this embodiment, the data heat index includes a block proportion f of a target logical block in the SSD to a total logical block and a write data frequency r of the target block, where the target block is a logical block in the logical block of the SSD, where the write access number is greater than or equal to a current hot and cold data pre-determination threshold.
S12, calculating write amplification coefficients corresponding to SSD when different data heat indexes are obtained, and selecting a target data heat index corresponding to the minimum write amplification coefficient. In this embodiment, the write amplification factor of the current statistical period is calculated according to the SSD at different data heat indexes, and the write amplification factor with the smallest value is selected from the write amplification factors of the current statistical period as the target data heat index.
And S13, acquiring a cold and hot data pre-judging threshold corresponding to the target data heat index, and acquiring an optimal cold and hot data pre-judging threshold.
S14, calibrating the cold and hot data of the logic blocks in the SSD according to the optimal cold and hot data pre-judging threshold value.
Specifically, a logical block of the logical blocks of the SSD, in which the number of write accesses in the current statistical period is greater than or equal to the current optimal hot and cold data pre-determination threshold, is marked as a hot data logical block, and a logical block of the logical blocks of the SSD, in which the number of write accesses in the current statistical period is less than the current optimal hot and cold data pre-determination threshold, is marked as a cold data logical block.
According to the cold and hot data calibration method provided by the embodiment of the invention, based on the fact that the cold and hot data pre-judging threshold counted in the last statistic period is the data heat index of the SSD when different values are taken in the preset threshold value set, the target data heat index corresponding to the time when the value of the write amplification coefficient is minimum is selected according to the write amplification coefficient corresponding to the time when the SSD is different in the data heat index, and the cold and hot data pre-judging threshold corresponding to the time when the target data heat index is obtained is used as the optimal cold and hot data pre-judging threshold, so that cold and hot data calibration is carried out on the logic blocks in the SSD according to the obtained optimal cold and hot data pre-judging threshold.
In the embodiment of the present invention, the acquiring the data heat index of the SSD when the cold and hot data pre-judging threshold counted in the previous counting period is different values in the preset threshold value set includes: acquiring historical access information of each logic block in the SSD in the last statistics period, wherein the historical access information comprises write access times, overflow times of a counter for counting the write access times and received user write times; when the number of the user writing times is larger than a preset number of writing times threshold, calculating a cold and hot data pre-judging threshold according to the history access information to be the data heat index of the SSD when different values in a preset threshold value set are obtained. Wherein, the value set of the cold and hot data prejudging threshold value CounterThreshold is [ MaxCounter, maxCounter-1, …,1,0].
In practical application, the most important part of cold and hot data calibration is to perform cold and hot marking on current data according to historical access information of the data. In order to achieve the above objective, an embodiment of the present invention provides a system architecture diagram suitable for the cold and hot data calibration method provided by the present invention. As shown in fig. 2, the system architecture proposed by the present invention at least includes the following parts:
LBA frequency counter: the counting information of the Counter consists of LBA addresses and corresponding write access times Counter, and is responsible for recording the accessed times of LBAs corresponding to all logic blocks in the SSD, namely the write access times, in a period of time. It is noted that each of the write access times Counter described above may correspond to 1 LBA address, or a segment of consecutive LBA addresses. The embodiment of the present invention is only described by taking 1 write access count Counter corresponding to 1 LBA address as an example, and is not limited thereto.
Overflow counter: and counting the overflow times OverFillingnum of the write access times Counter of the LBA frequency Counter in the current time period.
Write counter: and counting the received user writing times WriteNum in the current time period.
LBA cold and hot data indicator: the indicator consists of LBA address and 1bit identification information and is responsible for indicating whether the current LBA is hot data or cold data.
In this embodiment, a flowchart for implementing statistics of historical access information is shown in fig. 3, and the specific flow is as follows:
the current write data is queried for user writes or GC writes. If the user writes, continuing the flow, and if the user writes, directly ending the flow;
acquiring write access times Counter corresponding to LBA addresses of a currently written logical block;
judging whether the current Counter number is smaller than MaxCoter, if so, the counter+ =1; otherwise, overlaillingnum+=1. Wherein MaxCounter is related to the number of bits employed by the counter, in one example, a 4bit counter is employed, then maxcounter=15;
WriteNum+ =1, and judging whether the current WriteNum meets the condition WriteNum < = MaxWriteNum, if yes, ending the flow; if the data heat index is not met, calculating a cold and hot data pre-judging threshold according to the historical access information, wherein the cold and hot data pre-judging threshold is a data heat index of the SSD when different values are taken in a preset threshold value set, and entering a cold and hot data identification flow of the current statistical period. Wherein MaxWriteNum is a preset write time threshold.
In this embodiment, when the frequency information of the written data is counted and the Counter corresponding to the current LBA reaches the maximum value, the overflow value is recorded through the overfillngnum parameter for the subsequent cold and hot data identification, so that the problem of inaccurate calibration caused by overflow of the Counter of the write access times is avoided.
Further, through the above flow, statistics of historical access information of each LBA in a certain time is completed. After the SSD finishes the cold and hot data calibration, the statistics device is informed to return to zero and enter the next statistics period, namely WriteNum=0 and OverFillingnNum=0; counter=0 for all LBAs.
In the embodiment of the present invention, the obtaining the write amplification coefficients corresponding to the SSD at different data heat indexes includes: and searching write amplification coefficients corresponding to each data heat index when different block use state coefficients are used according to a preset corresponding relation table, wherein the block use state coefficients comprise a reserved space proportion value OP and a reserved space heat data proportion value p, the reserved space proportion value OP is a proportion value of the total physical blocks in an OP reserved space in the SSD to the total physical blocks operable by a user in the SSD, the reserved space heat data proportion value p is a proportion value of the physical blocks marked as heat data in the OP reserved space to the total physical blocks in the OP reserved space, and the corresponding relation table comprises a corresponding relation between the data heat index when different block use state coefficients and the write amplification coefficients in the SSD use process. The op reserved space (Over-provisioning) refers to a hidden space which exists in the solid state disk and is controlled by the main control chip and is not operable by a user, and the hidden space is used for controlling operations of various optimization mechanisms, such as GC recycling, wear leveling and the like.
According to the historical data heat index of the written data, the invention adaptively calculates the cold and hot data discrimination proportion with minimum write amplification and the proportion of the cold and hot data physical block. In particular, calculating the proportion of the op-block of cold-hot data minimizes write amplification.
In this embodiment, when WriteNum > MaxWriteNum, a cold and hot data identification flow is entered, and the flow obtains an optimal cold and hot data judgment proportion and a current SSD optimal cold and hot data block allocation proportion according to historical access information counted in a previous time period, and simultaneously, each LBA is labeled with a cold and hot data for querying when data is written later, and the specific implementation flow is as follows:
traversing a threshold value set CounterThreshold= [ MaxCounter, maxCounter-1, …,1,0]Respectively calculating the cold and hot data pre-judging threshold value as the data heat index of SSD when different values are taken in the preset threshold value set, wherein the data heat index is the target editing block LBA of which Counter is greater than or equal to Counter threshold Target object The block proportion f of the total logic block and the corresponding write data frequency r.
Specifically, the block proportion f of the target logical block to the total logical block in the SSD may be calculated according to a first calculation model, where the first calculation model is as follows:
f = length(LBA Target object )/Total
The write data frequency r of the target block can be calculated according to a second calculation model, wherein the first calculation model is as follows:
r = [sum(Counter(LBA target object ))+OverFillingNum]/WriteNum；
Wherein LBA in the above formula Target object Refers to a target logical block, of logical blocks of an SSD, in which the number of write accesses Counter is greater than or equal to the current hot and cold data pre-determination threshold, length (LBA Target object ) Representing LBA in current SSD Target object Total represents the Total number of all LBAs in the current SSD; counter (LBA) Target object ) Representation purposeMarking logical block LBA Target object The corresponding write access number Counter, overfillngnum represents the overflow number of the Counter for counting the write access number, and WriteNum represents the received user write number.
And in the traversal process, a preset corresponding relation table is inquired according to the data heat index (comprising f and r) and the block usage state coefficient of the SSD so as to obtain a corresponding theoretical write amplification coefficient.
In this embodiment, the reserved space proportion value op is a result calculated in real time according to the current state of the SSD. Generally, as the service time of the SSD is longer, the ops gradually decrease. Specifically, the calculation mode of op is that:
op = TotalBlockNum/ UserBlockNum–1
the UserBlockNum is the number of physical blocks operable by a user in the SSD, and the TotalBlockNum is the total number of physical blocks operable by the user in the SSD and physical blocks in the OP reserved space.
Traversing p= [0, 0.1, 0.2, …, 1], looking up the write amplification coefficients WAF (p, op, r, f) corresponding to the respective data heat indicators when different blocks use state coefficients.
It should be noted that, in addition to obtaining the write amplification factor by adopting a table look-up method, the corresponding write amplification factor may also be obtained by using a preset calculation model according to a function relationship between each parameter and the write amplification factor, and the present invention is not limited in particular.
In this embodiment, after selecting the target data heat index corresponding to the minimum write amplification factor, the block proportion in the target data heat index is used as the optimal cold and hot data judgment proportion in the current statistical period; and calculating the number of physical blocks to be marked as hot data and the number of physical blocks to be marked as cold data in the current SSD according to the optimal cold and hot data judging proportion, and informing the SSD controller to adjust cold and hot data blocks according to the indication.
Further, calculating the number of physical blocks to be identified as hot data and the number of physical blocks to be identified as cold data in the current SSD according to the optimal cold-hot data decision ratio specifically includes:
Acquiring a corresponding reserved space thermal data proportion value when the write amplification coefficient is minimum, and taking the acquired reserved space thermal data proportion value as an optimal reserved space thermal data proportion value;
and estimating the number HotBuckNum of the physical blocks to be marked as hot data and the number ColdBuckNum of the physical blocks to be marked as cold data in the current SSD according to the optimal cold and hot data judging proportion and the optimal reserved space hot data proportion value.
Specifically, through the traversal process, the SSD controller obtains an optimal reserved space thermal data proportion value p corresponding to the minimum waf_min Optimum for the production of a product Optimal cold and hot data decision ratio f Optimum for the production of a product And after the optimal CounterThreshold, estimating the number HotBlockNum of the physical blocks to be marked as hot data and the number ColdBlockNum of the physical blocks to be marked as cold data in the current SSD according to the following calculation formula:
HotBlockNum=round(f optimum for the production of a product *UserBlockNum+p Optimum for the production of a product *(TotalBlockNum-UserBlockNum))；
ColdBlockNum = TotalBlockNum – HotBlockNum；
Wherein f Optimum for the production of a product For optimal cold and hot data judgment proportion, p Optimum for the production of a product For the optimal hot data proportion value of the reserved space, userBlockNum is the number of physical blocks operable by a user in the SSD, totalBlockNum is the total number of physical blocks operable by the user in the SSD and physical blocks in the OP reserved space, and round () is a rounding function.
SSD performs adjustment control of the hot and cold data block based on the above HotBuckNum and ColdBuckNum. The specific adjustment control mode comprises the following steps: and judging whether the number of the physical blocks currently marked as hot data in the SSD is equal to the estimated number of the physical blocks to be marked as hot data in the current SSD, if not, controlling the physical blocks in the SSD so that the number of the physical blocks marked as hot data in the SSD is equal to the estimated number of the physical blocks to be marked as hot data in the current SSD. For example, assuming that the number of current hot data blocks is greater than the calculated result, GC may be started for the hot data blocks of SSD, and the empty blocks after GC are marked as cold data blocks. It should be noted that, in addition to the GC mode, other implementations may be used to implement the adjustment of the hot and cold data blocks, which is not limited in this aspect of the invention, and the adjustment may be performed immediately or slowly, that is, only one block may be adjusted in a period of time.
In an alternative embodiment of the present invention, before calculating the number of physical blocks to be identified as hot data and the number of physical blocks to be identified as cold data in the current SSD according to the optimal hot-cold data decision ratio, the method further comprises the steps of: calculating a second data writing frequency of the current statistical period according to the historical cold and hot data judging proportion of the previous statistical period, and taking the historical cold and hot data judging proportion and the second data writing frequency as updated data heat indexes; searching the corresponding relation table according to the updated data heat index and the history block use state coefficient of the last statistic period to obtain a corresponding write amplification coefficient, and taking the corresponding write amplification coefficient as a history write amplification coefficient; judging whether the difference value between the historical write amplification coefficient and the minimum value of the write amplification coefficient of the current statistical period is smaller than or equal to a preset adjustment threshold value; and if the difference value is smaller than or equal to a preset adjustment threshold value, the step of calculating the number of physical blocks to be identified as hot data and the number of physical blocks to be identified as cold data in the current SSD according to the optimal cold-hot data judgment proportion is not executed.
In an optional embodiment of the present invention, before taking the block proportion identified by the target data heat index as the optimal cold and hot data decision proportion of the current statistical period, the method further includes:
calculating a second data writing frequency of the current statistical period according to the historical cold and hot data judging proportion of the previous statistical period, and taking the historical cold and hot data judging proportion and the second data writing frequency as updated data heat indexes; searching the corresponding relation table according to the updated data heat index and the history block use state coefficient of the last statistic period to obtain a corresponding write amplification coefficient, and taking the corresponding write amplification coefficient as a history write amplification coefficient; judging whether the difference value between the historical write amplification coefficient and the minimum value of the write amplification coefficient of the current statistical period is smaller than or equal to a preset adjustment threshold value; and if the difference value is smaller than or equal to a preset adjustment threshold value, taking the historical cold and hot data judgment proportion as the optimal cold and hot data judgment proportion of the current statistical period.
In this embodiment, as shown in fig. 4, the specific implementation steps for calculating the second write data frequency of the current statistical period according to the historical cold-hot data decision ratio of the previous statistical period include:
Selecting the maximum value which can be represented by a counter for counting the number of write accesses as a cold and hot data pre-judging threshold, and initializing parameters n=0, sum_f=0 and sum_r=the overflow number of the counter for counting the number of write accesses, namely, overFillingnNum;
acquiring the number of second target logic blocks with the write access times of the current statistical period being the current cold and hot data pre-judging threshold value, and calculating the data heat index of the current SSD to obtain a block proportion f_n and a corresponding write data frequency r_n;
and updating sum_f and sum_r according to the block proportion f_n and the corresponding write data frequency r_n, wherein:
sum_f = sum_f+f_n
sum_r = sum_r+r_n；
judging whether the updated sum_f is smaller than the historical cold and hot data judgment proportion;
if yes, updating a cold and hot data prejudging threshold value and n, wherein:
cold-hot data prejudgement threshold = cold-hot data prejudgement threshold-1, n = n +1;
calculating the block proportion f_n and the corresponding write data frequency r_n of the current cycle period according to the updated cold and hot data pre-judging threshold value until the updated sum_f is greater than or equal to the historical cold and hot data judging proportion;
when the updated sum_f is greater than or equal to the historical cold and hot data judgment proportion, calculating a second data writing frequency r of the current statistical period according to sum_f, sum_r, f_n and r_n of the current cyclic period final The formula is as follows:
。
in this embodiment, the SSD controller calculates the write data frequency r of the current cycle corresponding to f_pre using the history hot and cold data decision ratio f_pre of the previous cycle final The write amplification factor waf_pre in this configuration is acquired.
Judging whether the following inequality is satisfied, wherein WAF_pre-WAF_min < = WAF_threshold;
if so, the f_pre parameter of the previous period is still adopted as the optimal cold and hot data judgment proportion of the current statistical period, and at the moment, the adjustment operation of the cold and hot data block is not performed any more. Wherein WAF_min is the minimum write amplification factor obtained above, and WAF_threshold is a preset adjustment threshold.
By the method, when the write amplification factor change caused by the adjustment of the predicted optimal cold and hot data judgment proportion is not large, the adjustment of f_pre to f is not needed Optimum for the production of a product The original cold and hot data judgment proportion is kept to reduce the frequency of cold and hot data block change, and the frequent change of the cold and hot data block of the SSD is relieved, so that the GC frequency is reduced, and the system robustness is improved.
Further, in the embodiment of the present invention, when the difference between the minimum value of the historical write amplification factor and the write amplification factor of the current statistical period is smaller than or equal to a preset adjustment threshold, the method may further use the historical cold and hot data decision ratio as an optimal cold and hot data decision ratio, and perform cold and hot data calibration on a logic block in the SSD according to the optimal cold and hot data decision ratio.
Specifically, in one embodiment, calibrating the cold and hot data of the logic block in the SSD according to the optimal cold and hot data decision ratio may specifically include steps not shown in the following figures:
s20, calculating the number of logic blocks prejudged as hot data logic blocks in the SSD according to the optimal cold and hot data judgment proportion to obtain a first number TotalHotLBANum, wherein the specific formula is as follows:
TotalHotLBANum = round(f optimum for the production of a product * PagesPerSSD)
Where PagesPerSSD represents the total number of SSDpages visible to the user, i.e., the total number of LBAs the user can write.
S21, selecting the maximum value in the preset threshold value set as a cold and hot data pre-judging threshold value, namely setting the current CounterThreshold value as the MaxCoulter in the preset threshold value set.
S22, acquiring the number of first hot data logic blocks to be selected, wherein the write access times of the first hot data logic blocks in the logic blocks of the SSD in the current statistical period are greater than the current hot and cold data pre-judging threshold value, and obtaining a second number LBANum1; the first hot data logic block to be selected is a logic block with the write access times in the current statistical period being larger than the current hot and cold data pre-judging threshold value in the logic blocks of the SSD.
If the second number LBANum1 is less than or equal to the first number TotalHotLBANum, step S23 is executed, and if the second number LBANum1 is greater than the corresponding first number TotalHotLBANum, step S28 is executed;
S23, marking a first to-be-selected hot data logic block as a hot data logic block, and updating a first quantity according to a preset first updating model, wherein the first updating model is as follows: updated first quantity = current first quantity-second quantity;
s24, obtaining the number of second hot data logic blocks to be selected, wherein the write access times of the second hot data logic blocks in the logic blocks of the SSD in the current statistical period are equal to the current hot and cold data pre-judging threshold value, and obtaining a third number LBANum2. Specifically, the number of LBAs with the current Counter value being Counter threshold is obtained and recorded as LBANum2, and the second hot data logic block to be selected is a logic block with the write access frequency equal to the current hot and cold data pre-judging threshold value in the current statistical period in the logic blocks of the SSD.
Step S25 is performed when the third number lban 2 is greater than the updated first number totalhotlban, and step S27 is performed when the third number lban 2 is less than or equal to the updated first number totalhotlban;
s25, selecting a corresponding number of logic blocks from the second hot data logic blocks to be selected according to a preset selection rule, wherein the corresponding number of logic blocks are marked as hot data logic blocks according to the current first number, and the rest of logic blocks in the second hot data logic blocks to be selected are marked as cold data logic blocks, and the selection rule is to preferentially select the logic blocks with the last statistic period marked as hot data in the second hot data logic blocks to be selected.
S26, marking all logic blocks with the write access times smaller than the current cold and hot data pre-judging threshold value in the logic blocks of the SSD as cold data.
S27, marking all the second hot data logic blocks to be selected as hot data logic blocks, updating the first quantity according to a preset second updating model, updating a hot and cold data pre-judging threshold according to a preset third updating model, and returning to the step S24;
wherein the second update model is as follows:
updated first quantity = current first quantity-third quantity;
the third update model is as follows:
updated cold data prejudgement threshold = current cold data prejudgement threshold-1.
S28, judging whether the current cold and hot data pre-judging threshold value is smaller than the maximum value which can be represented by a counter for counting the number of write accesses in the history access information;
if yes, executing the step S29-1, otherwise, executing the step S29-2;
s29-1, updating a cold and hot data pre-judging threshold according to a preset fourth updating model, and returning to the step S22; wherein the fourth update model is as follows: updated hot and cold data prejudgment threshold = current hot and cold data prejudgment threshold +1;
s29-2, randomly selecting a corresponding number of logic blocks from the first hot data logic blocks to be selected according to the current first number to be marked as hot data logic blocks. At this point, other logical blocks in the SSD may be marked as cold data.
When the logic blocks in the SSD are marked with hot data according to the optimal hot and cold data judgment proportion of the current statistical period, the logic blocks larger than the CounterThreshold are required to be ensured to be marked with hot data, and when part of the logic blocks equal to the CounterThreshold are marked with hot data, the logic block with the last statistical period marked with hot data can be preferentially selected from the logic blocks with hot data to be selected which meet the preset condition, so that the hot data of the last period can be prevented from being marked with cold data in the current period and being migrated to the cold data block as far as possible, and the migration quantity is reduced.
Specifically, in another embodiment, calibrating the cold and hot data of the logic block in the SSD according to the optimal cold and hot data decision ratio may specifically include steps not shown in the following figures:
s30, calculating the number of logic blocks in SSD, which are pre-judged as hot data logic blocks, according to the optimal cold and hot data judgment proportion to obtain a first number TotalHotLBANum;
s31, selecting the maximum value in a preset threshold value set as a cold and hot data preset threshold value;
s32, acquiring the number of first hot data logic blocks to be selected, wherein the write access times of the first hot data logic blocks in the logic blocks of the SSD in the current statistical period are greater than the current hot and cold data pre-judging threshold value, and obtaining a second number LBANum1; the first hot data logic block to be selected is a logic block with the write access times in the current statistical period being larger than the current hot and cold data pre-judging threshold value in the logic blocks of the SSD.
If the second number LBANum1 is less than or equal to the first number TotalHotLBANum, step S33 is executed, and if the second number LBANum1 is greater than the corresponding first number TotalHotLBANum, step S40 is executed;
s33, marking a first hot data logic block to be selected as a hot data logic block, and updating a first quantity according to a preset first updating model, wherein the first updating model is as follows: updated first quantity = current first quantity-second quantity;
s34, obtaining the number of second hot data logic blocks to be selected, wherein the write access times of the second hot data logic blocks in the logic blocks of the SSD in the current statistical period are equal to the current hot and cold data pre-judging threshold value, and obtaining a third number LBANum2; the second hot data logic block to be selected is a logic block with the write access times in the current statistical period equal to the current hot and cold data pre-judging threshold value in the logic blocks of the SSD.
Step S35 is performed when the third number lban 2 is greater than the updated first number totalhotlban, and step S40 is performed when the third number lban 2 is less than or equal to the updated first number totalhotlban;
s35, counting the number of logic blocks marked as hot data in the last counting period in the logic blocks with the write access times equal to the current hot and cold data pre-judging threshold value to obtain a fourth number HotLBANum;
Judging whether the fourth number of HotLBANUs is greater than or equal to the updated first number of TotalHotLBANUs, if so, executing the step S36, otherwise, executing the step S37;
s36, randomly selecting a corresponding number of logic blocks from the logic blocks marked as hot data in the last statistic period in the step S35 according to the updated first number, marking the corresponding number of logic blocks as hot data logic blocks, and marking the rest logic blocks in the second hot data logic blocks to be selected as cold data logic blocks;
s37, marking all the logic blocks marked as hot data in the last statistic period in the step S35 as hot data logic blocks, randomly selecting a corresponding number of logic blocks from the second hot data logic blocks to be selected according to the updated difference value between the current first number and the fourth number to be selected as hot data logic blocks, and marking the rest logic blocks in the second hot data logic blocks to be selected as cold data logic blocks;
s38, marking all logic blocks with the write access times smaller than the current cold and hot data pre-judging threshold value in the logic blocks of the SSD as cold data.
S39, when the third quantity LBANum2 is smaller than or equal to the updated first quantity TotalHotLBANum, the second to-be-selected hot data logic blocks are all marked as hot data logic blocks, the first quantity is updated according to a preset second updating model, a hot and cold data pre-judging threshold is updated according to a preset third updating model, and the step S34 is returned;
Wherein the second update model is as follows:
updated first quantity = current first quantity-second quantity;
the third update model is as follows:
updated cold data prejudgement threshold = current cold data prejudgement threshold-1.
S40, judging whether the current cold and hot data pre-judging threshold value is smaller than the maximum value which can be represented by a counter for counting the number of write accesses in the history access information;
if yes, executing the step S41-1, otherwise, executing the step S41-2;
s41-1, updating a cold and hot data pre-judging threshold according to a preset fourth updating model, and returning to the step S22; wherein the fourth update model is as follows: updated hot and cold data prejudgment threshold = current hot and cold data prejudgment threshold +1;
s41-2, randomly selecting a corresponding number of logic blocks from the first hot data logic blocks to be selected according to the current first number to be marked as hot data logic blocks. At this point, other logical blocks in the SSD may be marked as cold data.
By the method, the fact that hot data in the previous period are marked as cold data in the current period and are migrated to the cold data block can be avoided as far as possible, and migration quantity is reduced.
According to the cold and hot data calibration method provided by the invention, the cold and hot data is accurately identified and marked according to the historical data heat index and the calculated cold and hot data proportion calibration data, and the write amplification factor in the SSD use process is effectively reduced. In particular, among the plurality of LBA data under the same Counter value, the cold and hot properties are not changed as much as possible so as to reduce the migration data amount.
For the purposes of simplicity of explanation, the methodologies are shown and described as a series of acts, it is to be understood and appreciated by one of ordinary skill in the art that the methodologies are not limited by the order of acts, as some acts may, in accordance with the methodologies, take place in other order or concurrently. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred embodiments, and that the acts are not necessarily required by the embodiments of the invention.
In addition, the embodiment of the invention also provides a cold and hot data calibration device, which comprises a functional module for realizing the cold and hot data calibration method. FIG. 5 schematically shows a schematic structural diagram of a cold and hot data calibration device according to an embodiment of the present invention. Referring to fig. 5, the apparatus for calibrating cold and hot data according to the embodiment of the present invention specifically includes a first obtaining module 501, a calculating module 502, a second obtaining module 503, and a calibrating module 504, where:
the first obtaining module 501 is configured to obtain a data heat index of the SSD when the cold and hot data pre-judging threshold counted in the previous counting period is different values in the preset threshold value set;
The calculating module 502 is configured to calculate write amplification coefficients corresponding to the SSD when the thermal indexes of the data are different, and select a target thermal index corresponding to the data when the write amplification coefficient has the smallest value;
the second obtaining module 503 is configured to obtain a cold and hot data pre-judging threshold corresponding to the target data heat index, so as to obtain an optimal cold and hot data pre-judging threshold;
and the calibration module 504 is used for calibrating the cold and hot data of the logic blocks in the SSD according to the optimal cold and hot data prejudgment threshold value.
For the device embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and the relevant points refer to the part of the description of the method embodiment, and have corresponding technical effects.
In addition, the embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to realize the steps of the cold and hot data calibration method.
In this embodiment, the cold and hot data calibration method may be stored in a computer readable storage medium if implemented in the form of a software functional unit and sold or used as a separate product. Based on such understanding, the present invention may implement all or part of the flow of the method of the above embodiment, or may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of each of the method embodiments described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth. It should be noted that the computer readable medium contains content that can be appropriately scaled according to the requirements of jurisdictions in which such content is subject to legislation and patent practice, such as in certain jurisdictions in which such content is subject to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.
In addition, the embodiment of the invention also provides electronic equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the steps of the cold and hot data calibration method when executing the computer program. Such as steps S11-S14 shown in fig. 1. Alternatively, the processor may implement the functions of each module/unit in the embodiments of the apparatus for calibrating cold and hot data, when executing the computer program, for example, the first obtaining module 501, the calculating module 502, the second obtaining module 503, and the calibrating module 504 shown in fig. 5.
In a particular embodiment, the electronic device may be an SSD device.
According to the cold and hot data calibration method, the device, the storage medium and the electronic equipment, based on the fact that cold and hot data pre-judging thresholds counted in the previous statistic period are data heat indexes of the SSD when different values are taken in the preset threshold value set, according to the corresponding write amplification coefficients of the SSD when the different data heat indexes are taken, the target data heat index corresponding to the time when the write amplification coefficients are the smallest is selected, the cold and hot data pre-judging threshold corresponding to the time when the target data heat index is obtained is used as the optimal cold and hot data pre-judging threshold, cold and hot data calibration is carried out on a logic block in the SSD according to the obtained optimal cold and hot data pre-judging threshold.
Furthermore, those skilled in the art will appreciate that while some embodiments herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, any of the claimed embodiments can be used in any combination.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (9)

1. The cold and hot data calibration method is characterized by comprising the following steps of:
the method for acquiring the data heat index of the SSD when the cold and hot data pre-judging threshold counted in the last counting period is different values in the preset threshold value set comprises the following steps: acquiring historical access information of each logic block in the SSD in the last statistics period, wherein the historical access information comprises write access times, overflow times of a counter for counting the write access times and received user write times; when the user writing times are larger than a preset writing times threshold, calculating a cold and hot data pre-judging threshold according to the history access information to be a data heat index of SSD when different values in a preset threshold value set are obtained; the data heat index comprises the block proportion of a target logic block in the SSD to the total logic block and the write data frequency of the target block, wherein the target block is a logic block with write access times larger than or equal to the current cold and hot data pre-judging threshold value in the logic block of the SSD;
Calculating corresponding write amplification coefficients of the SSD when different data heat indexes are obtained, and selecting a target data heat index corresponding to the time when the value of the write amplification coefficient is minimum; the calculating the write amplification coefficients corresponding to the SSD when the heat indexes of the data are different comprises the following steps: searching write amplification coefficients corresponding to each data heat index when using state coefficients of different blocks according to a preset corresponding relation table, wherein the block using state coefficients comprise a reserved space proportion value and a reserved space heat data proportion value, the reserved space proportion value is a proportion value of total physical blocks in an OP reserved space in an SSD to total physical blocks operable by a user in the SSD, the reserved space heat data proportion value is a proportion value of physical blocks marked as heat data in the OP reserved space to the total physical blocks in the OP reserved space, and the corresponding relation table comprises a corresponding relation between the data heat index when using state coefficients of different blocks and the write amplification coefficients in the SSD using process;
acquiring a cold and hot data pre-judging threshold corresponding to the target data heat index to obtain an optimal cold and hot data pre-judging threshold;
and calibrating the cold and hot data of the logic block in the SSD according to the optimal cold and hot data prejudging threshold value.
2. The method of claim 1, wherein calculating the data heat index of the SSD when the cold and hot data pre-determined threshold is different values in the set of pre-set threshold values according to the historical access information comprises:
according to a first calculation model, calculating the block proportion f of a target logic block to a total logic block in the SSD, wherein the first calculation model is as follows:
f＝length(LBA target object )/Total
According to the second calculation model, calculating the write data frequency r of the target block, wherein the first calculation model is as follows:
r＝[sum(Counter(LBA target object ))+OverFillingNum]/WriteNum
Wherein LBA Target object Refers to a target logical block, of logical blocks of an SSD, in which the number of write accesses is greater than or equal to the current hot and cold data pre-determination threshold, length (LBA Target object ) Representing LBA in current SSD Target object Total represents the Total number of all LBAs in the current SSD; counter (LBA) Target object ) Representing target logical block LBAs Target object The corresponding write access times, overfillgnum, represents the overflow times of the counter for counting the write access times, and WriteNum represents the received user write times.
3. The method of claim 1, wherein after selecting the target data heat index corresponding to the minimum write amplification factor, the method further comprises:
Taking the block proportion in the target data heat index as the optimal cold and hot data judgment proportion of the current statistical period;
and calculating the number of physical blocks to be identified as hot data and the number of physical blocks to be identified as cold data in the current SSD according to the optimal cold and hot data judging proportion.
4. The method of claim 3, wherein calculating the number of physical blocks to be identified as hot data and the number of physical blocks to be identified as cold data in the current SSD based on the optimal hot-cold data decision ratio comprises:
acquiring a corresponding thermal data proportion value of the reserved space when the write amplification coefficient is minimum, and acquiring an optimal thermal data proportion value of the reserved space;
and estimating the number HotBuckNum of the physical blocks to be marked as hot data and the number ColdBuckNum of the physical blocks to be marked as cold data in the current SSD according to the optimal cold-hot data judging proportion and the optimal reserved space hot data proportion value, wherein the calculation formula is as follows:
HotBlockNum＝round(f optimum for the production of a product *UserBlockNum+p Optimum for the production of a product *(TotalBlockNum-
UserBlockNum))；
ColdBlockNum＝TotalBlockNum–HotBlockNum；
Wherein f Optimum for the production of a product For optimal cold and hot data judgment proportion, p Optimum for the production of a product For the optimal reserved space thermal data proportion value, userBlockNum is the number of physical blocks operable by a user in the SSD, totalBlockNum is the total number of physical blocks operable by the user in the SSD and physical blocks in the OP reserved space, and round () is a rounding function;
And judging whether the number of the physical blocks currently marked as hot data in the SSD is equal to the estimated number of the physical blocks to be marked as hot data in the current SSD, if not, controlling the physical blocks in the SSD so that the number of the physical blocks marked as hot data in the SSD is equal to the estimated number of the physical blocks to be marked as hot data in the current SSD.
5. The method of claim 3, wherein prior to calculating the number of physical blocks to be identified as hot data and the number of physical blocks to be identified as cold data in the current SSD based on the optimal hot-cold data decision ratio, the method further comprises:
calculating a second data writing frequency of the current statistical period according to the historical cold and hot data judging proportion of the previous statistical period, and taking the historical cold and hot data judging proportion and the second data writing frequency as updated data heat indexes;
searching the corresponding relation table according to the updated data heat index and the history block use state coefficient of the last statistic period to obtain a corresponding write amplification coefficient, and taking the corresponding write amplification coefficient as a history write amplification coefficient;
judging whether the difference value between the historical write amplification coefficient and the minimum value of the write amplification coefficient of the current statistical period is smaller than or equal to a preset adjustment threshold value;
And if the difference value is smaller than or equal to a preset adjustment threshold value, the step of calculating the number of physical blocks to be identified as hot data and the number of physical blocks to be identified as cold data in the current SSD according to the optimal cold-hot data judgment proportion is not executed.
6. The method of any of claims 1-5, wherein calibrating the hot and cold data for the logical blocks in the SSD according to the optimal hot and cold data pre-determined threshold comprises:
and marking a logic block with the write access frequency of the logic block of the SSD being greater than or equal to the optimal cold and hot data pre-judging threshold value in the current statistical period as a hot data logic block, and marking a logic block with the write access frequency of the logic block of the SSD being less than the optimal cold and hot data pre-judging threshold value in the current statistical period as a cold data logic block.
7. A cold and hot data calibration device, characterized in that the device comprises a functional module for implementing the cold and hot data calibration method according to any one of claims 1-6.
8. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any of claims 1-6.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method according to any one of claims 1-6 when the computer program is executed.
CN202310898643.1A 2023-07-21 2023-07-21 Cold and hot data calibration method and device, storage medium and electronic equipment Active CN116820351B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310898643.1A CN116820351B (en) 2023-07-21 2023-07-21 Cold and hot data calibration method and device, storage medium and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310898643.1A CN116820351B (en) 2023-07-21 2023-07-21 Cold and hot data calibration method and device, storage medium and electronic equipment

Publications (2)

Publication Number Publication Date
CN116820351A CN116820351A (en) 2023-09-29
CN116820351B true CN116820351B (en) 2024-04-09

Family

ID=88139222

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310898643.1A Active CN116820351B (en) 2023-07-21 2023-07-21 Cold and hot data calibration method and device, storage medium and electronic equipment

Country Status (1)

Country Link
CN (1) CN116820351B (en)

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102799534A (en) * 2012-07-18 2012-11-28 上海宝存信息科技有限公司 Storage system and method based on solid state medium and cold-hot data identification method
WO2017092480A1 (en) * 2015-12-04 2017-06-08 华为技术有限公司 Data migration method and device
US9747202B1 (en) * 2013-03-14 2017-08-29 Sandisk Technologies Llc Storage module and method for identifying hot and cold data
CN108121666A (en) * 2017-11-16 2018-06-05 成都华为技术有限公司 The rubbish recovering method and device of a kind of flash memory
CN108984124A (en) * 2018-07-13 2018-12-11 深圳忆联信息系统有限公司 Reduce the method, apparatus, computer equipment and storage medium for writing amplification
CN109448770A (en) * 2018-09-21 2019-03-08 华中科技大学 A kind of optimization method of cross bar structure resistance-variable storing device
CN109542358A (en) * 2018-12-03 2019-03-29 浪潮电子信息产业股份有限公司 A kind of cold and hot data separation method of solid state hard disk, device and equipment
CN113885801A (en) * 2021-09-30 2022-01-04 中国工商银行股份有限公司 Memory data processing method and device
CN114296644A (en) * 2021-12-17 2022-04-08 合肥大唐存储科技有限公司 Data management method and device for solid state disk
CN115629708A (en) * 2022-10-14 2023-01-20 苏州浪潮智能科技有限公司 Method, device, terminal and medium for monitoring cold and hot data in storage system
CN116126253A (en) * 2023-04-19 2023-05-16 北京得瑞领新科技有限公司 Flow control method, device, storage medium and equipment for SSD (solid state disk)
CN116225334A (en) * 2023-03-10 2023-06-06 郑州云海信息技术有限公司 Cold and hot data storage method, device and medium
CN116303119A (en) * 2023-05-19 2023-06-23 珠海妙存科技有限公司 Method, system and storage medium for identifying cold and hot data

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103631538B (en) * 2013-12-05 2017-04-05 华为技术有限公司 Cold and hot data identification threshold value calculation, device and system
US10114557B2 (en) * 2014-05-30 2018-10-30 Sandisk Technologies Llc Identification of hot regions to enhance performance and endurance of a non-volatile storage device
CN107506135B (en) * 2016-06-14 2022-05-06 杭州海康威视数字技术股份有限公司 Data processing method, device and system

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102799534A (en) * 2012-07-18 2012-11-28 上海宝存信息科技有限公司 Storage system and method based on solid state medium and cold-hot data identification method
US9747202B1 (en) * 2013-03-14 2017-08-29 Sandisk Technologies Llc Storage module and method for identifying hot and cold data
WO2017092480A1 (en) * 2015-12-04 2017-06-08 华为技术有限公司 Data migration method and device
CN108121666A (en) * 2017-11-16 2018-06-05 成都华为技术有限公司 The rubbish recovering method and device of a kind of flash memory
CN108984124A (en) * 2018-07-13 2018-12-11 深圳忆联信息系统有限公司 Reduce the method, apparatus, computer equipment and storage medium for writing amplification
CN109448770A (en) * 2018-09-21 2019-03-08 华中科技大学 A kind of optimization method of cross bar structure resistance-variable storing device
CN109542358A (en) * 2018-12-03 2019-03-29 浪潮电子信息产业股份有限公司 A kind of cold and hot data separation method of solid state hard disk, device and equipment
CN113885801A (en) * 2021-09-30 2022-01-04 中国工商银行股份有限公司 Memory data processing method and device
CN114296644A (en) * 2021-12-17 2022-04-08 合肥大唐存储科技有限公司 Data management method and device for solid state disk
CN115629708A (en) * 2022-10-14 2023-01-20 苏州浪潮智能科技有限公司 Method, device, terminal and medium for monitoring cold and hot data in storage system
CN116225334A (en) * 2023-03-10 2023-06-06 郑州云海信息技术有限公司 Cold and hot data storage method, device and medium
CN116126253A (en) * 2023-04-19 2023-05-16 北京得瑞领新科技有限公司 Flow control method, device, storage medium and equipment for SSD (solid state disk)
CN116303119A (en) * 2023-05-19 2023-06-23 珠海妙存科技有限公司 Method, system and storage medium for identifying cold and hot data

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
A High Performance and Energy-Efficient Cold Data Eviction Algorithm for 3D-TSV Hybrid ReRAM/MLC NAND SSD;Chao Sun;Kousuke Miyaji;Koh Johguchi;Ken Takeuchi;IEEE transactions on circuits and systems. I, Regular papers;20141231;第61卷(第2期);第382-392页 *
一种多路Flash存储结构的热数据判断方法;钟诚;岳丽华;韦鹏;计算机工程;20090620(第12期);第256-259页 *
固态盘热数据识别算法研究;黄彬;杜晨杰;浙江万里学院学报;20171115(第06期);第82-87页 *
大容量闪存的层次型热数据识别框架;黄滨;俞建新;计算机工程;20081120(第22期);第66-68页 *

Also Published As

Publication number Publication date
CN116820351A (en) 2023-09-29

Similar Documents

Publication Publication Date Title
US11693568B2 (en) Workload-adaptive overprovisioning in solid state storage drive arrays
KR102388718B1 (en) Cache over-provisioning in a data storage device
US10534716B2 (en) Limiting access operations in a data storage device
US9021185B2 (en) Memory controller and methods for enhancing write performance of a flash device
US8341350B2 (en) Analyzing sub-LUN granularity for dynamic storage tiering
US8386708B2 (en) Method for coupling sub-LUN load measuring metadata size to storage tier utilization in dynamic storage tiering
US20120054414A1 (en) Solid state drive and data control method thereof
EP3588259A1 (en) Garbage collection method for storage media, storage medium, and program product
US20170235486A1 (en) Enhanced multi-streaming though statistical analysis
KR20110050443A (en) Managing storage of cached content
JP2014110058A (en) Host read command return reordering based on time estimation of flash read command completion
JP2011192260A (en) Semiconductor storage device
US20120185638A1 (en) Method and system for cache endurance management
WO2016088234A1 (en) Storage device which extends useful lifetime of non-volatile semiconductor memory of different characteristics
CN108139872A (en) A kind of buffer memory management method, cache controller and computer system
CN116126253B (en) Flow control method, device, storage medium and equipment for SSD (solid state disk)
CN114020573A (en) Decision level prediction method, storage medium and SSD device
CN116820351B (en) Cold and hot data calibration method and device, storage medium and electronic equipment
CN116627355B (en) Cold and hot data calibration method and device and electronic equipment
JP5687649B2 (en) Method for controlling semiconductor memory device
US8996786B2 (en) Nonvolatile memory system and block management method
US20200097215A1 (en) Adaptive solid state device management based on data expiration time
US20230067208A1 (en) Workload-Adaptive Overprovisioning in Solid State Storage Drive Arrays
CN114385092A (en) Erasing method of solid state disk flash memory array and solid state disk main control chip
CN112130763B (en) Solid state disk wear balancing method based on temperature monitoring

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant